Community Data Science Workshops (Spring 2016)/Day 3 Projects

Building and visualizing datasets using data.seattle.gov and Google apps

In this project, we will explore a few ways to analyze and visualize data from data.seattle.gov. We will examine trends in the construction of houses, townhomes, and condos in Seattle over the last five years, in order to determine the true extent of the current "construction boom".

We'll download data from two APIs (Socrata open data platform and Google Maps)
We'll analyze trends over time in our dataset
We'll visualize our data in graphs and heatmaps

Topics we will cover

Writing and testing complex API queries
Reading and writing CSV and JSON files
Filtering and aggregating data
Combining data from multiple APIs
Graphing and mapping data

Part 1: Downloads

If you are confused by anything today, go back and refresh your memory with the Day 0 setup and tutorial and Day 0 tutorial

Step 1

Download today's scripts: Python scripts

Step 2

Unzip the folder, navigate to it in your terminal, and add some data cleanup code to the

download_building_permit_data.py

script before running it:

In that file, just before the

writer.writerow

line, add this block:

if 'latitude' not in x or 'longitude' not in x:
    continue

This will clean up some issues with missing lat/long coordinates.

Step 3

Now, get the dataset by running the script you just fixed:

python download_building_permit_data.py

Now open up the CSV output file to verify that you got all the data you asked for.

You can view the full set of permit applications at: https://data.seattle.gov/Permitting/Building-Permits-Current/mags-97de

Challenge question: how could we change our API query to download the applicant names for all COMMERCIAL building permits issued within the past year?

Part 2: charting new construction by month

Question: Has the rate of residential construction increased since 2010?

Now we want to learn whether there is, in fact, a housing boom in Seattle. We'll do this by counting how many new permits are issued each month, and then plot these on a graph.

Run the script

residential_permits_by_month.py

Now open up the CSV output file and check your data. Does this look right to you?

Challenge question: how could we separate out single family homes from apartments, and count/plot them separately?

Part 3: charting new construction by neighborhood

Question: which Seattle neighborhoods have had the most multifamily residential construction (apartments and townhomes) since 2010?

Now we want to learn where all this new construction is happening. We'll do this by sending the address for each MULTIFAMILY permit to the Google Geolocation API, which will return the neighborhood where that address is located.

Run the script

multifamily_permits_by_neighborhood.py

Now open up the CSV output file and check your data. Does this look right to you?

Challenge question: In which Seattle neighborhood is the cost of new apartment construction projects highest, on average?

Part 4: Mapping new home construction in Seattle

Question: what locations have experienced the highest density of new construction since 2010?

Now we will try to get an even more detailed picture of where this construction is occuring, using Google Fusion tables, a powerful visualization application that makes it easy to plot data on a map.

Go here and follow the steps to upload your new_all_2010-2015.csv file.

Create a point map and heatmap.
Experiment with filtering and weighting the points on your map.

Example map: https://www.google.com/fusiontables/data?docid=1gm0wVqnK7zQ7hgp5bsKoBIetMZ75jo-VMs5noDoJ#map:id=3

Going further

Try to answer these additional questions that draw on the data and methods we're learning today. Ask a mentor if you get stuck!

Which developer has spent the most on new construction in Seattle since 2010?
How many townhouses have been constructed in Seattle since 2010?
Where in Seattle are the most commercial buildings being constructed in 2015?
How does the rate of residential construction in Seattle from 2010-2015 compare to the previous 5 years?