Community Data Science Workshops (Spring 2015)/Day 3 Projects/Civic data: Difference between revisions

Latest revision as of 01:18, 17 September 2015

Building and visualizing datasets using data.seattle.gov and Google apps[edit]

In this project, we will explore a few ways to gather data from data.seattle.gov using the Socrata API, the Google Maps API, and Google Fusion tables.

Today's dataset: Click here to view the Seattle Building Permits database

Today's scripts: Click here to download the today's scripts

Goals[edit]

Collect data on building permits in Seattle, by neighborhood and by year awarded.
Combine those data with neighborhood data from the Google Maps API
Download the results in a CSV file
Upload results to Google's Fusion Table visualization engine

Topics we will cover[edit]

creating API queries and filtering and grouping the results
combining results from multiple APIs
downloading API results as CSV files
building visualizations of timeseries and geolocated data

Questions we will (attempt to) answer with our data[edit]

which Seattle neighborhoods have had the most multifamily residential construction (apartments and townhomes) since 2010?
how much has the rate of apartment construction increased since 2010?
where are the most expensive construction projects currently being built?

Resources[edit]

If you are confused by anything today, go back and refresh your memory with the Day 0 setup and tutorial and Day 0 tutorial

Sample API queries

Building permit data: https://data.seattle.gov/resource/mags-97de.json?$where=issue_date%20IS%20NOT%20NULL%20&action_type=NEW&category=MULTIFAMILY&$limit=100
Google maps location (using street address): http://maps.googleapis.com/maps/api/geocode/json?address=5601+22ND+AVE+NW+Seattle+WA
Google maps location (using lat/long): http://maps.googleapis.com/maps/api/geocode/json?latlng=47.66979666%2C-122.38570052

Sample charts/maps

Google Fusion table map of permits 2010-2015 [1]

Help resources and inspiration

About Google maps API: https://developers.google.com/maps/documentation/geocoding/#ReverseGeocoding
About Google Fusion tables: https://support.google.com/fusiontables/answer/2571232
Google Fusion Tables mapmaking tutorial: https://support.google.com/fusiontables/answer/2527132?hl=en&topic=2573107&ctx=topic
API sandbox tool: https://www.hurl.it
API-powered app: http://web6.seattle.gov/mnm/
Socrata API help resources: http://dev.socrata.com/consumers/getting-started.html

filtering results: http://dev.socrata.com/docs/filtering.html
dealing with timestamps: http://dev.socrata.com/docs/datatypes/timestamp.html
writing API queries: http://dev.socrata.com/docs/queries.html

Other data.seattle.gov datasets with neighborhood, timeseries, and/or location data

Instructional videos

Other Socrata sites that use this API

@@ Line 1: / Line 1: @@
-[[File:Socrata-square-color.png|right|250px]]
+[[File:Seattle_building_cost_heatmap.png|right|250px]]
-[[File:SeattleGovLogoHome.png|right|250px]]
+[[File:Socrata-square-color.png|right|150px]]
+[[File:SeattleGovLogoHome.png|right|150px]]
 __NOTOC__
 == Building and visualizing datasets using data.seattle.gov and Google apps ==
 In this project, we will explore a few ways to gather data from [https://data.seattle.gov data.seattle.gov] using the [[w:Socrata|Socrata]] API, the Google Maps API, and Google Fusion tables.
+;Today's dataset: [https://data.seattle.gov/Permitting/Building-Permits-Current/mags-97de Click here to view the Seattle Building Permits database]
+;Today's scripts: [http://jtmorgan.net/cdsw/may9cdsw2.zip Click here to download the today's scripts]
 === Goals ===
@@ Line 11: / Line 18: @@
 * Combine those data with neighborhood data from the Google Maps API
 * Download the results in a CSV file
-* Upload results to Google's Fu
+* Upload results to Google's [https://support.google.com/fusiontables/answer/2571232 Fusion Table] visualization engine
-=== Download the Day 3 Socrata project ===
-<font size="+1">[http://jtmorgan.net/cdsw/may9scripts.zip Click here to download the Socrata scripts]</font>
-If you are confused by these steps, go back and refresh your memory with the [[Community Data Science Workshops (Spring 2015)/Day 0 setup and tutorial|Day 0 setup and tutorial]] and [[Community Data Science Workshops (Spring 2015)/Day 0 tutorial|Day 0 tutorial]]
+=== Topics we will cover ===
-(Estimated time: 10 minutes)
+* creating API queries and filtering and grouping the results
+* combining results from multiple APIs
+* downloading API results as CSV files
+* building visualizations of timeseries and geolocated data
-=== Topics to cover ===
-* explain Socrata open data platform, exists on other government websites
+=== Questions we will (attempt to) answer with our data ===
-* navigate to [http://en.wikipedia.org/w/api.php api page] and show the documentation, point out examples
+# which Seattle neighborhoods have had the most multifamily residential construction (apartments and townhomes) since 2010?
-* introduce the [https://www.hurl.it API sandbox] as a tool for building queries
+# how much has the rate of apartment construction increased since 2010?
+# where are the most expensive construction projects currently being built?
-=== Example questions ===
-* What Seattle neighborhood has the most art galleries? - [http://jtmorgan.net/cdsw/neighborhood_culture1.py neighborhood_culture1.py]
-* What Seattle neighborhood has the most square feet devoted to arts & culture?  - [http://jtmorgan.net/cdsw/neighborhood_culture2.py neighborhood_culture2.py]
-* Which trail gets more bike traffic per month--the [[w:Burke-Gilman_Trail|Burke Gilman]] or the [[w:Mountains_to_Sound_Greenway|Mountain to Sound Trail]]?  - [http://jtmorgan.net/cdsw/traffic_counter1.py traffic_counter1.py], [http://jtmorgan.net/cdsw/traffic_counter2.py traffic_counter2.py]
-* Do people use the trails less when it's cold? [http://jtmorgan.net/cdsw/monthly_weather.py monthly_weather.py]
-* Does the Burke-Gilman get more bike or pedestrian traffic? [http://jtmorgan.net/cdsw/bike_and_peds1.py bike_and_peds1.py]
-* What is the primary commute direction on the Burke Gilman? [http://jtmorgan.net/cdsw/bike_and_peds1.py bike_and_peds2.py]
-;Other example questions
+=== Other questions you can answer with these data ===
-* What day of the week does the Burke Gilman have the most total traffic?
+# which Seattle developer has built the most apartments and townhomes since 2010?
-* What day has the most pedestrian traffic?
+# where in Seattle are the most single-family homes being constructed?
-* What ''time of day'' has the most southbound traffic, on average?
-* How many shoplifting calls has SPD responded to this month so far?
 === Resources ===
+''If you are confused by anything today, go back and refresh your memory with the [[Community Data Science Workshops (Spring 2015)/Day 0 setup and tutorial|Day 0 setup and tutorial]] and [[Community Data Science Workshops (Spring 2015)/Day 0 tutorial|Day 0 tutorial]]''
+;Sample API queries
+* Building permit data: https://data.seattle.gov/resource/mags-97de.json?$where=issue_date%20IS%20NOT%20NULL%20&action_type=NEW&category=MULTIFAMILY&$limit=100
+* Google maps location (using street address): http://maps.googleapis.com/maps/api/geocode/json?address=5601+22ND+AVE+NW+Seattle+WA
+* Google maps location (using lat/long): http://maps.googleapis.com/maps/api/geocode/json?latlng=47.66979666%2C-122.38570052
+;Sample charts/maps
+* Google Fusion table map of permits 2010-2015 [https://www.google.com/fusiontables/DataSource?docid=1aOSWYuXXqh7U2bnUsBH7mM2w3JfsYHzjk29ihbi8]
+;Help resources and inspiration
+* About Google maps API: https://developers.google.com/maps/documentation/geocoding/#ReverseGeocoding
 * About Google Fusion tables: https://support.google.com/fusiontables/answer/2571232
 * Google Fusion Tables mapmaking tutorial: https://support.google.com/fusiontables/answer/2527132?hl=en&topic=2573107&ctx=topic
@@ Line 50: / Line 62: @@
 :* writing API queries: http://dev.socrata.com/docs/queries.html
-;Data.seattle.gov datasets with lat/long or street addresses
+;Other data.seattle.gov datasets with neighborhood, timeseries, and/or location data
 *[https://data.seattle.gov/Community/Seattle-Cultural-Space-Inventory/vsxr-aydq Seattle Cultural Space Inventory]
 *[https://data.seattle.gov/Transportation/MTS-Trail-west-of-I-90-Bridge/u38e-ybnc MTS trail bike/ped traffic]
@@ Line 67: / Line 79: @@
 * https://data.cityofchicago.org/
 * https://data.cityofnewyork.us/
+[[Category:Spring_2015_series]]