Community Data Science Workshops (Spring 2015)/Day 3 Lecture: Difference between revisions

Latest revision as of 01:13, 17 September 2015

Material for the lecture[edit]

For the lecture, you will need two files. Download both of these to your computer by using right or control click on the link and then using Save as or Save link as. Keep track of where you put the files.

Overview of the day[edit]

Lecture
- Our philosophy around data visualization
- Introduce some new programming tools!
- We're going to walk through some analysis of edits to Harry Potter in Wikipedia, start to finish
- We'll focus on manipulating data in Python
- Visualizing things in Google Docs
Lunch (vegetarian Greek!)
Project based work
- More Harry Potter on Wikipedia project (or your own topic) on doing analysis using Google Docs
- Matplotlib
- Civic Data - More interactive working on projects
- Room for you to to work on your projects!
Wrap-up!

Lecture outline[edit]

Step 1: Pre-Requisites

My philosophy about data analysis: use the tools you have
Four things in Python I have to teach you:
- while loops
  - infinite loops
  - loops with a greater than or less than
- break / continue
- "\t".join()
- defining your own functions with def foo(argument):

Step 2: Walking through a Program

Walk-through of get_hpwp_dataset.py
Look at dataset with more and/or in spreadsheet

Step 3: Loading Data Back In

Load data into Python
- review of opening files
  - we can also open them for reading with open('file', 'r', encoding="utf-8")
- csv.DictReader()
Basic counting: hpwp-minor.py
- Answer question: What proportion of edits to Wikipedia Harry Potter articles are minor?
  - Count the number of minor edits and calculate proportion
Looking at time series data hpwp-trend.py
- "Bin" data by day to generate the trend line
Exporting and visualizing data
- Export dataset on edits over time
- Export dataset on articles over users
- Load data into Google Docs

@@ Line 3: / Line 3: @@
 For the lecture, you will need two files. Download both of these to your computer by using right or control click on the link and then using ''Save as'' or ''Save link as''. Keep track of where you put the files.
-* http://mako.cc/teaching/2014/cdsw-autumn/build_hpwp_dataset.py
+* http://mako.cc/teaching/2015/cdsw-spring/harrypotter-wikipedia-cdsw.zip
-* http://nada.com.washington.edu/~mako/hp_wiki.csv
+* http://communitydata.cc/~mako/hp_wiki.tsv
 == Overview of the day ==
@@ Line 16: / Line 16: @@
 * Lunch (vegetarian Greek!)
 * Project based work
-** Project and challenge based continuition of the work in here focusing on Google Docs
+** More [[Harry Potter on Wikipedia]] project (or your own topic) on doing analysis using Google Docs
-** [[Community Data Science Workshops/Saturday May 31st Matplotlib Session|Matplotlib]]!
+** [[Matplotlib]]
+** Civic Data - More interactive working on projects
 ** Room for you to to work on your projects!
 * Wrap-up!
 == Lecture outline ==
+'''Step 1: Pre-Requisites'''
 * My philosophy about data analysis: ''use the tools you have''
@@ Line 29: / Line 32: @@
 *** loops with a greater than or less than
 ** break / continue
-** string.join()
+** "\t".join()
 ** defining your own functions with <code>def foo(argument):</code>
+'''Step 2: Walking through a Program'''
 * Walk-through of <code>get_hpwp_dataset.py</code>
 * Look at dataset with <code>more</code> and/or in spreadsheet
+'''Step 3: Loading Data Back In'''
 * Load data into Python
 ** review of opening files
-*** we can also open them for reading
+*** we can also open them for reading with <code>open('file', 'r', encoding="utf-8")</code>
-** csv module and and csv.reader() function
 ** csv.DictReader()
-* Basic counting
+* Basic counting: <code>hpwp-minor.py</code>
 ** Answer question: ''What proportion of edits to Wikipedia Harry Potter articles are minor?''
 *** Count the number of minor edits and calculate proportion
-* Looking at time series data
+* Looking at time series data <code>hpwp-trend.py</code>
 ** "Bin" data by day to generate the trend line
 * Exporting and visualizing data
@@ Line 48: / Line 56: @@
 ** Load data into Google Docs
-We mostly worked on these questions in the afternoon:
+[[Category:Spring_2015_series]]
-* More advanced counting
-** Answer question: ''What are the most edited articles on Harry Potter?''
-*** Count the number of edits per articles
-** Answer question: ''Who are the most active editors on articles in Harry Potter?''
-*** Count the number of edits per user