Community Data Science Workshops (Spring 2015)/Day 3 Lecture: Difference between revisions
From CommunityData
| Line 23: | Line 23: | ||
== Lecture outline == | == Lecture outline == | ||
Step 1: Pre-Requistes | '''Step 1: Pre-Requistes''' | ||
* My philosophy about data analysis: ''use the tools you have'' | * My philosophy about data analysis: ''use the tools you have'' | ||
| Line 34: | Line 34: | ||
** defining your own functions with <code>def foo(argument):</code> | ** defining your own functions with <code>def foo(argument):</code> | ||
Step 2: Walking through a Program | '''Step 2: Walking through a Program''' | ||
* Walk-through of <code>get_hpwp_dataset.py</code> | * Walk-through of <code>get_hpwp_dataset.py</code> | ||
* Look at dataset with <code>more</code> and/or in spreadsheet | * Look at dataset with <code>more</code> and/or in spreadsheet | ||
Step 3: Loading | '''Step 3: Loading Data Back In''' | ||
* Load data into Python | * Load data into Python | ||
| Line 45: | Line 45: | ||
*** we can also open them for reading with <code>open('file', 'r', encoding="utf-8")</code> | *** we can also open them for reading with <code>open('file', 'r', encoding="utf-8")</code> | ||
** csv.DictReader() | ** csv.DictReader() | ||
* Basic counting | * Basic counting: <code>hpwp-minor.py</code> | ||
** Answer question: ''What proportion of edits to Wikipedia Harry Potter articles are minor?'' | ** Answer question: ''What proportion of edits to Wikipedia Harry Potter articles are minor?'' | ||
*** Count the number of minor edits and calculate proportion | *** Count the number of minor edits and calculate proportion | ||
Revision as of 04:41, 4 May 2015
Material for the lecture
For the lecture, you will need two files. Download both of these to your computer by using right or control click on the link and then using Save as or Save link as. Keep track of where you put the files.
- https://github.com/makoshark/harrypotter-wikipedia-cdsw/raw/master/build_hpwp_dataset.py
- http://wiki.communitydata.cc/~mako/hp_wiki.csv
Overview of the day
- Lecture
- Our philosophy around data visualization
- Introduce some new programming tools!
- We're going to walk through some analysis of edits to Harry Potter in Wikipedia, start to finish
- We'll focus on manipulating data in Python
- Visualizing things in Google Docs
- Lunch (vegetarian Greek!)
- Project based work
- Harry Potter on Wikipedia project on doing analysis using Google Docs
- Matplotlib!
- Room for you to to work on your projects!
- Wrap-up!
Lecture outline
Step 1: Pre-Requistes
- My philosophy about data analysis: use the tools you have
- Four things in Python I have to teach you:
- while loops
- infinite loops
- loops with a greater than or less than
- break / continue
- "\t".join()
- defining your own functions with
def foo(argument):
- while loops
Step 2: Walking through a Program
- Walk-through of
get_hpwp_dataset.py - Look at dataset with
moreand/or in spreadsheet
Step 3: Loading Data Back In
- Load data into Python
- review of opening files
- we can also open them for reading with
open('file', 'r', encoding="utf-8")
- we can also open them for reading with
- csv.DictReader()
- review of opening files
- Basic counting:
hpwp-minor.py- Answer question: What proportion of edits to Wikipedia Harry Potter articles are minor?
- Count the number of minor edits and calculate proportion
- Answer question: What proportion of edits to Wikipedia Harry Potter articles are minor?
- Looking at time series data
- "Bin" data by day to generate the trend line
- Exporting and visualizing data
- Export dataset on edits over time
- Export dataset on articles over users
- Load data into Google Docs
