Editing Intro to Programming and Data Science (Spring 2020)
From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
= Course Information = | = Course Information = | ||
:'''COM | :'''COM 574: Introduction to Programming and Data Science''' | ||
:'''Location:''' BRNG 2273 | :'''Location:''' BRNG 2273 | ||
:'''Class Hours:''' | :'''Class Hours:''' Tuesdays; 5:30-8:20pm | ||
== Instructor == | == Instructor == | ||
:'''Instructor:''' [https://jeremydfoote.com Jeremy Foote] | :'''Instructor:''' [https://jeremydfoote.com Jeremy Foote] | ||
:'''Email:''' jdfoote@purdue.edu | :'''Email:''' jdfoote@purdue.edu | ||
:'''Office Hours:''' | :'''Office Hours:''' Thursdays; 12:30-2:30pm; BRNG 2156 | ||
Line 19: | Line 19: | ||
The main goal of the class is to help you to complete a preliminary, independent, data-centric project. As part of this project, you (on your own or in a team) will write software to collect data from web APIs, process and clean that data, and produce statistics, hypothesis tests, and graphical visualizations that address questions you are interested in. | The main goal of the class is to help you to complete a preliminary, independent, data-centric project. As part of this project, you (on your own or in a team) will write software to collect data from web APIs, process and clean that data, and produce statistics, hypothesis tests, and graphical visualizations that address questions you are interested in. | ||
I will consider this class a complete success if, at the end, every student can: | I will consider this class a complete success if, at the end, every student can: | ||
Line 28: | Line 29: | ||
* Identify multiple ways that computational approaches are being used for social science research. | * Identify multiple ways that computational approaches are being used for social science research. | ||
* Feel comfortable taking more advanced computational methods courses or learning new techniques on your own. | * Feel comfortable taking more advanced computational methods courses or learning new techniques on your own. | ||
= Required resources and texts = | = Required resources and texts = | ||
Line 39: | Line 38: | ||
== Readings == | == Readings == | ||
* Required text: '''[https://www.py4e.com/book Python for | * Required text: '''[https://www.py4e.com/book Python for Everyone]''' by Charles R. Severance. The book is [https://creativecommons.org/licenses/by/3.0/us/ freely licensed] and available online for free. You can also buy the book if you prefer a hard copy. | ||
I will list required chapters in the weekly notes below. In general, you should expect to spend far more time working on programming tasks than reading. Much like math or other technical courses, this course will build on itself every week. You should make every effort to cover the reading and exercise material every week in preparation for the next week. | I will list required chapters in the weekly notes below. In general, you should expect to spend far more time working on programming tasks than reading. Much like math or other technical courses, this course will build on itself every week. You should make every effort to cover the reading and exercise material every week in preparation for the next week. | ||
* Other readings: Throughout the year we will read and discuss examples of computational social science that I find particularly well done or interesting | * Other readings: Throughout the year we will read and discuss examples of computational social science that I find particularly well done or interesting. I will put these documents on Blackboard. If you come across additional examples that you think the class would benefit from, please suggest them to me. | ||
* Optional readings: Matthew Salganik's book 'Bit by Bit: Social Research in the Digital Age' is a wonderful introduction to computational social science. We will not be discussing it in class but I highly recommend it. | * Optional readings: Matthew Salganik's book 'Bit by Bit: Social Research in the Digital Age' is a wonderful introduction to computational social science. We will not be discussing it in class but I highly recommend it. | ||
Line 61: | Line 60: | ||
* I will hold office hours Thursday afternoons and by appointment. If you come with a programming question, I will expect that you have already tried to solve it yourself in multiple ways and that you have discussed it with at least two classmates. This policy lets me have time to help more students, but it's also a useful strategy. Often [https://en.wikipedia.org/wiki/Rubber_duck_debugging just trying to explain your code] can help you to recognize where you've gone wrong. | * I will hold office hours Thursday afternoons and by appointment. If you come with a programming question, I will expect that you have already tried to solve it yourself in multiple ways and that you have discussed it with at least two classmates. This policy lets me have time to help more students, but it's also a useful strategy. Often [https://en.wikipedia.org/wiki/Rubber_duck_debugging just trying to explain your code] can help you to recognize where you've gone wrong. | ||
* I am also available by email. You can reach me at [mailto:jdfoote@purdue.edu jdfoote@purdue.edu]. I try hard to maintain a boundary between work and home and I typically respond only on weekdays during business hours (~9-5) but during the week I will generally respond within 24 hours. | * I am also available by email. You can reach me at [mailto:jdfoote@purdue.edu jdfoote@purdue.edu]. I try hard to maintain a boundary between work and home and I typically respond only on weekdays during business hours (~9-5) but during the week I will generally respond within 24 hours. | ||
= Assignments = | = Assignments = | ||
Line 81: | Line 78: | ||
''I strongly urge you'' to produce a project that will further your academic career outside of the class. There are many ways that this can happen. Some obvious options are to prepare a project that you can submit for publication, that you can use as pilot analysis that you can report in a grant or thesis proposal, and/or that fulfills a degree requirement. | ''I strongly urge you'' to produce a project that will further your academic career outside of the class. There are many ways that this can happen. Some obvious options are to prepare a project that you can submit for publication, that you can use as pilot analysis that you can report in a grant or thesis proposal, and/or that fulfills a degree requirement. | ||
There are several intermediate milestones and deadlines to help you accomplish a successful research project. Unless otherwise noted, all deliverables should be submitted via | There are several intermediate milestones and deadlines to help you accomplish a successful research project. Unless otherwise noted, all deliverables should be submitted via Blackboard. | ||
== Project plan and dataset identification == | |||
;Due date: January 28, 2020 | ;Due date: January 28, 2020 | ||
Line 94: | Line 91: | ||
* A short (several sentences?) description of how the project will fit into your career trajectory. | * A short (several sentences?) description of how the project will fit into your career trajectory. | ||
== Project planning document == | |||
;Due date: Thursday, March 10, 2020 | ;Due date: Thursday, March 10, 2020 | ||
Line 106: | Line 103: | ||
One approach that I have found helpful is outlined [[CommunityData:Planning document|on this wiki page]]. | One approach that I have found helpful is outlined [[CommunityData:Planning document|on this wiki page]]. | ||
== Project presentation and paper == | |||
;Paper due date: May 5, 2020 | ;Paper due date: May 5, 2020 | ||
Line 114: | Line 111: | ||
;Maximum length: 8 minutes | ;Maximum length: 8 minutes | ||
=== The paper === | |||
Ideally, I expect you to produce a high quality short research paper that you might revise and submit for publication. I do not expect the paper to be ready for publication, but it should contain polished drafts of all the necessary components of a scholarly quantitative empirical research study. In terms of the structure, please see the page on the [[structure of a quantitative empirical research paper]]. | Ideally, I expect you to produce a high quality short research paper that you might revise and submit for publication. I do not expect the paper to be ready for publication, but it should contain polished drafts of all the necessary components of a scholarly quantitative empirical research study. In terms of the structure, please see the page on the [[structure of a quantitative empirical research paper]]. | ||
Line 126: | Line 123: | ||
I am also open to projects that are in the form of a Jupyter notebook, but I expect the same sorts of content to be present. | I am also open to projects that are in the form of a Jupyter notebook, but I expect the same sorts of content to be present. | ||
=== The presentation === | |||
The presentation will provide an opportunity to share a brief summary of your project and findings with the other members of the class. However, don't treat it as a comprehensive overview of your paper: I would rather you tell a subset of the story well than the whole story in a rushed fashion. For instance, you can give a completely successful presentation by describing the motivation and walking through one plot in your paper. Since you will all give other research presentations throughout your career, I strongly encourage you to take the opportunity to refine your academic presentation skills. | The presentation will provide an opportunity to share a brief summary of your project and findings with the other members of the class. However, don't treat it as a comprehensive overview of your paper: I would rather you tell a subset of the story well than the whole story in a rushed fashion. For instance, you can give a completely successful presentation by describing the motivation and walking through one plot in your paper. Since you will all give other research presentations throughout your career, I strongly encourage you to take the opportunity to refine your academic presentation skills. | ||
Line 142: | Line 139: | ||
== Weekly Coding Challenges == | == Weekly Coding Challenges == | ||
Most weeks I will give you all a set of weekly coding challenges before the end of class that will involve | Most weeks I will give you all a set of weekly coding challenges before the end of class that will involve changing or adding to code that I've given you. These coding challenges will be turned in on Blackboard but will not be graded. I encourage you to work together on these challenges but to be sure that you understand the concepts yourself. | ||
I will share my solutions to each of the coding challenges in the subsequent class or via email. As you will see over the course of the semester, there are many possible solutions to many programming problems and my own approaches will often be different than yours. That's completely fine! Coding is a creative act! | I will share my solutions to each of the coding challenges in the subsequent class or via email. As you will see over the course of the semester, there are many possible solutions to many programming problems and my own approaches will often be different than yours. That's completely fine! Coding is a creative act! | ||
You are welcome to discuss the exercises on our | You are welcome to discuss the exercises on our Blackboard discussion board but please do not share answers to challenges more than 12 hours before they are due. After that, you are welcome and encouraged to share your solutions and/or to discuss different approaches. We will discuss a few of the exercises during class and I will randomly choose a few students to explain their solutions. | ||
= Grades = | = Grades = | ||
This course will follow a "self-assessment" philosophy. I am more interested in helping you to learn things that will be useful to you than | This course will follow a "self-assessment" philosophy. I am more interested in helping you to learn things that will be useful to you than grading your. The university still requires grades, so you will be leading the evaluation of your work. This will be completed with me in four stages, at the end of weeks 4, 8, 12, and 16. In each stage, you will reflect on what you have accomplished thus far, how it has met, not met, or exceeded expectations, based both on rubrics and personal goals and objectives. At each of these stages you will receive feedback on your assessments. By the end of the semester, you should have a clear vision of your accomplishments and growth, which you will turn into a grade. As the instructor-of-record, I maintain the right to disagree with your assessment and alter grades as I see fit, but any time that I do this it will be accompanied by an explanation and discussion. These personal assessments, reflecting both honest and meaningful reflection of your work will be the most important factor in final grades. | ||
I suggest that we use the following rubric in our assessment: | I suggest that we use the following rubric in our assessment: | ||
Line 192: | Line 183: | ||
D/F: These are reserved for cases in which students do not complete work or participate. Students may also be | D/F: These are reserved for cases in which students do not complete work or participate. Students may also be | ||
impeding the ability of others to learn. | impeding the ability of others to learn. | ||
= Schedule = | = Schedule = | ||
'''NOTE''' This section will be modified throughout the course to meet the class's needs. Check back in weekly. | '''NOTE''' This section will be modified throughout the course to meet the class's needs. Check back in weekly. | ||
Line 217: | Line 207: | ||
* Have a working python environment on your personal laptop. | * Have a working python environment on your personal laptop. | ||
* Have written your first program in the python language. | * Have written your first program in the python language. | ||
Line 223: | Line 214: | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Finish Day 1 exercises and tutorials | * Finish Day 1 exercises and tutorials | ||
'''Readings (before class):''' | '''Readings (before class):''' | ||
* Python for | * Python for Everyone, chapters 1-2 | ||
* | * Foote, J., Shaw, A., & Hill, B.M. (2017). [https://jeremydfoote.com/files/foote_computational_2017.pdf Computational analysis of social media scholarship]. In Burgess, J., Poell, T., Marwick, A. (Eds.), The Sage Handbook of Social Media. Sage. | ||
'''Class Schedule:''' | '''Class Schedule:''' | ||
* [[Intro to Programming and Data Science (Spring 2020)/Day_2_Lecture|Lecture notes]] | |||
* Project time — We'll begin working on the [[wordplay]] projects independently or in small groups. | |||
Here are your [[Intro to Programming and Data Science (Spring 2020)/Day_2_Coding_Challenges|Exercises]] | |||
'''By the end of class you will:''' | |||
* Have written a program with loops and lists. | |||
* Have a better understanding of the expectations for your final project, and be ready to hand in your initial assignment. | |||
== Week 3: Conditionals and Functions (January 28) == | == Week 3: Conditionals and Functions (January 28) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Final project idea (turn in on | * Final project idea (turn in on Blackboard). | ||
* Finish | * Finish Wordplay examples | ||
'''Readings:''' | '''Readings:''' | ||
* Python for | * Python for Everyone, chapters 3-4 | ||
''' | '''Course plan:''' | ||
* Go over last week's assignment. | |||
* Dictionaries and aggregations [[Intro to Programming and Data Science (Spring 2020)/Day 3 Notes|Day 3 Notes]] | |||
* Discuss average, median using the wordplay data. | |||
* Project time — We'll begin working on a series of project based on the [http://mako.cc/teaching/2015/cdsw-autumn/babynames.zip Baby names] project. | |||
* [[Intro to Programming and Data Science (Spring 2020)/Day 3 Coding Challenges|Day 3 Coding Challenges]] | * [[Intro to Programming and Data Science (Spring 2020)/Day 3 Coding Challenges|Day 3 Coding Challenges]] | ||
'''Resources:''' | |||
* [[Python_data_types_cheat_sheet]] A cheat sheet with everything we've covered in class so far including today. | |||
== Week 4: Iteration, strings, and lists (February 4) == | == Week 4: Iteration, strings, and lists (February 4) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* First | * First self-assessment reflection is due. | ||
* Finish Baby Names examples. | |||
'''Readings:''' | '''Readings:''' | ||
* Python for | * Python for Everyone | ||
* chapters_to_read = [5, 6, 8] | |||
'''Course Plan''' | |||
' | * Basic visualizations in python | ||
* Project time - We'll reuse the babynames code. | |||
* [[Intro to Programming and Data Science (Spring 2020)/Day 4 Coding Challenges|Day 4 Coding Challenges]] | * [[Intro to Programming and Data Science (Spring 2020)/Day 4 Coding Challenges|Day 4 Coding Challenges]] | ||
Line 271: | Line 276: | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* | |||
* Turn in (on Blackboard) your solutions to the Day 4 coding challengss | |||
'''Readings:''' | '''Readings:''' | ||
book = open('Python for Everyone', 'r') | |||
book = open('Python for | |||
for chapter in book: | for chapter in book: | ||
if chapter = '7': | if chapter = '7': | ||
Line 282: | Line 287: | ||
book.close() | book.close() | ||
''' | '''Course Plan''' | ||
* Go over last week's assignment. | |||
* Go over last week's assignment and review histograms. | |||
* Discuss APIs and downloading data from the internet. Refer to [[Intro to Programming and Data Science (Spring 2020)/Day 5 Notes|Day 5 Notes]] | |||
* Spend time on [[Intro to Programming and Data Science (Spring 2020)/Day 5 Coding Challenges|Day 5 Coding Challenges]] | * Spend time on [[Intro to Programming and Data Science (Spring 2020)/Day 5 Coding Challenges|Day 5 Coding Challenges]] | ||
== Week 6: | |||
== Week 6: Dictionaries and networks (February 18) == | |||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Turn in (on | |||
* Turn in (on Blackboard) your solutions to the Day 4 coding challenges | |||
'''Readings:''' | '''Readings:''' | ||
* Python for Everyone, chapters 9 and 10 | |||
''' | '''Course Plan''' | ||
* Discuss data downloading and cleaning. Refer to [[Intro to Programming and Data Science (Spring 2020)/Day 7 Notes|Day 7 Notes]] | |||
* | * We will be discussing this data set: https://hub.mph.in.gov/dataset/aries-crash-data-2007-2017/resource/cc90589c-72d8-4d92-a5fe-73254b555c73 | ||
* Spend time on [[Intro to Programming and Data Science (Spring 2020)/Day 7 Coding Challenges|Day 7 Coding Challenges]] which are group challenges. | |||
== Week 7: | |||
== Week 7: Pandas and dataframes (February 25) == | |||
'''Assignment Due:''' | '''Assignment Due:''' | ||
Final Project Proposal. | |||
'''Readings:''' | '''Readings:''' | ||
''' | '''Course Plan''' | ||
* | |||
* Discuss pivot tables in Excel | |||
* [[Intro to Programming and Data Science (Spring 2020)/Day 8 notes|Day 8 notes]] | |||
== Week 8: | == Week 8: Jupyter and Data Visualization (March 3) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Second | * Second self-assessment reflection is due. | ||
'''Readings:''' | |||
'''Course Plan''' | |||
* Follow up from last week: let's discuss inference and A/B testing. | |||
** [https://www.exp-platform.com/Documents/2016-11BestRefutedCausalClaimsFromObservationalStudies.pdf Examples of bad observational studies] | |||
* Visualization dos and don'ts. We'll discuss the European Environmental Agency's [https://www.eea.europa.eu/data-and-maps/daviz/learn-more/chart-dos-and-donts list of advice for making charts]. **I will refer to this guide as a grade your final projects.** | |||
* Two options for remainder of class. You can work through this introductory guide to visualization in python or you can work on your final project. I'll be here to answer any questions. | |||
'''Optional visualization in python tutorial''' | |||
Self-guided visualization tutorial in python. [https://raw.githubusercontent.com/guyrt/teaching/master/2019/Com520B/VisualizationNotebook.ipynb Download here]. Save the file in a new directory in your desktop and open it with jupyter notebook | |||
If you are on Windows, you may run into an issue with missing path variables. [https://stackoverflow.com/questions/52821162/jupyter-notebook-failed-to-load-dll This SO post helped me solve it.] | |||
== Week 9: Collecting data with APIs (March 10) == | == Week 9: Collecting data with APIs (March 10) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
'''Readings:''' | '''Readings:''' | ||
== March 17: SPRING BREAK == | == March 17: SPRING BREAK == | ||
Line 379: | Line 367: | ||
== Week 10: Cleaning data and operationalization (March | |||
== Week 10: Cleaning data and operationalization (March 24) == | |||
'''Assignment Due:''' | '''Assignment Due:''' | ||
'''Readings:''' | '''Readings:''' | ||
''' | '''Course Plan''' | ||
== Week 11: Storing code and data (March 31) == | |||
'''Assignment Due:''' | '''Assignment Due:''' | ||
'''Readings:''' | '''Readings:''' | ||
''' | '''Course Plan''' | ||
* | * We will learn about using the version control system Git and the Git hosting site Github | ||
== Week 12: | |||
== Week 12: Screen scraping (April 7) == | |||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Third self-assessment reflection is due. | * Third self-assessment reflection is due. | ||
'''Readings:''' | '''Readings:''' | ||
''' | '''Course Plan''' | ||
== Week 13: Statistical summaries and tests (April | == Week 13: Statistical summaries and tests (April 14) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
'''Readings:''' | |||
'''Course Plan''' | |||
== Week 14: Introduction to computational text analysis (April 21) == | |||
'''Assignment Due:''' | '''Assignment Due:''' | ||
'''Readings:''' | '''Readings:''' | ||
''' | '''Course Plan''' | ||
== Week 15: Project presentations ( | == Week 15: Project presentations (April 28) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Final project presentations | * Final project presentations | ||
'''Readings:''' | '''Readings:''' | ||
'''Course Plan''' | |||
''' | * We will listen to each of the final project presentations. | ||
* We will listen | |||
== Week 16: Finals week ( | == Week 16: Finals week (April 28) == | ||
'''Assignment Due:''' | '''Assignment Due:''' | ||
* Final paper due | * Final paper due | ||
= Administrative Notes = | = Administrative Notes = |