Editing Statistics and Statistical Programming (Fall 2020)

From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
<div style="float:right;" width=30%; class="toclimit-3">__TOC__</div>
<div style="float:right;" class="toclimit-2">__TOC__</div>


;Statistics and Statistical Programming
:'''Statistics and Statistical Programming'''
:Media, Technology & Society (MTS) 525 and Communication Studies 395
::Media, Technology & Society (MTS) 525
:Tuesdays & Thursdays 1-2:50pm CT
::Tuesdays & Thursdays 10-11:50am (via Zoom)
:Fall 2020
::Spring, 2019
:Northwestern University
::Northwestern University


;Course websites
:'''Instructor:''' [http://aaronshaw.org Aaron Shaw] ([mailto:aaronshaw@northwestern.edu aaronshaw@northwestern.edu])
: [https://canvas.northwestern.edu/courses/122522 Canvas] for [https://canvas.northwestern.edu/courses/122522/announcements announcements], [https://canvas.northwestern.edu/courses/122522/assignments assignments], and some [https://canvas.northwestern.edu/courses/122522/files files].
::Office Hours: <TBA> or by appointment
: [https://northwestern.zoom.us Zoom] for synchronous course meetings.
::<location tba>
: [https://discord.com Discord] for discussions and chat.
: [https://wiki.communitydata.science/Statistics_and_Statistical_Programming_(Fall_2020) This wiki page] for nearly everything else.


;'''Instructor:''' [http://aaronshaw.org Aaron Shaw] ([mailto:aaronshaw@northwestern.edu aaronshaw@northwestern.edu])
:'''Teaching Assistant:''' <TBA>
:Office Hours: Thursday 10am-12pm and by appointment
::Office Hours: <tba>
:Please use [[User:Aaronshaw/OH|office hours signups (with location information)]]
::<location tba>
:Also usually available via chat during "business hours."


;'''Teaching Assistant:''' [http://nickmvincent.com Nick Vincent] ([mailto:nickvincent@u.northwestern.edu nickvincent@u.northwestern.edu])
:'''Course Websites''':
:Office Hours: Monday 10am-12pm and by appointment. I'll try to respond to any asynchronous questions in a timely fashion during "business hours" (9a-5p Central Time), and will also have OH by appointment. I'll respond best to email (above), but am also happy to use Discord for quicker back-and-forth.
:* We will use [https://canvas.northwestern.edu/courses/90927 Canvas] for [https://canvas.northwestern.edu/courses/90927/announcements announcements], [https://canvas.northwestern.edu/courses/90927/assignments turning in most assignments], and maybe [https://canvas.northwestern.edu/courses/90927/discussion_topics discussions] the other possibility is [https://discord.com Discord].
:I am happy to try out alternative communication software for OH!
:* Everything else will be linked on this page.


<br>
[[File:Datasaurus.gif|left|450px|frame|Image from [https://www.autodeskresearch.com/publications/samestats Matejka and Fitzmaurice, ''CHI'', 2017]|link=https://www.autodeskresearch.com/publications/samestats]]
<br clear=all>


== Course information ==
== Overview and learning objectives ==
=== Overview and learning objectives ===


This course provides a get-your-hands-dirty introduction to inferential statistics and statistical programming mostly for applications in the social sciences and social computing. My main objectives are for all participants to acquire the conceptual, technical, and practical skills to conduct your own statistical analyses and become more sophisticated consumers of quantitative research in communication, human computer interaction (HCI), and adjacent disciplines.
This course provides a get-your-hands-dirty introduction to inferential statistics and statistical programming mostly for applications in the social sciences and social computing. My main objectives are for all participants to acquire the conceptual, technical, and practical skills to conduct your own statistical analyses and become more sophisticated consumers of quantitative research in communication, human computer interaction (HCI), and adjacent disciplines.
Line 43: Line 36:
You are not required to know much about statistics or statistical programming to take this class. I will assume some (very little!) knowledge of the basics of empirical research methods and design, basic algebra and arithmetic, and a willingness to work to learn the rest. In general we are not going to cover most of the math behind the techniques we'll be learning. Although we may do some math, this is not a math class. This course will also not require knowledge of calculus or matrix algebra. I will *not* do proofs on the board. Instead, the class is unapologetically focused on the application of statistical methods. Likewise, while some exposure to R, other programming languages, or other statistical computing resources will be helpful, it is not assumed.
You are not required to know much about statistics or statistical programming to take this class. I will assume some (very little!) knowledge of the basics of empirical research methods and design, basic algebra and arithmetic, and a willingness to work to learn the rest. In general we are not going to cover most of the math behind the techniques we'll be learning. Although we may do some math, this is not a math class. This course will also not require knowledge of calculus or matrix algebra. I will *not* do proofs on the board. Instead, the class is unapologetically focused on the application of statistical methods. Likewise, while some exposure to R, other programming languages, or other statistical computing resources will be helpful, it is not assumed.


'''Why this course? Why statistical programming? Why R?'''
== Why this course? Why statistical programming? Why R? ==


Many comparable courses in statistics and quantitative methods do not emphasize statistical programming. So why bother? By learning statistical programming you will gain a deeper understanding of both the principles behind your analysis techniques as well as the tools you use to apply those techniques. In addition, a solid grasp of statistical programming will prepare you to create reproducible research, avoid common errors, and enable both greater durability and validity of your work.  
Many comparable courses in statistics and quantitative methods do not focus on statistical programming and use easier-to-learn statistical software than R. So why bother? By learning statistical programming you will gain a deeper understanding of both the principles behind your analysis techniques as well as the tools you use to apply those techniques. In addition, a solid grasp of statistical programming will prepare you to create reproducible research, avoid common errors, and enable both greater durability and validity of your work.  


Other programming languages are also well suited to statistics, including Stata and Python. I do most of my work with R, so that guides my choice for the course. That said, I opt to use and teach with R for a few reasons:
Other programming languages are also well suited to statistics, including Stata and Python. I am most comfortable and capable with R, so that guides my choice for the course. However, I like to use and teach with R for a few reasons:
* R is freely available and open source.
* R is freely available and open source.
* R is the most widely used package in statistics and several social scientific fields.
* R is becoming the most widely used package in statistics and many social science fields.
* R (along with Stata) will be used in most of the advanced stats classes I hope you will take after this course.
* R (along with Stata) will be used in most of the advanced stats classes I hope you will take after this course.
* R is better general purpose programming language than Stata which means that R programming skills will let you solve non-statistical problems and may make it easier to learn other programming languages like Python.
* R is better general purpose programming language than software like Stata which means that R programming skills will let you solve non-statistical problems and may make it easier to learn other programming languages like Python.


=== Format and structure ===
== A note about this syllabus ==
<!---
I expect everybody to come to class, every week, with a laptop and a power cord, ready to answer any question on the problem set and having uploaded code related the the programming questions. The class is listed as nearly 3 hours long and, with the exception of short breaks, I intend to use the entire period. Please be in class on time, plugged in, and ready to go.
--->


This course will proceed in a '''remote''' format that includes ''asynchronous'' and ''synchronous'' elements (more on those below). In general, the organization of the course adopts a "flipped" approach where participants consume, discuss, and process instructional materials outside of "class" and we use synchronous meetings to answer questions, address challenges or concerns, work through solutions, and hold semi-structured discussions.
This syllabus will be a dynamic document that will evolve throughout the quarter. Although the core expectations are fixed, the details will shift. As a result, please keep in mind the following:


The course introduces ''both'' basic statistical concepts as well as applications of those concepts through statistical programming. As a result, we will usually dedicate part of each week to a particular set of concepts and part of each week to applied data analysis and/or interpretation. A brief description of how I expect it all to work follows below. We'll talk about it more during the first class session.
# '''Assignments and readings are ''frozen'' 1 week before they are due.''' I will not add readings or assignments less than one week before they are due. If I forget to add something or fill in a "To Be Determined" less than one week before it's due, it is dropped. If you plan to read or work more than one week ahead, contact me first.
 
# '''Substantial changes to the syllabus or course materials will be announced.''' Please closely monitor your email and/or [https://canvas.northwestern.edu the announcements section on the course website on Canvas]. When I make changes, these changes will be recorded in [https://wiki.communitydata.science/index.php?title=Statistics_and_Statistical_Programming_(Fall_2020)&action=history  the edit history of this page] so that you can track what has changed. I will also do my best to summarize these changes in an announcement on Canvas that will be emailed to everybody in the class.
====Asynchronous elements of the course====
# '''The course design may adapt throughout the quarter.''' As this is a new format for this course, I may iterate and prototype course design elements rapidly along the way. To this end, I will ask you for voluntary anonymous feedback — especially toward the beginning of the quarter. Please let me know what is working and what can be improved. In the past, I have made many adjustments based on this feedback and I expect to do so again.
 
These include all readings, recorded lectures/slides, tutorials, textbook exercises, problem sets, and other assignments. I expect you to complete (or at least attempt to complete!) these outside of our class meeting times. I also strongly encourage you to identify, submit, and discuss questions about the material '''before each class meeting''' whenever possible.
 
We will use Discord for everyday discussions and chat related to the course. In general, the teaching team will try to keep an eye on the various server channels during "business hours." To the extent that we can respond to questions and concerns there, we'll do so. We'll also use the discussion channels to identify topics that might benefit from synchronous conversation during the course meetings. Hopefully, writing and talking about questions and concerns outside of the synchronous course meetings will help support accountability, learning, and more effective use of our meeting time.
 
For nearly all of the "instructional" material introducing particular statistical concepts and techniques, you are assigned materials from the OpenIntro textbook and lecture materials created by the textbook authors. Please note that this means I will not deliver lectures during our class meetings. Please also note that this means you are responsible for coordinating your working groups and any collaborative work with other members of the class outside of our class meeting times.
 
====Synchronous elements of the course====
 
The synchronous elements of the course will be the two weekly class meetings that will happen via video conference (Zoom). These are scheduled to run for a maximum of 110 minutes. Each session will include multiple short breaks.
 
We will use the class meetings to discuss and work through any questions or challenges you encounter in the materials assigned for that day. This means that I encourage you to identify, submit, and discuss questions about the material '''before each class meeting''' whenever possible. Doing so will give the teaching team time to sift, sort, and organize the questions into a hopefully-cohesive plan for each class session that is tailored to the specific concerns you encounter in the material. Obviously, we anticipate that questions will arise during the class sessions too as well and we'll do our best to adapt as we go.
 
A couple of other notes about the synchronous course meetings:
* Aaron plans to record the course meetings and have them available to class participants only via Zoom/Canvas. Please get in touch if you have concerns or requests about this.
* The teaching team will do our best to notice and respond to any questions or comments that come up via Discord or Zoom during the class. Please do what you can to support these efforts.
* You might want to create/acquire something like [https://www.mccormick.northwestern.edu/news/articles/2020/08/back-to-school-hack-shares-students-handwritten-work-and-teacher-response-in-real-time.html NU Mechanical Engineering Professor Michael Peshkin's homebrew document camera] to facilitate sharing hand-written notes/drawings during class.
 
In addition, because randomness is extremely important in statistics, I may occasionally '''randomly assign''' different working groups to share and discuss their solutions to selected textbook exercises or problem set questions during class. These random assignments will be announced ahead of time so that the group has an opportunity to prepare. The idea here is to structure some participation in the synchronous sessions to ensure an equitable distribution of the responsibility to discuss questions, answers, points of confusion, and alternatives.
 
==== Working groups ====
 
At the start of the course you will be assigned to a small working group. This will be a group of 2-3 students (exact numbers will depend on the final enrollment) with whom you may meet outside of class time to discuss, complete, and/or review your weekly assignments (as well as some of the research project assignments). The groups will rotate at least once during the quarter to ensure that you get to work with different members of the class. The main idea is to support collaborative learning, peer support, and accountability. While the specifics of exactly when and how you work with your working group will largely be up to you, the teaching team will provide [[Statistics_and_Statistical_Programming_(Fall_2020)/Working_groups_template|suggestions in the form of a template]] that you can use as a starting point.
 
As a general rule, we strongly encourage you to collaborate with members of your working group on any/all weekly (minor) assignments. You may, if you choose, also collaborate with others in your group or the class on your research project (major) assignments; however, collaborative research projects should be discussed with a member of the teaching team and all research project assignment submissions should include the names of all collaborators.
 
<!---
Although the day-to-day routine will vary, each class session will generally include the following:
* Quick updates about assignments, projects, and meta-discussion about the class.
* Discussion of '''programming challenges''' due that day (and related to the previous week's R lecture materials).
* Discussion of  '''statistics questions''' related to new material in Diez, Barr, and Çetinkaya-Rundel.
* Discussion of any exemplary empirical paper we have read and the '''empirical paper questions'''.
--->


=== Textbook, readings, and resources ===
== Books and resources ==


This class will use a freely-licensed textbook:
This class will use a freely-licensed textbook:
Line 117: Line 74:
* Verzani, John. 2014. ''Using R for Introductory Statistics, Second Edition''. 2 edition. Boca Raton: Chapman and Hall/CRC. ([https://en.wikipedia.org/wiki/Special:BookSources/978-1-4665-9073-1 Various Sources]; [https://www.amazon.com/Using-Introductory-Statistics-Second-Chapman/dp/1466590734/ref=mt_hardcover?_encoding=UTF8&me= Amazon])
* Verzani, John. 2014. ''Using R for Introductory Statistics, Second Edition''. 2 edition. Boca Raton: Chapman and Hall/CRC. ([https://en.wikipedia.org/wiki/Special:BookSources/978-1-4665-9073-1 Various Sources]; [https://www.amazon.com/Using-Introductory-Statistics-Second-Chapman/dp/1466590734/ref=mt_hardcover?_encoding=UTF8&me= Amazon])
* Wickham, Hadley. 2010. ''ggplot2: Elegant Graphics for Data Analysis''. 1st ed. 2009. Corr. 3rd printing 2010 edition. New York: Springer. ([https://link.springer.com/book/10.1007%2F978-3-319-24277-4 Springer/NU Libraries]; [https://en.wikipedia.org/wiki/Special:BookSources/978-0-596-80915-7 Various Sources])
* Wickham, Hadley. 2010. ''ggplot2: Elegant Graphics for Data Analysis''. 1st ed. 2009. Corr. 3rd printing 2010 edition. New York: Springer. ([https://link.springer.com/book/10.1007%2F978-3-319-24277-4 Springer/NU Libraries]; [https://en.wikipedia.org/wiki/Special:BookSources/978-0-596-80915-7 Various Sources])
* Wickham, Hadly and Grolemund, Garret. 2017. ''R for Data Science''. Sebastopol, CA: O'Reilly. ([https://r4ds.had.co.nz/ Online version]).


There are also some invaluable non-textbook resources:
There are also some invaluable non-textbook resources:
Line 123: Line 79:
* [ftp://cran.r-project.org/pub/R/doc/contrib/Baggott-refcard-v2.pdf Baggott's R Reference Card v2] — Print this out. Take it with you everywhere and look at it dozens of times a day. You will learn the language faster!
* [ftp://cran.r-project.org/pub/R/doc/contrib/Baggott-refcard-v2.pdf Baggott's R Reference Card v2] — Print this out. Take it with you everywhere and look at it dozens of times a day. You will learn the language faster!
* [https://stackoverflow.com/questions/tagged/r StackOverflow R Tag] — Somebody already had your question about how to do ''X'' in R. They asked it, and several people have answered it, on StackOverflow. Learning to read this effectively will take time but as build up some basic familiarity with R and with StackOverflow, it will get easier. I promise.
* [https://stackoverflow.com/questions/tagged/r StackOverflow R Tag] — Somebody already had your question about how to do ''X'' in R. They asked it, and several people have answered it, on StackOverflow. Learning to read this effectively will take time but as build up some basic familiarity with R and with StackOverflow, it will get easier. I promise.
* [http://rseek.org/ Rseek] — Rseek is a modified version of Google that just searches R websites online. Sometimes, R is hard to search because R is a common letter. This has become much easier over time as R has become more popular, but it can still be an issue sometimes and Rseek is a good solution.
* [http://rseek.org/ Rseek] — Rseek is a modified version of Google that just search R websites online. Sometimes, R is hard to search before because R is a common letter. This has become much easier over time as R has become more popular but it might still be the case sometimes and Rseek is a good solution.
* [https://ggplot2.tidyverse.org/ ggplot2 documentation] — ggplot is a powerful data visualization package for R that I recommend highly. The documentation is indispensable for learning how to use it.
* [https://ggplot2.tidyverse.org/ ggplot2 documentation] — Ggplot is a powerful data visualization package for R that I recommend highly. The documentation is indispensable for learning how to use it.
* [https://depts.washington.edu/acelab/proj/Rstats/index.html Statistical Analysis and Reporting in R] — A set of resources created and distributed by Jacob Wobbrock (University of Washington, School of Information) in conjunction with a MOOC he teaches. Contains cheatsheets, code snippets, and data to help execute commonly encountered statistical procedures in R.
* [https://depts.washington.edu/madlab/proj/Rstats/ Statistical Analysis and Reporting in R] — A set of resources created and distributed by Jacob Wobbrock (University of Washington, School of Information) in conjunction with a MOOC he teaches. Contains cheatsheets, code snippets, and data to help execute commonly encountered statistical procedures in R.
* [https://www.datacamp.com DataCamp] offers introductory R courses. Northwestern usually has some free accounts that get passed out via Research Data Services each quarter. Apparently, if you are taking or teaching relevant coursework, instructors can [https://www.datacamp.com/groups/education request] free access to DataCamp for their courses from DataCamp. If folks are interested in this, I can reach out.
* [https://www.datacamp.com DataCamp] offers introductory R courses. Northwestern usually has some free accounts that get passed out via Research Data Services each quarter. Apparently, if you are taking or teaching relevant coursework, instructors can [https://www.datacamp.com/groups/education request] free access to DataCamp for their courses from DataCamp. If folks are interested in this, I can reach out.


Line 131: Line 87:
* If you are planning to analyze large-scale data (i.e., data that won't fit in memory on your laptop) then you will want to sign up for a research allocation on Quest, which is Northwestern's high-performance computing cluster. Instructions on how to do that are [[Statistics_and_Statistical_Programming_(Spring_2019)/Quest_at_Northwestern|here]].
* If you are planning to analyze large-scale data (i.e., data that won't fit in memory on your laptop) then you will want to sign up for a research allocation on Quest, which is Northwestern's high-performance computing cluster. Instructions on how to do that are [[Statistics_and_Statistical_Programming_(Spring_2019)/Quest_at_Northwestern|here]].


=== Weekly (minor) assignments ===
== Assignments ==
 
The assignments in this class focus on applied statistical research design, analysis, and interpretation. There will be no graded exams or quizzes. Unless otherwise noted, all assignments are due at the end of the day (i.e., 11:59pm on the day they are due).
 
=== Weekly problem sets and participation ===
 
Each week I will post a problem set. Some of these will be taken from the textbooks and some will not. They will include:


In order to support continuous progress towards the learning goals for the course, I have assigned some textbook exercises or a problem set ahead of every class. These assignments will provide the basis on which the teaching team will assess and provide feedback on your participation and engagement with the course material.
* '''Statistics questions''' about statistical concepts, principles, and interpretation.
* '''Programming challenges''' that you must solve using R.
* '''Empirical paper questions''' about other assigned readings.  


The first week or so of the course is textbook-focused to get us warmed up. Starting in week 2, we will do more statistical programming and apply the textbook concepts using R and RStudio. In general, we will cover the problem sets in the first session of the week and the textbook materials in the second session.  
You should submit your solutions to the programming challenges (feel free to submit the others if you like, but they're not required!) ahead of each class session. While I will not grade them, we will spend a good chunk of class going through the answers to the assignment due on that day.


==== Textbook exercises ====
Because randomness is extremely important in statistics, I will use a small R program to '''randomly call on''' students to walk through your answer to statistics questions and empirical paper questions in class. We'll then discuss the answers, address points of confusion, and consider alternative approaches as a group.
The focus is on self-assessment of your understanding of the textbook material and you do not need to hand in anything. I expect that you will work on the exercises, review and discuss solutions, and submit any questions ahead of or during class. Please note that solutions to odd-numbered problems appear in the back of the book. The teaching team will distribute solutions to even-numbered problems as well.


==== Problem sets ====
For the programming challenges, you should submit code for your solutions before class (more on how in a moment) so we can walk through the material together. If you get completely stuck on a problem, that's okay, but please share whatever code you have so that you can tell us what you did and what you were thinking.
The course will include problem sets and these may incorporate several kinds of questions:


* '''Statistics questions''' about statistical concepts and principles.
Coming to class will be profoundly important to learning the material and to your final grade. Although the problem sets will not be graded, it is critical that you be present and able to discuss your answers to each of the questions. Your ability to do so will figure prominently in your participation grade for the course (40% of your final grade).
* '''Programming challenges''' that you should solve using R.
* '''Empirical paper questions''' about other assigned readings.  


For the problem sets, I ask that you submit your work [https://canvas.northwestern.edu/courses/122522/assignments via Canvas 24 hours before class] (i.e., Monday afternoon for our Tuesday class sessions). Details of exactly how this will work will be elaborated during the first class. For the programming challenges, you should submit code and text for your solutions (again, more on how later). If you get completely stuck on a problem, that's okay, but please provide whatever you have.
I strongly encourage you to form groups to work on the problem sets if you find that helpful; however, you must still submit your work individually and respond to my cold-call prompts in class individually to help ensure that you learn and understand the material.


Problem sets will be evaluated on a complete/incomplete basis. Although the problem sets will not be assigned a letter grade, they are a central focus of the course and completing them will support your mastery of the material in multiple ways. Working through them on schedule will also make it possible for you to participate in the synchronous course meetings and online discussions of course material effectively. Your ability to do so will figure prominently in your participation grade for the course (see the section on grading and assessment below).
I evaluate participation along four dimensions: attendance, preparation, engagement, and contribution. These are quite similar to the dimensions described in the "Participation Rubric" section of [https://mako.cc/teaching/assessment.html Benjamin Mako Hill's assessment page] and [https://reagle.org/joseph/zwiki/Teaching/Assessment/Participation.html Joseph Reagle's participation assessment rubric]. Exceptional participation means excelling along all four dimensions. Please note that participation ≠ talking more and I encourage all of us to seek [https://reagle.org/joseph/zwiki/Teaching/Best_Practices/Learning/Balance_in_Discussion.html balance in our classroom discussions].


=== Research project (major) assignments ===
=== Research project ===


==== Overview ====
As a demonstration of your learning in this course, you will design and carry out a quantitative research project, start to finish. This means you will all:
As a demonstration of your learning in this course, you will design and carry out a quantitative research project, start to finish. This means you will all:


Line 159: Line 118:
* '''Find a dataset''' — Very quickly, you should identify a dataset you will use to complete this project. For most of you, I suspect you will be engaging in secondary data analysis or a analysis of a previously collected dataset.
* '''Find a dataset''' — Very quickly, you should identify a dataset you will use to complete this project. For most of you, I suspect you will be engaging in secondary data analysis or a analysis of a previously collected dataset.
* '''Engage in descriptive data analysis''' — Use R to calculate descriptive statistics and visualizations to describe your data.
* '''Engage in descriptive data analysis''' — Use R to calculate descriptive statistics and visualizations to describe your data.
* '''Motivate and test at least one hypothesis about relationships between two or more variables''' — I'm happy to discuss alternatives to formal hypothesis testing procedures (even if some of them are beyond the scope of this course).
* '''Motivate and test at least one hypothesis about relationships between two or more variables'''
* '''Report and interpret your findings''' — You will do this in both a short paper and a short (recorded) presentation.
* '''Report and interpret your findings''' — You will do this in both a short paper and a short presentation.
* '''Ensure that your work is replicable''' — You will need to provide code and data for your analysis in a way that makes your work replicable by other researchers.
* '''Ensure that your work is replicable''' — You will need to provide code and data for your analysis in a way that makes your work replicable by other researchers.


''I strongly urge you'' to produce a project that will further your academic career outside of the class. There are many ways that this can happen. Some obvious options are to prepare a project that you can submit for publication, use as pilot analysis that you can report in a grant or thesis proposal, and/or use to fulfill a degree requirement.
''I strongly urge you'' to produce a project that will further your academic career outside of the class. There are many ways that this can happen. Some obvious options are to prepare a project that you can submit for publication, use as pilot analysis that you can report in a grant or thesis proposal, and/or that fulfills a degree requirement.


There are several intermediate milestones, deliverables, and deadlines to help you accomplish a successful research project. Unless otherwise noted, all deliverables should be submitted via Canvas by 5pm CT on the day they are due.
There are several intermediate milestones and deadlines to help you accomplish a successful research project. Unless otherwise noted, all deliverables should be submitted via Canvas.


==== Project plan and dataset identification ====


==== Research project plan and dataset identification ====
;Due date: Thursday, April 18, 2019
 
;Due date: October 9, 2020, 5pm CT
;Maximum length: 500 words (~1-2 pages)
;Maximum length: 500 words (~1-2 pages)


Line 176: Line 134:


* An abstract of the proposed study including the topic, research question, theoretical motivation, object(s) of study, and anticipated research contribution.
* An abstract of the proposed study including the topic, research question, theoretical motivation, object(s) of study, and anticipated research contribution.
* An identification of the dataset you will use and a description of the rows and columns or type(s) of data it will include. If you do not currently have access to these data, explain why and when you will.
* An identification of the dataset you will use and a description of the columns or type of data it will include. If you do not currently have access to these data, explain why and when you will.
* A short (several sentences?) description of how the project will fit into your career trajectory.
* A short (several sentences?) description of how the project will fit into your career trajectory.


==== Project planning document ====
;Due date: Thursday, May 16, 2019
;Maximum length: ~5 pages
The project planning document is a basic shell/outline of an empirical quantitative research paper. Your planning document should should have the following sections: (a) Rationale, (b) Objectives; (b.1) General objectives; (b.2) Specific objectives; (c) (Null) hypotheses; (d) Conceptual diagram and explanation of the relationship(s) you plan to test; (e) Measures; (f) Dummy tables/figures; (g) anticipated finding(s) and research contribution(s). Longer descriptions of each of these planning document sections (as well as a few others) can be found [[CommunityData:Planning document|on this wiki page]].
I have also provided three example planning documents via our Canvas site:
* [https://canvas.northwestern.edu/files/6908602/download?download_frd=1 One by public health researcher Mika Matsuzaki]. The first planning document I ever saw and still one of the best. It's missing a measures section. It's also focused on a research context that is probably very different from yours, but try not to get bogged down by that and imagine how you might map the structure of the document to your own work.
* [https://canvas.northwestern.edu/files/6919735/download?download_frd=1 One by Jim Maddock] created as part of a qualifying exam earlier in 2019. Jim doesn't provide dummy tables or anticipated findings/contributions, but he has an especially phenomenal explanation of the conceptual relationships and processes he wants to test.
* [https://canvas.northwestern.edu/files/6908606/download?download_frd=1 One provided as an appendix to Gerber and Green's excellent textbook, ''Field Experiments: Design, Analysis, and Interpretation'' (FEDAI)]. It's over-detailed and incredibly long for our purposes, but nevertheless an exemplary approach to planning empirical quantitative research in a careful, intentional way that is worthy of imitation.
==== Project presentation and paper ====
;Paper due date: Monday, June 10, 2019
;Maximum length: 6000 words (~20 pages)
;Presentation due date: Thursday, May 30 or Thursday, June 6, 2019
;Maximum length: 8 minutes
''The paper:'' Ideally, I expect you to produce a high quality short research paper that you might revise and submit for publication and/or a dissertation milestone. I do not expect the paper to be ready for publication, but it should contain polished drafts of all the necessary components of a scholarly quantitative empirical research study. In terms of the structure, please see the page on the [[structure of a quantitative empirical research paper]].
As noted above, you should also provide data, code, and any documentation sufficient to enable the replication of all analysis and visualizations. If that is not possible/appropriate for some reason, please talk to me so that we can find another solution.
Because the emphasis in this class is on statistics and methods and because I'm not an expert in each of your fields, I'm happy to assume that your paper, proposal, or thesis chapter has already established the relevance and significance of your study and has a comprehensive literature review, well-grounded conceptual approach, and compelling reason why this research is important. As a result, you need not focus on these elements of the work in your written submission. Instead, feel free to start with a brief summary of the purpose and importance of this research followed by an introduction of your research questions or hypotheses. If you provide more detail, that's fine, but I won't give you detailed feedback on these parts and they will not figure prominently in my assessment of the work.
I have a strong preference for you to write the paper individually, but I'm open to the idea that you may want to work with others in the class. Please contact me ''before'' you attempt to pursue a collaborative final paper.
I do not have strong preferences about the style or formatting guidelines you follow for the paper and its bibliography. However, ''your paper must follow a standard format'' (e.g., [https://cscw.acm.org/2019/submit-papers.html ACM SIGCHI CSCW format] or [https://www.apastyle.org/index APA 6th edition] ([https://templates.office.com/en-us/APA-style-report-6th-edition-TM03982351 Word] and [https://www.overleaf.com/latex/templates/sample-apa-paper/fswjbwygndyq LaTeX] templates)) that is applicable for a peer-reviewed journal or conference proceedings in which you aim to publish the work (they all have formatting or submission guidelines published online and you should follow them). This includes the references. I also strongly recommend that you use reference management software to handle your bibliographic sources.
'' [[Statistics_and_Statistical_Programming_(Spring_2019)/Final_project_presentations|The presentation:]]'' The presentation will provide an opportunity to share a brief summary of your project and findings with the other members of the class. Since you will all give other research presentations throughout your career, I strongly encourage you to take the opportunity to refine your academic presentation skills. The document [https://canvas.northwestern.edu Creating a Successful Scholarly Presentation] (file will be posted to Canvas) may be useful.
: More details about the presentation goals, format suggestions, and more are available [[Statistics_and_Statistical_Programming_(Spring_2019)/Final_project_presentations|on this page]]
=== Grading ===
I will assign grades (usually a numeric value ranging from 0-10) for each of the following aspects of your performance. The percentage values in parentheses are weights that will be applied to calculate your overall grade for the course.
* Participation: 40%
* Proposal identification: 5%
* Final project planning document: 5%
* Final project presentation: 10%
* Final project paper: 40%


===== Notes on finding a dataset =====
My assessment of your paper will reflect the clarity of the written work, the effective execution and presentation of quantitative empirical analysis, as well as the quality and originality of the analysis. Throughout the quarter, we will talk a lot about the qualities of exemplary quantitative research. I expect your final project to embody these exemplary qualities.


In order to complete your final project, you will each need a dataset. If you already have a dataset for the project you plan to conduct, great! If not, fear not! There are many datasets to draw from. Some ideas are below (please suggest others, provide updated links, or report problems). The teaching team will also be available to help you brainstorm/find resources if needed:
== Note on finding a dataset ==
 
In order to complete your project, you will each need a dataset. If you already have a dataset for the project you plan to conduct, great! If not, there are many datasets to draw from. Some ideas are below. Jeremy and Aaron will also be available to help you brainstorm/find resources if needed:


* Ask your advisor for a dataset they have collected and used in previous papers. Are there other variables you could use? Other relationships you could analyze?
* Ask your advisor for a dataset they have collected and used in previous papers. Are there other variables you could use? Other relationships you could analyze?
Line 191: Line 195:
* Use the [http://scientificdata.isa-explorer.org/index.html ISA Explorer] to find datasets. Keep in mind the large majority of datasets it will search are drawn from the natural sciences.
* Use the [http://scientificdata.isa-explorer.org/index.html ISA Explorer] to find datasets. Keep in mind the large majority of datasets it will search are drawn from the natural sciences.
* The City of Chicago has one of the best [https://data.cityofchicago.org/ data portal sites] of any municipality in the U.S. (and better than many federal agencies). There are also numerous administrative datasets released by other public entities (try searching!) that you might find inspiring.
* The City of Chicago has one of the best [https://data.cityofchicago.org/ data portal sites] of any municipality in the U.S. (and better than many federal agencies). There are also numerous administrative datasets released by other public entities (try searching!) that you might find inspiring.
* [http://fivethirtyeight.com FiveThirtyEight.com] has published a [https://cran.r-project.org/web/packages/fivethirtyeight/vignettes/fivethirtyeight.html GitHub repository and an R package] with pre-processed and cleaned versions of many of the datasets they use for articles published on their website.
<!---
* If you interested in studying online communities, there are some great resources for accessing data from Reddit, Wikipedia, and StackExchange. See [https://files.pushshift.io/reddit/ pushshift] for dumps of Reddit data, [https://meta.wikimedia.org/wiki/Research:Data here] for an overview of Wikipedia's data resources, and [https://data.stackexchange.com/ Stack Exchange's data portal].
* <TODO fix/update accordingly> Set up a meeting with Jennifer Muilenburg — Data Curriculum and Communications Librarian who runs [https://www.lib.washington.edu/digitalscholarship/services/data research data services at the UW libraries]. Her email is: libdata@uw.edu I've have talked to her about this course and she is excited about meeting with you to help.
* The NY Times is publishing a [https://github.com/nytimes/covid-19-data COVID-19 data repository] that includes county-level metrics for deaths, mask usage, and other pandemic-related data. The release a lot of it as frequently updated .csv files and the repository includes documentation of the measurements, data collection details, and more.
-->
* The Community Data Science Collective and colleagues have created a [[COVID-19_Digital_Observatory| COVID-19 digital observatory]] (hosted in part right here on this wiki!) that publishes a bunch of pandemic-related data as csv and json files.
* [http://fivethirtyeight.com FiveThirtyEight.com] has published a [https://cran.r-project.org/web/packages/fivethirtyeight/vignettes/fivethirtyeight.html GitHub repository and an R package] with pre-processed and cleaned versions of many of the datasets they use for articles published on their website.  
* The [https://openpolicing.stanford.edu Stanford Open Policing project] has published a huge archive of policing data related mostly to traffic stops in states and many cities of the U.S. We'll use at least one of these files for a problem set.
 
=== Human subjects research, IRB, and ethics ===
In general, you are responsible for making sure that you're on the right side of the IRB requirements and that your work meets applicable ethical norms and standards.


==== Research project planning document ====
Class projects generally do not need IRB approval, but research for publications, dissertations, and sometimes even pilot studies generally fall under IRB purview. You should ''not'' plan to seek IRB approval/determination retroactively. If your study may involve human subjects and you may ever publish it in any form, you will need IRB oversight of some sort.


;Due date: October 30, 2020, 5pm CT
Secondary analysis of anonymized data is generally not considered human subjects research, but I strongly suggest that you get a determination from [https://irb.northwestern.edu/ the Northwestern IRB] before you start. For work that is not considered human subjects research, this can often happen in a few hours or days. If you need to list a faculty sponsor or Principal Investigator, that should ideally be your advisor. If that doesn't make sense for some reason, please talk to me.
;Suggested length: ~5 pages


The project planning document is a shell/outline of an empirical quantitative research paper. Your planning document should should have the following sections: (a) Rationale, (b) Objectives; (b.1) General objectives; (b.2) Specific objectives; (c) (Null) hypotheses; (d) Conceptual diagram and explanation of the relationship(s) you plan to test; (e) Measures; (f) Dummy tables/figures; (g) anticipated finding(s) and research contribution(s). Longer descriptions of each of these planning document sections (as well as a few others) can be found [[CommunityData:Planning document|on this wiki page]].
== Structure of Class ==


I will also provide three example planning documents via our Canvas site (links to-be-updated for 2020 edition of the course):
I expect everybody to come to class, every week, with a laptop and a power cord, ready to answer any question on the problem set and having uploaded code related the the programming questions. The class is listed as nearly 3 hours long and, with the exception of short breaks, I intend to use the entire period. Please be in class on time, plugged in, and ready to go.
* [https://canvas.northwestern.edu/files/9439380/download?download_frd=1 One by public health researcher Mika Matsuzaki]. The first planning document I ever saw and still one of the best. It's missing a measures section. It's also focused on a research context that is probably very different from yours, but try not to get bogged down by that and imagine how you might map the structure of the document to your own work.
* [https://canvas.northwestern.edu/files/9421229/download?download_frd=1 One by Jim Maddock] created as part of a qualifying exam early in 2019. Jim doesn't provide dummy tables or anticipated findings/contributions, but he has an especially phenomenal explanation of the conceptual relationships and processes he wants to test.  
* [https://canvas.northwestern.edu/files/9439379/download?download_frd=1 One provided as an appendix to Gerber and Green's excellent textbook, ''Field Experiments: Design, Analysis, and Interpretation'' (FEDAI)]. It's over-detailed and over-long for the purposes of this assignment, but nevertheless an exemplary approach to planning empirical quantitative research in a careful, intentional way that is worthy of imitation.


==== Research project presentation ====
When it comes to the statistics material, this will mostly be a so-called "flipped" classroom. This means we will rely on the textbook and other resources to introduce the material and we will use the class sessions to discuss questions as they come up.


;Presentation due date: December 3, 2020, 5pm CT
The problem sets each week will
;Maximum length: 10 minutes


<!-- TODO revisit old presentations page to update/adapt
Although the day-to-day routine will vary, each class session will generally include the following:
[[Statistics_and_Statistical_Programming_(Spring_2019)/Final_project_presentations]]
* Quick updates about assignments, projects, and meta-discussion about the class.
--->
* Discussion of '''programming challenges''' due that day (and related to the previous week's R lecture materials).
You will also create and record a short presentation of your final project. The presentation will provide an opportunity to share a brief overview of your project and findings with the other members of the class. Since you will all give other research presentations throughout your career, I strongly encourage you to take the opportunity to refine your academic presentation skills. The document [https://canvas.northwestern.edu/files/9439377/download?download_frd=1 Creating a Successful Scholarly Presentation] (file posted to Canvas) may be useful.
* Discussion of  '''statistics questions''' related to new material in Diez, Barr, and Çetinkaya-Rundel.
* Discussion of any exemplary empirical paper we have read and the '''empirical paper questions'''.
 
== Schedule ==
 
When reading the schedule below, the following key might help resolve ambiguity: §n denotes chapter n; §n.x denotes section x of chapter; §n.x-y denotes sections x through y of chapter n.
 
=== Week 1: Thursday April 4: Introduction, Setup, and Data and Variables ===
 
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 1]]


Additional details about the presentation goals, format suggestions, resources, and more will be provided later in the quarter.
Please complete the readings and assignment prior to class so that we can discuss them and start talking through some of the examples in R together.


==== Research project paper ====
'''Required Readings:'''


;Paper due date: December 10, 2020, 5pm CT
* Diez, Barr, and Çetinkaya-Rundel: §1 (Introduction to data)
;Maximum length: 6000 words (~20 pages)
* Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks. ''Proceedings of the National Academy of Sciences'' 111(24):8788–90. [[http://www.pnas.org/content/111/24/8788.full Open Access]]


I expect you to produce a short, high quality research paper that you might revise, extend, and submit for publication and/or a dissertation milestone. I do not expect the paper to be ready for publication, but it should contain polished drafts of all the necessary components of a scholarly quantitative empirical research study. In terms of the structure, please see the page on the [[structure of a quantitative empirical research paper]].
'''Recommended Readings:'''


As noted above, you should also provide data, code, and any documentation sufficient to enable the replication of all analysis and visualizations. If that is not possible/appropriate for some reason, please talk to me so that we can find another solution.
* Verzani: §1 (Getting Started), §2 (Univariate data) [[https://canvas.northwestern.edu/verzani_ch1-ch2.pdf Available via Canvas]]
* Verzani: §A (Programming)
* Healy: §2 (and skim the preferatory material as well as §1)
'''Assignment (Complete before class):'''


Because the emphasis in this class is on statistics and methods and because I'm probably not an expert in the substance of your research domain, I'm happy to assume that your paper, proposal, or thesis chapter has already established the relevance and significance of your study and has a comprehensive literature review, well-grounded conceptual approach, and compelling reason why this research is important. As a result, you need not focus on these elements of the work in your written submission. Instead, feel free to start with a brief summary of the purpose and importance of this research followed by an introduction of your research questions or hypotheses. If you provide more detail, that's fine, but I won't give you detailed feedback on these parts and they will not figure prominently in my assessment of the work.
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 1]]


I have a strong preference for you to write the paper individually, but I'm open to the idea that you may want to work with others in the class. Please contact me ''before'' you attempt to pursue a collaborative final paper.
'''Lectures:'''
* [https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w01-R_lecture.zip Week 1 R lecture materials] (.zip file)
* [https://communitydata.cc/~ads/teaching/2019/stats/screencasts/w01-s01-intro.webm Week 1 screencast (part 1, 23 minutes)] (the video should load directly in browser window)
* [https://communitydata.cc/~ads/teaching/2019/stats/screencasts/w01-s02-intro.webm Week 1 screencast (part 2, 27 minutes)]


I do not have strong preferences about the style or formatting guidelines you follow for the paper and its bibliography. However, ''your paper must follow a standard format'' (e.g., [https://cscw.acm.org/2019/submit-papers.html ACM SIGCHI CSCW format] or [https://www.apastyle.org/index APA 6th edition] ([https://templates.office.com/en-us/APA-style-report-6th-edition-TM03982351 Word] and [https://www.overleaf.com/latex/templates/sample-apa-paper/fswjbwygndyq LaTeX] templates)) that is applicable for a peer-reviewed journal or conference proceedings in which you might aim to publish the work (they all have formatting or submission guidelines published online and you should follow them). This includes the references. I also strongly recommend that you use reference management software like Zotero to handle your bibliographic sources.
'''Resources:'''
* [https://www.openintro.org/download.php?file=os3_slides_01&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §1 Lecture Notes]
* [https://www.openintro.org/stat/videos.php OpenIntro Video Lectures] including some for §1


==== Human subjects research, IRB, and ethics ====
=== Week 2: Thursday April 11: Probability and Visualization ===
In general, you are responsible for making sure that you're on the right side of the IRB requirements and that your work meets applicable ethical norms and standards.
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 2]]
* Questions? Topics you'd like to discuss? Add them to the [https://canvas.northwestern.edu/courses/90927/discussion_topics/601700 Canvas discussion] for this week's material.


Class projects generally do not need IRB approval, but research for publications, dissertations, and sometimes even pilot studies do fall under IRB purview. You should ''not'' plan to seek IRB approval/determination retroactively. If your study may involve human subjects and you may ever publish it in any form, you will need IRB oversight of some sort.
'''Required Readings:'''


Secondary analysis of anonymized data is generally not considered human subjects research, but I strongly suggest that you get a determination from [https://irb.northwestern.edu/ the Northwestern IRB] before you start. For work that is not considered human subjects research, this can often happen in a few hours or days. If you need to list a faculty sponsor or Principal Investigator, that should ideally be your advisor. If that doesn't make sense for some reason, please talk to me.
* Diez, Barr, and Çetinkaya-Rundel: §2 (Probability)
* Shaw, Aaron and Yochai Benkler. 2012. A tale of two blogospheres: Discursive practices on the left and right. ''American Behavioral Scientist''. 56(4): 459-487. [[https://doi.org/10.1177%2F0002764211433793 available via NU libraries]]


Research ethics are broad and complex topic. We'll talk about issues related to ethics and quantitative empirical research a bit more during class, but will likely only scratch the surface. I strongly encourage you to pursue further reading, conversation, coursework, and reflection as you consider how to understand and apply ethical principles in the context of your own research and teaching.
'''Recommended Readings:'''
* Verzani: §3.1-2 (Bivariate data), §4 (Multivariate data), §5 (Multivariate graphics) <!---[[https://faculty.washington.edu/makohill/com521/verzani-usingr-ch3.1-2_ch4_ch5.pdf Available with UW NetID]]--->
* [https://seeing-theory.brown.edu/ Seeing Theory] §1 (Basic Probability) and §2 (Compound Probability). (Note: this site provides a beautiful visual introduction to core concepts in probability and statistics).
<!---
* Buechley, Leah and Benjamin Mako Hill. 2010. “LilyPad in the Wild: How Hardware’s Long Tail Is Supporting New Engineering and Design Communities.” Pp. 199–207 in ''Proceedings of the 8th ACM Conference on Designing Interactive Systems.'' Aarhus, Denmark: ACM. [[https://mako.cc/academic/buechley_hill_DIS_10.pdf PDF available on my personal website]]
--->
* Healy: §3.


=== Grading and assessment ===
'''Assignment (Complete Before Class):'''


I will assign grades (usually a numeric value ranging from 0-10) for each of the following aspects of your performance. The percentage values in parentheses are weights that will be applied to calculate your overall grade for the course.
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 2]]


* Weekly participation: 40%
'''Lectures:'''
* Proposal identification: 5%
* [https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w02-R_lecture.Rmd Week 2 R lecture materials] (.Rmd file)
* Final project planning document: 5%
* [https://communitydata.cc/~ads/teaching/2019/stats/screencasts/w02.webm Week 2 screencast (17 minutes)]
* Final project presentation: 10%
* Final project paper: 40%


The teaching team will jointly and holistically evaluate your participation along four dimensions: attendance, preparation, engagement, and contribution. These are quite similar to the dimensions described in the "Participation Rubric" section of [https://mako.cc/teaching/assessment.html Benjamin Mako Hill's assessment page] and [https://reagle.org/joseph/zwiki/Teaching/Assessment/Participation.html Joseph Reagle's participation assessment rubric]. Exceptional participation means excelling along all four dimensions. Please note that participation ≠ talking/typing more and I encourage all of us to seek [https://reagle.org/joseph/zwiki/Teaching/Best_Practices/Learning/Balance_in_Discussion.html balance in our discussions].
'''Resources:'''


The teaching team's assessment of your final project proposal, planning document, presentation, and paper will reflect the clarity of the work, the effective execution and presentation of quantitative empirical analysis, as well as the quality and originality of the analysis. A more detailed assessment rubric will be provided. Throughout the quarter, we will talk about the qualities of exemplary quantitative research. In general, I expect your final project to embody these exemplary qualities.
* [https://www.openintro.org/download.php?file=os3_slides_02&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §2 Lecture Notes]
* [https://www.openintro.org/stat/videos.phpOpenIntro Video Lectures] including 2 short videos for §2


=== Policies ===
=== Week 3: Thursday April 18: Distributions ===


==== General course policies ====
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 3]]


[[User:Aaronshaw/Classroom_policies|General policies]] on a wide variety of topics including classroom equity, attendance, academic integrity, accommodations, late assignments, and more are provided [[User:Aaronshaw/Classroom_policies|on Aaron's class policies page]]. Below are some policy statements specific to this course and quarter.
'''Required Readings:'''


==== Teaching and learning in a pandemic ====
* Diez, Barr, and Çetinkaya-Rundel: §3.1-3.2, §3.4: You should read the rest of the chapter (§3.3 and §3.5). I won't assign problem set questions about it but it's still important to be familiar with.


The Covid-19 pandemic will impact this course in various ways, some of them obvious and tangible and others harder to pin down. On the obvious and tangible front, we have things like a mix of remote and (a)synchronous instruction, the fact that many of us will not be anywhere near campus or each other this year, and the unusual academic calendar. These will reshape our collective "classroom" experience in major ways.  
'''Recommended Readings:'''
* Verzani: §6 (Populations)
* [https://seeing-theory.brown.edu/ Seeing Theory] §3 (Probability Distributions).


On the "harder to pin down" side, many of us may experience elevated levels of exhaustion, stress, uncertainty and/or distraction. We may need to provide unexpected support to family, friends, or others in our communities. I have personally experienced all of these things at various times over the past six months and I expect that some of you have too. It is a difficult time.
'''Assignment (Complete Before Class):'''


I believe it is important to acknowledge these realities of the situation and create the space to discuss and process them in the context of our class throughout the quarter. As your instructor and colleague, I commit to do my best to approach the course in an adaptive, generous, and empathetic way. I will try to be transparent and direct with you throughout—both with respect to the course material as well as the pandemic and the university's evolving response to it. I ask that you try to extend a similar attitude towards everyone in the course. When you have questions, feedback, or concerns, please try to share them in an appropriate way. If you require accommodations of any kind at any time (directly related to the pandemic or not), please contact the teaching team.
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 3]]


==== Expectations for synchronous remote sessions ====
'''Lectures:'''


The following are some baseline expectations for our synchronous remote class sessions. I expect that these can and will evolve. Please feel free to ask questions, suggest changes, or raise concerns during the quarter. I welcome all input.
* [https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w03-R_lecture.Rmd Week 3 R lecture materials] (.Rmd file)
* All members of the class are expected to create a supportive and welcoming environment that is respectful of the conditions under which we are participating in this class.
* [https://communitydata.cc/~ads/teaching/2019/stats/screencasts/w03.webm Week 3 screencast (19 minutes)]
* All members of the class are expected to take reasonable steps to create an effective teaching/learning environment for themselves and others.


And here are suggested protocols for any video/audio portions of our class:
'''Resources:'''
* Please mute your microphone whenever you're not speaking and learn to use [https://en.wikipedia.org/wiki/Push-to-talk "push-to-talk"] if/when possible.
* Video is optional for all students at all times, although if you're willing/able to keep the instructor company in the video channel that would be nice.
* If you need to excuse yourself at any time and for any reason you may do so.
* Children, family, pets, roommates, and others with whom you may share your workspace are welcome to join our class as needed.


==== Syllabus revisions ====
* [https://www.openintro.org/download.php?file=os3_slides_03&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §3 Lecture Notes]
* [https://www.openintro.org/stat/videos.php OpenIntro Video Lectures] including 2 videos for §3.1 and §3.2


This syllabus will be a dynamic document that will evolve throughout the quarter. Although the core expectations are fixed, the details will shift. As a result, please keep in mind the following:
=== Week 4: Thursday April 25: Statistical significance and hypothesis testing ===
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 4]]


# '''Assignments and readings are ''frozen'' 1 week before they are due.''' I will not add readings or assignments less than one week before they are due. If I forget to add something or fill in a "To Be Determined" less than one week before it's due, it is dropped. If you plan to read or work more than one week ahead, contact me first.
'''Required Readings:'''
# '''Substantial changes to the syllabus or course materials will be announced.''' Please closely monitor your email and/or [https://canvas.northwestern.edu the announcements section on the course website on Canvas]. When I make changes, these changes will be recorded in [https://wiki.communitydata.science/index.php?title=Statistics_and_Statistical_Programming_(Fall_2020)&action=history  the edit history of this page] so that you can track what has changed. I will also do my best to summarize these changes in an announcement on Canvas that will be emailed to everybody in the class.
# '''The course design may adapt throughout the quarter.''' As this is a new format for this course, I may iterate and prototype course design elements rapidly along the way. To this end, I will ask you for voluntary anonymous feedback — especially toward the beginning of the quarter. Please let me know what is working and what can be improved. In the past, I have made many adjustments based on this feedback and I expect to do so again.


==== Statistics and power ====
* Diez, Barr, and Çetinkaya-Rundel: §4 (Foundations for inference)


The subject matter of this course—statistics and statistical programming—has historical and present-day affinities with a variety of oppressive ideologies and projects, including white supremacy, discrimination on the basis of gender and sexuality, state violence, genocide, and colonialism. It has also been used to challenge and undermine these projects in various ways. I will work throughout the quarter to acknowledge and represent these legacies accurately, at the same time as I also strive to advance equity, inclusion, and justice through my teaching practice, the selection of curricular materials, and the cultivation of an inclusive classroom environment. Please see my [[User:Aaronshaw/Classroom_policies|general classroom policies]] for more on some of these topics.
'''Recommended Readings:'''
* Verzani: §7 (Statistical inference), §8 (Confidence intervals)
* [https://seeing-theory.brown.edu/ Seeing Theory] §4 (Frequentist Inference)


== Schedule (with all the details) ==
'''Assignment (Complete Before Class):'''


When reading the schedule below, the following key might help resolve ambiguity: §n denotes chapter n; §n.x denotes section x of chapter; §n.x-y denotes sections x through y (inclusive) of chapter n.
* [https://docs.google.com/forms/d/e/1FAIpQLScMkAPwWQUjB4C5wtbkemkNZYjNl3ipO4Dg5wsORFmdfduEtA/viewform?usp=sf_link Mid-quarter course evaluation survey] (by Monday please!)
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 4]]


=== Week 1 (9/17) ===
'''Lectures:'''
==== September 17: Intro and setup ====
*[https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w04-R_lecture.Rmd Week 4 R lecture materials] (.Rmd file)
*(No screencast for this week)


;[[Statistics_and_Statistical_Programming_(Fall_2020)/w01_session_plan|Session plan]]
'''Resources:'''


<blockquote>''Note: Aaron doesn't actually expect you to complete these before class on September 17''</blockquote>
* [https://www.openintro.org/download.php?file=os3_slides_04&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §4 Lecture Notes]
* [https://www.openintro.org/stat/videos.php OpenIntro Video Lectures] including 7 videos for nearly all of §4


'''Required'''
=== Week 5: Thursday May 2: Continuous Numeric Data & ANOVA ===
* Read this syllabus, discuss any questions/concerns with the teaching team.
* Complete [https://apps3.cehd.umn.edu/artist/user/scale_select.html pre-course assessment of statistical concepts] (access code TBA via email). Estimated time to do this is 30-40 minutes. '''Submission deadline: September 18, 11:00pm Chicago time'''
* Confirm course registration and access to [https://www.openintro.org/book/os/ the textbook] (pdf download available for $0 and b&w paperbacks for $20) as well as any software and web-services you'll need for course (Zoom, Discord, Canvas, this wiki, R, RStudio). Discord invites will be sent via email.
* Complete [https://wiki.communitydata.science/Statistics_and_Statistical_Programming_(Fall_2020)/pset0 problem set #0]


'''Recommended'''
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 5|Session plan]]
* Work through one (or more) introduction(s) to R and Rstudio so that you can complete problem set 0. Here are several suggestions:
** '''From Aaron:''' The [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w01-R_tutorial.html Week 01 R tutorial] (you should also download the [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w01-R_tutorial.rmd .rmd version of the tutorial] that you can open and read/edit in RStudio). These are accompanied by the R and Rstudio intro screencasts ([https://communitydata.cc/~ads/teaching/2019/stats/screencasts/w01-s01-intro.webm Part 1] and [https://communitydata.cc/~ads/teaching/2019/stats/screencasts/w01-s02-intro.webm Part 2]) Aaron created for the 2019 version of the course.
** Modern Dive [https://moderndive.netlify.app/index.html Statistical inference via data science] Chapter 1: [https://moderndive.netlify.app/1-getting-started.html Getting started with R].
** [https://rladiessydney.org/courses/ryouwithme/ RYouWithMe] course [https://rladiessydney.org/courses/ryouwithme/01-basicbasics-0/ "Basic basics" 1 & 2] (and maybe 3 if you're feeling ambitious).
** Verzani §1 (Getting started).
** Healy §2 (Get started).


=== Week 2 (9/22, 9/24) ===
'''Required Readings:'''
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w02_session_plan|Session plans]]
==== September 22: Data and variables ====
'''Required'''
* Read Diez, Çetinkaya-Rundel, and Barr: §1.1-1.3 (Introduction to data).
* Watch [https://www.youtube.com/playlist?list=PLkIselvEzpM6pZ76FD3NoCvvgkj_p-dE8 Lecture materials for §1.1-3 (Videos 1-4 in the playlist)].
* Submit, review, and respond to questions or requests for discussion via Discord or some other means.


==== September 24: Numerical and categorical data ====
* Diez, Barr, and Çetinkaya-Rundel: §5 (Inference for numerical data)
'''Required'''
<!---* Buechley, Leah and Benjamin Mako Hill. 2010. “LilyPad in the Wild: How Hardware’s Long Tail Is Supporting New Engineering and Design Communities.” Pp. 199–207 in ''Proceedings of the 8th ACM Conference on Designing Interactive Systems.'' Aarhus, Denmark: ACM. [[https://mako.cc/academic/buechley_hill_DIS_10.pdf PDF from Hill's website]]--->
* Read Diez, Çetinkaya-Rundel, and Barr: §2.1-2 (Numerical and categorical data).
* Sweetser, K. D., & Metzgar, E. (2007). Communicating during crisis: Use of blogs as a relationship management tool. ''Public Relations Review'', 33(3), 340–342. [[https://doi.org/10.1016/j.pubrev.2007.05.016 Available through NU Libraries]]
* Review [https://www.youtube.com/playlist?list=PLkIselvEzpM6pZ76FD3NoCvvgkj_p-dE8 Lecture materials for §2.1 and §2.2 (Videos 6-7 in the playlist)].
* Reinhart, §1
* Complete '''exercises from OpenIntro §2:''' 2.12, 2.13, 2.16, 2.20, 2.23, 2.30 (and remember that solutions to odd-numbered problems are in the book!)
* Submit, review, and respond to questions or requests for discussion via Discord or some other means.


=== Week 3 (9/29, 10/1) ===
'''Recommended Readings:'''
* Verzani: §9 (significance tests), §12 (Analysis of variance)
* Gelman, Andrew and Hal Stern. 2006. “The Difference Between ‘Significant’ and ‘Not Significant’ Is Not Itself Statistically Significant.” ''The American Statistician'' 60(4):328–31. [[http://dx.doi.org/10.1198/000313006X152649 Available through NU Libraries]]


;[[Statistics_and_Statistical_Programming_(Fall_2020)/w03_session_plan|Session plans]]
'''Assignment (Complete Before Class):'''


==== September 29: R fundamentals: Import, transform, tidy, and describe data ====
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 5]]
'''Required'''
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset1|problem set #1]] (due Monday, September 28 at 1pm Central)


'''Recommended'''
'''Lectures:'''
* [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w03-R_tutorial.html Week 3 R tutorial] (note that you can access .rmd or .pdf versions by replacing the suffix of the URL accordingly).
* No new R material for this week.
* Additional material from any of the recommended R learning resources suggested last week or elsewhere in the syllabus. In particular, you may find the ModernDive, RYouWithMe, Healy, and/or Wickham and Grolemund resources valuable.
<!---
<!---
'''Resources'''
* [[Statistics and Statistical Programming (Spring 2019)/R lecture outline: Week 5]]
* [https://science.sciencemag.org/content/187/4175/398 UCB admissions paper]
* [https://communitydata.cc/~mako/2017-COM521/com521-week_05-ttests_and_anova.ogv Week 5 R lecture screencast: t-tests] (~22 minutes)
* [https://openpolicing.stanford.edu Stanford OpenPolicing Project]
* [https://communitydata.cc/~mako/2017-COM521/com521-week_05-for_if.ogv Week 5 R lecture screencast: for loops and if statements] (~12 minutes)
--->
--->


==== October 1: Probability ====
'''Resources:'''
'''Required'''
 
* Read Diez, Çetinkaya-Rundel, and Barr: §3 (Probability).  
* [https://www.openintro.org/download.php?file=os3_slides_05&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §5 Lecture Notes]
* Watch [https://www.youtube.com/watch?list=PLkIselvEzpM5EgoOajhw83Ax_FktnlD6n&v=rG-SLQ2uF8U Probability introduction] and [https://www.youtube.com/watch?v=HxEz4ZHUY5Y&list=PLkIselvEzpM5EgoOajhw83Ax_FktnlD6n&index=2 Probability trees] OpenIntro lectures (just videos 1 and 2 in the playlist).
 
* Complete '''exercises from OpenIntro §3:''' 3.12, 3.15, 3.22, 3.28, 3.34, 3.38
=== Week 6: Thursday May 9: Categorical data ===
 
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 6|Session plan]]
'''Required Readings:'''
 
* Diez, Barr, and Çetinkaya-Rundel: §6.1-6.4 (Inference for categorical data).
* Buechley, Leah and Benjamin Mako Hill. 2010. “LilyPad in the Wild: How Hardware’s Long Tail Is Supporting New Engineering and Design Communities.” Pp. 199–207 in ''Proceedings of the 8th ACM Conference on Designing Interactive Systems.'' Aarhus, Denmark: ACM. [[https://mako.cc/academic/buechley_hill_DIS_10.pdf PDF available on Hill's personal website]]
* Reinhart, §4 and §5.
 
'''Recommended Readings:
* Diez, Barr, and Çetinkaya-Rundel: §6.5-6.6 (Small samples and randomization inference)
* Verzani: §3.4 (Bivariate categorical data); §10.1-10.2 (Goodness of fit)
* Gelman, Andrew and Eric Loken. 2014. “The Statistical Crisis in Science Data-Dependent Analysis—a ‘garden of Forking Paths’—explains Why Many Statistically Significant Comparisons Don’t Hold Up.” ''American Scientist'' 102(6):460. [[https://www.americanscientist.org/issues/pub/2014/6/the-statistical-crisis-in-science/1 Available through NU Libraries]] (This is a reworked version of [http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf this unpublished manuscript] which provides a more detailed examples.)
 
'''Assignment (Complete Before Class):'''
 
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 6]]
 
'''Lectures:'''
*[https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w06-R_lecture.Rmd Week 6 R lecture materials] (.Rmd file)
*(No screencast for this week)
 
'''Resources:'''
* [https://www.openintro.org/download.php?file=os3_slides_06&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §6 Lecture Notes]
* [https://www.openintro.org/stat/videos.php OpenIntro Video Lectures] including 4 videos for §7
 
=== Week 7: Thursday May 16: Linear Regression ===
* [[Statistics and Statistical Programming (Spring 2019)/Session plan: Week 7|Session plan]]
'''Required Readings:'''
 
* Diez, Barr, and Çetinkaya-Rundel: §7 (Introduction to linear regression)
* OpenIntro eschews a mathematical approach to correlation. Look over [https://en.wikipedia.org/wiki/Correlation_and_dependence the Wikipedia article on correlation and dependence] and pay attention to the formulas. It's tedious to compute, but you should be aware of what goes into it.
* Lampe, Cliff, and Paul Resnick. 2004. “Slash(Dot) and Burn: Distributed Moderation in a Large Online Conversation Space.” In ''Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04)'', 543–550. New York, NY, USA: ACM. doi:10.1145/985692.985761. [[http://dx.doi.org/10.1145/985692.985761 Available via NU libraries]]
 
'''Recommended Readings:'''
* Verzani: §11.1-2 (Linear regression).
* [https://seeing-theory.brown.edu/ Seeing Theory] §5 (Regression Analysis)
 
'''Assignment (Complete Before Class):'''
 
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 7]]
* Final project planning document (see details above!)
 
'''Lectures:'''
* [https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w07-R_lecture.Rmd Week 7 R lecture materials]
 
'''Resources:'''
* [https://www.openintro.org/download.php?file=os3_slides_07&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §7 Lecture Notes]
* [https://www.openintro.org/download.php?file=os3_slides_08&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §8 Lecture Notes]
* [https://www.openintro.org/stat/videos.php OpenIntro Video Lectures] including 4 videos for §7 and 3 videos on the sections §8.1-8.3
 
=== Week 8: Thursday May 23: Polynomial Terms, Interactions, and Logistic Regression ===
* [[Statistics_and_Statistical_Programming_(Spring_2019)/Session plan: Week 8|Session plan]]
 
'''Required Readings:'''
* Diez, Barr, and Çetinkaya-Rundel: §8 (Multiple and logistic regression)
* [https://onlinecourses.science.psu.edu/stat501/node/301 Lesson 8: Categorical Predictors] and [https://onlinecourses.science.psu.edu/stat501/node/318 Lesson 9: Data Transformations] from the PennState Eberly College of Science STAT 501 Regression Methods Course. There are several subparts (many quite short), please read them all carefully.
* (Revisit) Lampe, Cliff, and Paul Resnick. 2004. “Slash(Dot) and Burn: Distributed Moderation in a Large Online Conversation Space.” In ''Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04)'', 543–550. New York, NY, USA: ACM. doi:10.1145/985692.985761. [[http://dx.doi.org/10.1145/985692.985761 Available via NU libraries]]
* Reinhart, §8 and §9.


'''Resources'''
'''Recommended Readings:'''
* [https://seeing-theory.brown.edu/index.html#secondPage Seeing Theory §1-2 (Basic Probability and Compound Probability)]
* Verzani: §11.3 (Linear regression), §13.1 (Logistic regression)
* Ioannidis, John P. A. 2005. “Why Most Published Research Findings Are False.” ''PLoS Medicine'' 2(8):e124. [[http://dx.doi.org/10.1371%2Fjournal.pmed.0020124 Open Access]]
* Head, Megan L., Luke Holman, Rob Lanfear, Andrew T. Kahn, and Michael D. Jennions. 2015. “The Extent and Consequences of P-Hacking in Science.” ''PLOS Biology'' 13(3):e1002106. [[http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002106 Open Access]]


=== Week 4 (10/6, 10/8) ===
'''Assignment (Complete Before Class):'''
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w04_session_plan|Session plans]]


==== October 6: Emotional contagion and more advanced R fundamentals: import, tidy, transform, and simulate data; write functions ====
* [[Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 8]]
'''Required'''
* Read the paper below as well as the attendant [https://www.pnas.org/content/111/29/10779.1 "Expression of editorial concern"] and [https://www.pnas.org/content/111/29/10779.2 "Correction"] that were subsequently appended to it.
:Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. “Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks.” ''Proceedings of the National Academy of Sciences'' 111(24):8788–90. [[http://www.pnas.org/content/111/24/8788.full Open access]]
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset2|problem set #2]] (due Monday, October 5 at 1pm CT)


'''Recommended'''
'''Lectures:'''
* [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w04-R_tutorial.html Week 4 R tutorial] (as usual, also available as .rmd or .pdf)
*[https://communitydata.science/~ads/teaching/2019/stats/r_lectures/w08-R_lecture.Rmd Week 8 R lecture materials]


==== October 8: Distributions ====
'''Resources:'''
'''Required'''
* Read Diez, Çetinkaya-Rundel, and Barr: §4.1-3 (Normal and binomial distributions).
* Watch [https://www.youtube.com/watch?list=PLkIselvEzpM6V9h55s0l9Kzivih9BUWeW&v=S_p5D-YXLS4 normal and binomial distributions] OpenIntro lectures (videos 1-3 in the playlist).
* Complete '''exercises from OpenIntro §4:''' 4.4, 4.6, 4.15, 4.22


'''Resources'''
* [https://www.openintro.org/download.php?file=os3_slides_08&referrer=/stat/slides/slides_0x.php Mine Çetinkaya-Rundel's OpenIntro §8 Lecture Notes]
* [https://seeing-theory.brown.edu/index.html#secondPage/chapter3 Seeing Theory §3 (Probability distributions)]
* [https://www.openintro.org/stat/videos.php OpenIntro Video Lectures] including a video on §8.4
* Mako Hill wrote this document which will likely be useful for many of you: [https://communitydata.cc/~mako/2017-COM521/logistic_regression_interpretation.html Interpreting Logistic Regression Coefficients with Examples in R]


==== October 9: [[#Research project plan and dataset identification|Research project plan and dataset identification]] due by 5pm CT ====
=== Week 9: Thursday May 30: Loose ends and Final Presentations (part 1)  ===
*'''Submit via [https://canvas.northwestern.edu/courses/122522/assignments Canvas]''' (due by 5pm CT)


=== Week 5 (10/13, 10/15) ===
* [[Statistics_and_Statistical_Programming_(Spring_2019)/Session plan: Week 9|Session plan]]
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w05_session_plan|Session plans]]
==== October 13: Descriptive analysis and visualization of data ====
'''Required'''
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset3|problem set #3]] (due Monday, October 12 at 1pm CT)


'''Recommended'''
'''Required readings:'''
* [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w05-R_tutorial.html Week 5 R tutorial] and [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w05a-R_tutorial.html Week 5 R tutorial supplement] (both, as usual, also available as .rmd or .pdf).
* Reinhart, §10 and §11.


==== October 15: Foundations for (frequentist) inference ====
'''[[Statistics_and_Statistical_Programming_(Spring_2019)/Final_project_presentations|Final presentations]]: (part 1)'''
'''Required'''
* First batch today. The rest next week.
* Read Diez, Çetinkaya-Rundel, and Barr: §5 (Foundations for inference).
* Watch [https://www.youtube.com/watch?v=oLW_uzkPZGA&list=PLkIselvEzpM4SHQojH116fYAQJLaN_4Xo foundations for inference] (videos 1-3 in the playlist) OpenIntro lectures.
* Complete [https://www.openintro.org/book/stat/why05/ Why .05?] OpenIntro video/exercise.
* Complete '''exercises from OpenIntro §5:''' 5.4, 5.8, 5.10, 5.17, 5.30, 5.35, 5.36


'''Resources'''
'''Resources:'''
* Kelly M., [https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-9713.2013.00693.x Emily Dickinson and monkeys on the stair Or: What is the significance of the 5% significance level?] ''Significance'' 10:5. 2013.
* [https://communitydata.cc/~ads/teaching/2019/stats/r_lectures/w09-R_lecture.html Week 9 R-lecture] (we will use this in class)
* [https://seeing-theory.brown.edu/index.html#secondPage/chapter4 Seeing Theory §4 (Frequentist Inference)]


=== Week 6 (10/20, 10/22) ===
=== Week 10: Thursday June 6: Fully reproducible research example, Replications, Final Presentations (part 2), and wrap-up ===
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w06_session_plan|Session plans]]
==== October 20: Reinforced foundations for inference ====
'''Required'''
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset4|problem set #4]] 
* Read Reinhart, §1.
* Revisit the Kramer et al. (2014) paper we read a few weeks ago:
:Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. “Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks.” ''Proceedings of the National Academy of Sciences'' 111(24):8788–90. [[http://www.pnas.org/content/111/24/8788.full Open access]] 


==== October 22: Inference for categorical data ====
* Fully [https://www.overleaf.com/read/tkdpdcspwtkp reproducible research example].
'''Required'''
* [https://canvas.northwestern.edu/courses/90927/files/folder/resources/Straub-Cook%20Replication Research replication study] by Polly Straub-Cook (UW Comm. Ph.D. student)
* Read Diez, Çetinkaya-Rundel, and Barr: §6 (Inference for categorical data).
:: (n.b.: cluster & heteroscedasticity robust standard errors!)
* Watch [https://www.youtube.com/watch?list=PLkIselvEzpM5Gn-sHTw1NF0e8IvMxwHDW&v=_iFAZgpWsx0 inference for categorical data] (videos 1-3 in the playlist) OpenIntro lectures.
* Complete '''exercises from OpenIntro §6:''' 6.10, 6.16, 6.22, 6.30, 6.40 (just parts a and b; part c gets tedious)


'''Resources'''
* '''[[Statistics_and_Statistical_Programming_(Spring_2019)/Final_project_presentations|Final presentations]]: (part 2)'''
* [https://gallery.shinyapps.io/CLT_prop/ OpenIntro Central limit theorem for proportions demo].
:: Second batch of presenters today.
* Closing thoughts
:: What next? Beyond your final projects...
:: Class social gathering


=== Week 7 (10/27, 10/29) ===
Followed by much rejoicing!
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w07_session_plan|Session plans]]
==== October 27: Applied inference for categorical data ====
'''Required'''
* Read Reinhart, §4 and §5 (both are quite short).
* Skim the following (all are referenced in the problem set)
**  Aronow PM, Karlan D, Pinson LE. (2018). The effect of images of Michelle Obama’s face on trick-or-treaters’ dietary choices: A randomized control trial. PLoS ONE 13(1): e0189693. [https://doi.org/10.1371/journal.pone.0189693 https://doi.org/10.1371/journal.pone.0189693]
** Buechley, Leah and Benjamin Mako Hill. 2010. “LilyPad in the Wild: How Hardware’s Long Tail Is Supporting New Engineering and Design Communities.” Pp. 199–207 in ''Proceedings of the 8th ACM Conference on Designing Interactive Systems.'' Aarhus, Denmark: ACM. [[https://mako.cc/academic/buechley_hill_DIS_10.pdf PDF available on Hill's personal website]]
** Shaw, Aaron and Yochai Benkler. 2012. A tale of two blogospheres: Discursive practices on the left and right. ''American Behavioral Scientist''. 56(4): 459-487. [[https://doi.org/10.1177%2F0002764211433793 available via NU libraries]]
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset5|problem set #5]]
'''Resources'''
* [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w06-R_tutorial.html Week 06 R tutorial] (it's very short!)


==== October 29: Inference for numerical data (part 1) ====
== Policies ==
'''Required'''
* Read Diez, Çetinkaya-Rundel, and Barr: §7.1-3 (Inference for numerical data: differences of means).
* Watch [https://www.youtube.com/watch?list=PLkIselvEzpM5G3IO1tzQ-DUThsJKQzQCD&v=uVEj2uBJfq0 inference for numerical data] (videos 1-4 in the playlist) OpenIntro lectures (and featuring one of the textbook authors!).
* Complete '''exercises from OpenIntro §7:''' 7.12, 7.24, 7.26


'''Resources'''
=== Attendance ===
* [https://gallery.shinyapps.io/CLT_mean/ OpenIntro Central limit theorem for means demo].


==== October 30: [[#Research project planning document|Research project planning document]] due 5pm CT====
Attendance in class is expected of all participants. If you need to miss class for any reason, please contact me ahead of time (email is best). Multiple unexplained absences will likely result in a lower grade or (in extreme circumstances) a failing grade. In the event of an absence, you are responsible for obtaining class notes, handouts, assignments, etc. You are also still responsible for turning in any assignments on time unless you make prior arrangements with me.
* Submit via [https://canvas.northwestern.edu/courses/122522/assignments/787297 Canvas] (due by 5pm CT)


=== Week 8 (11/3, 11/5) ===
=== In-class device usage ===
==== November 3: U.S. election day (no class meeting) ====


==== November 4: Interactive self-assessment due ====
Please refrain from any uses of digitally networked devices or other distraction machines that do not directly contribute to your engagement with the course material. If you struggle to comply with this policy, I may recommend you temporarily put away your device(s) or leave the classroom.
* Please submit results [https://canvas.northwestern.edu/courses/122522/assignments/799630 (via Canvas)] from the [https://communitydata.science/~ads/teaching/2020/stats/assessment/interactive_assessment.rmd interactive self-assessment] by 5pm CT.


==== November 5: Inference for numerical data (part 2) ====
=== Peers’ Work and In-Class Discussions ===
'''Required'''
* Read Diez, Çetinkaya-Rundel, and Barr: §7.4-5 (Inference for numerical data: power calculations, ANOVA, and multiple comparisons).
* Watch [https://www.youtube.com/watch?list=PLkIselvEzpM5G3IO1tzQ-DUThsJKQzQCD&v=uVEj2uBJfq0 inference for numerical data] (videos 4-8 in the playlist) OpenIntro lectures (and featuring one of the textbook authors!).
* Complete '''exercises from OpenIntro §7:''' 7.42, 7.44, 7.46


'''Resources'''
Throughout the course, you may receive, read, collaborate, and/or comment on classmates’ work. These assignments are for class use only. You may not share them with anybody outside of class without explicit written permission from the document’s author and pertaining to the specific piece.
* [https://www.openintro.org/go/?id=stat_better_understand_anova&referrer=/book/os/index.php OpenIntro supplement on ANOVA calculations] (useful if you think you'll be doing more ANOVAs).


=== Week 9 (11/10, 11/12) ===
It is essential to the success of this class that all participants feel comfortable discussing questions, thoughts, ideas, fears, reservations, apprehensions and confusion about works-in-progress, statistical concepts, independent research, and more. Therefore, you may not create any audio or video recordings during class time nor share verbatim comments with those not in class nor are you allowed to share using other methods -- e.g., social media -- any comments linked to people’s identities unless you get clear and explicit permission. If you want to share general impressions or specifics of in-class discussions with those not in class, please do so without disclosing personal identities or details.
==== November 10: Applied inference for numerical data (t-tests, power analysis, ANOVA) ====
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w09_session_plan|Session plans]]


'''Required'''
=== Academic Integrity ===
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset6|problem set #6]]


'''Resources'''
You are responsible for reading and abiding by the Northwestern University [https://www.northwestern.edu/provost/policies/academic-integrity/principles.html Principles Regarding Academic Integrity]. Personally, I expect you to exceed the minimal standards elaborated in those principles and to strive for admirable, extraordinary conduct in every aspect of your academic career. Feel free to ask me (the instructor) for clarification about this or related matters.
* [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w09-R_tutorial.html Week 09 R tutorial]


==== November 12: Linear regression ====
=== Deadlines ===  
'''Required'''
* Read Diez, Çetinkaya-Rundel, and Barr: §8 (Linear regression).
* Watch [https://www.youtube.com/playlist?list=PLkIselvEzpM63ikRfN41DNIhSgzboELOM linear regression] (videos 1-4 in the playlist) OpenIntro lectures.
* Read [https://www.openintro.org/go/?id=stat_more_inference_for_linear_regression&referrer=/book/os/index.php More inference for linear regression] (OpenIntro supplement).
* Complete '''exercises from OpenIntro §8:''' 8.6, 8.36, 8.40, 8.44
* Complete '''exercises from OpenIntro supplement:''' 4 and 5 (answers provided in the supplement).
'''Resources'''
* [https://seeing-theory.brown.edu/index.html#secondPage/chapter6 Seeing Theory §6 (Regression analysis)]


=== Week 10 (11/17, 11/19) ===
Emergencies happen. Unanticipated obstacles arise. If you cannot make a deadline, please contact me to figure out a schedule that will work. The more proactive and responsible you are, the more receptive I am likely be.
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w10_session_plan|Session plans]]
==== November 17: Applied linear regression ====
'''Required'''
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset7|Problem set #7]]


'''Resources'''
A word about extensions and incompletes: I strongly discourage them. In principle, I have no problem with extensions or incompletes. In practice, they tend to be a pain for everybody involved. If you absolutely must submit an assignment late, assume that I may require up to 1 month (4 weeks) to grade it. Please take this into account if you will need me to to submit a grade in order to receive your fellowship/diploma/visa/etc. by a particular date.
* [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/w10-R_tutorial.html Week 10 R tutorial]
==== November 19: Multiple and logistic regression ====
'''Required'''
* Read Diez, Çetinkaya-Rundel, and Barr: §9 (Multiple and logistic regression). (Skim §9.2-9.4)
** '''Disclaimer:''' Aaron doesn't like §9.2-9.3, but it should be useful to understand and discuss them, so we'll do that.
* Watch [https://www.youtube.com/playlist?list=PLkIselvEzpM5f1HYzIjFt52SD4izsJ2_I multiple and logistic regression] (videos 1-4 in the playlist) OpenIntro lectures.
* Read [https://www.openintro.org/go/?id=stat_interaction_terms&referrer=/book/os/index.php Interaction terms] (OpenIntro supplement).
* Read [https://www.openintro.org/go/?id=stat_nonlinear_relationships&referrer=/book/os/index.php Fitting models for non-linear trends] (OpenIntro supplement).
* Complete '''exercises from OpenIntro §9:''' 9.4, 9.13, 9.16, 9.18,


'''Resources'''
=== Accommodations ===


=== Week 11 (11/24) ===
I am totally happy to provide accommodations for religious observance, physical needs, or other circumstances as needed. Any student requesting accommodations related to a disability or other condition is required to register with AccessibleNU (847-467-5530) and provide professors with an accommodation notification from AccessibleNU, preferably within the first two weeks of class. All information will remain confidential. For more information, visit [https://www.northwestern.edu/accessiblenu/ AccessibleNU].
==== November 24: Applied multiple and logistic regression ====
;[[Statistics_and_Statistical_Programming_(Fall_2020)/w11_session_plan|Session plans]]
'''Required'''
* Complete [[Statistics_and_Statistical_Programming_(Fall_2020)/pset8|Problem set #8]]
'''Resources'''
* Mako Hill created (and Aaron updated) a brief tutorial on [https://communitydata.science/~ads/teaching/2020/stats/r_tutorials/logistic_regression_interpretation.html interpreting logistic regression coefficients with examples in R]


=== Week 12+ ===
=== Sexual Misconduct ===  


==== December 3: [[#Research project presentation|Research project presentation]] due by 5pm CT ====
All participants in this class are bound by the [https://www.northwestern.edu/sexual-misconduct/title-IX/university-policies/policy-on-sexual-misconduct.html Northwestern University sexual misconduct policy] Please note, that the core of the policy states, "Northwestern is committed to fostering an environment in which all members of our community are safe, secure, and free from sexual misconduct of any form, including, but not limited to, sexual assault, sexual exploitation, stalking, and dating and domestic violence." I take this very seriously. Please review the policy and speak to me if you have any questions or concerns.
'''[https://canvas.northwestern.edu/courses/122522/discussion_topics/856868 Post your video via this "Discussion" on Canvas]'''. Please view and provide constructive feedback on other's videos!


* '''Post videos directly to the "Discussion."''' The Canvas text editor has an option to upload/record a video. That's what you want.
=== Email protocol ===
* '''Please remember not to over-work/think this.''' I mentioned this in class, but just to reiterate, the focus of this assignment should not be your video editing skills. Please do what you can to record and convey your ideas clearly without devoting insane hours to creating the perfect video.
* '''Some resources for recording presentations:''' There are a bunch of ways you might record/share your video. Some ideas include using the embedded media recorder in Canvas (!) that can record with with your webcam (maybe attach a few visuals to accompany this?); recording a "meeting" with yourself in Zoom; and "Panopto," a piece of high-end video recording, sharing, and editing software that NU licenses for campus use. Here are some pointers:
** NU has a "digital learning resource hub" which provides some [https://digitallearning.northwestern.edu/resource-hub#for-students resources for students]. The first item in that list has pointers for recording yourself and posting to Canvas and includes info about the Canvas media recorder and Panopto.
** You should be able to use your NU zoom account to create a zoom meeting, record your meeting (in which you deliver your presentation and share your screen with any visuals), and then share a link to the recording via the "Recordings" item in the left-hand menu of your [https://northwestern.zoom.us/ https://northwestern.zoom.us/] account page.
** If nothing works, please get in touch.


==== December 4: Post-course assessment of statistical concepts due by 11pm CT ====
I receive too much email and I sometimes fail to keep up. If, for some reason, I do not respond to a message related to this course within 48 hours, please do not take it personally and feel free to re-send the message with a polite reminder. This will help me and I will not resent you for it.
Complete [https://apps3.cehd.umn.edu/artist/user/scale_select.html post-course assessment] (access code TBA VIA email). Submission deadline: December 4, 11:00pm Chicago time.


==== December 10: [[#Research project paper|Research project paper]] due by 5pm CT ====
'''[https://canvas.northwestern.edu/courses/122522/assignments/812317 Submit your paper, data, and code via Canvas].'''


== Credit and Notes ==
=== Credit and Notes ===


This syllabus has, in ways that should be obvious, borrowed and built on the [https://www.openintro.org/stat/index.php OpenInto Statistics curriculum]. Most aspects of this course design extend Benjamin Mako Hill's [[Statistics_and_Statistical_Programming_(Winter_2017)|COM 521 class]] from the University of Washington as well as a [[Statistics_and_Statistical_Programming_(Spring_2019)|prior iteration of the same course]] offered at Northwestern in Spring 2019.
This syllabus has, in ways that should be obvious, borrowed and built on the [https://www.openintro.org/stat/index.php OpenInto Statistics curriculum]. I also based nearly every aspect of the course design on Benjamin Mako Hill's [[Statistics_and_Statistical_Programming_(Winter_2017)|COM 521 class]].
Please note that all contributions to CommunityData are considered to be released under the Attribution-Share Alike 3.0 Unported (see CommunityData:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To protect the wiki against automated edit spam, we kindly ask you to solve the following CAPTCHA:

Cancel Editing help (opens in new window)