Statistics and Statistical Programming (Fall 2020)


 * Statistics and Statistical Programming
 * Media, Technology & Society (MTS) 525
 * Tuesdays & Thursdays 10-11:50am CT (synchronous sessions)
 * Fall 2020
 * Northwestern University


 * Instructor: Aaron Shaw ([mailto:aaronshaw@northwestern.edu aaronshaw@northwestern.edu])
 * Office Hours: Thursday 12-2pm and by appointment
 * Meeting location: Jitsi


 * Teaching Assistant: Nick Vincent ([mailto:nickvincent@u.northwestern.edu nickvincent@u.northwestern.edu])
 * Office Hours: I'll try to respond to any asynchronous questions in a timely fashion during "business hours" (9a-5p Central Time), and will also have OH by appointment. I'll also try to schedule some fixed time during which I'll hang out on a video call, hours TBA.
 * I'll likely use whatever conference we use for class sessions, but am happy to try out alternative communication software for OH!


 * Course Websites:
 * We will use Canvas for announcements, turning in most assignments, and maybe discussions the other possibility is Discord.
 * Everything else will be linked on this page.

Overview and learning objectives
This course provides a get-your-hands-dirty introduction to inferential statistics and statistical programming mostly for applications in the social sciences and social computing. My main objectives are for all participants to acquire the conceptual, technical, and practical skills to conduct your own statistical analyses and become more sophisticated consumers of quantitative research in communication, human computer interaction (HCI), and adjacent disciplines.

I will consider the course a complete success if every student is able to do all of the following things at the end of the quarter:
 * Design and execute a quantitative research project that involves statistical inference, start to finish.
 * Read, modify, and create short programs in the R statistical programming language.
 * Feel comfortable reading and interpreting papers that use basic statistical techniques.
 * Feel prepared to enroll in more specialized and advanced statistics courses.

The course will cover a number of techniques, likely including the following: t-tests; chi-squared tests; ANOVA; linear regression; and logistic regression. We will also consider salient issues in quantitative research such as reproducibility and "the statistical crisis in science." We may cover other topics as time and interest allow.

The course materials will consist of readings, problem sets, assessment exercises, and recorded lectures and screencasts (some created by me, some created by other people). The course requirements will emphasize active participation, self-evaluation, and will include a final project focused on the design and execution of an original piece of quantitative research. We will use the R programming language for all examples and assignments.

You are not required to know much about statistics or statistical programming to take this class. I will assume some (very little!) knowledge of the basics of empirical research methods and design, basic algebra and arithmetic, and a willingness to work to learn the rest. In general we are not going to cover most of the math behind the techniques we'll be learning. Although we may do some math, this is not a math class. This course will also not require knowledge of calculus or matrix algebra. I will *not* do proofs on the board. Instead, the class is unapologetically focused on the application of statistical methods. Likewise, while some exposure to R, other programming languages, or other statistical computing resources will be helpful, it is not assumed.

'''Why this course? Why statistical programming? Why R?'''

Many comparable courses in statistics and quantitative methods do not emphasize statistical programming. So why bother? By learning statistical programming you will gain a deeper understanding of both the principles behind your analysis techniques as well as the tools you use to apply those techniques. In addition, a solid grasp of statistical programming will prepare you to create reproducible research, avoid common errors, and enable both greater durability and validity of your work.

Other programming languages are also well suited to statistics, including Stata and Python. I do most of my work with R, so that guides my choice for the course. That said, I opt to use and teach with R for a few reasons:
 * R is freely available and open source.
 * R is the most widely used package in statistics and several social scientific fields.
 * R (along with Stata) will be used in most of the advanced stats classes I hope you will take after this course.
 * R is better general purpose programming language than Stata which means that R programming skills will let you solve non-statistical problems and may make it easier to learn other programming languages like Python.

Format and structure
This course will proceed in a remote format that includes asynchronous and synchronous elements (more on those below). In general, the organization of the course adopts a "flipped" approach where you consume instructional materials on your own or in groups and we use synchronous meetings to answer questions, address challenges or concerns, work through solutions, and hold semi-structured discussions.

The course introduces both basic statistical concepts as well as basic applications of those concepts through statistical programming. As a result, we will dedicate part of each week to a particular set of concepts and part of each week to applied data analysis and/or interpretation. A brief description of how I expect it all to work follows below. We'll talk about it more during the first class session.

Asynchronous elements of the course
These include all readings, recorded lectures/slides, tutorials, problem sets, and other assignments. I expect you to complete (or at least attempt to complete!) these outside of our class meeting times. For nearly all of the "instructional" material introducing particular statistical concepts and techniques, you are expected to use the OpenIntro textbook and lecture materials created by the textbook authors. Please note that this means I will not deliver lectures during our class meetings. Please also note that this means you are responsible for coordinating your problem set groups and any collaborative work with other members of the class outside of our class meeting times.

Synchronous elements of the course
The synchronous elements of the course will be the two weekly class meetings that will happen via video conference (platform TBD). These are scheduled to run for a maximum of 110 minutes. Each session will include multiple short breaks.

We will use the class meetings to discuss and work through any questions or challenges you encounter in the materials assigned for that day. This means that I will ask you to submit completed problem sets and questions the night before each class meeting. Doing so will give the teaching team time to sift, sort, and organize your submissions into a hopefully-cohesive plan for each class session that is tailored to the specific questions and concerns you encounter in the material.

Textbook, readings, and resources
This class will use a freely-licensed textbook:


 * Diez, David M., Christopher D. Barr, and Mine Çetinkaya-Rundel. 2019. OpenIntro Statistics. 4th edition. OpenIntro, Inc.

The texbook (in any format) is required for the course. You can download it at no cost and purchase hard copy versions in either full color ($60) or in black and white ($20). The B&W version is very affordable and I strongly recommend buying a hard copy for the purposes of the course and subsequent reference use. The book is excellent and has been adopted widely. It has also developed a large online community of students and teachers who have shared other resources. Lecture slides, videos, notes, and more are all freely licensed (many through the website and others elsewhere).

I will also assigning several chapters from the following:


 * Reinhart, Alex. 2015. Statistics Done Wrong: The Woefully Complete Guide. SF, CA: No Starch Press. (Safari online via NU libraries)

This book provides a readable conceptual introduction to some common failures in statistical analysis that you should learn to recognize and avoid. It was also written by a Ph.D. student. You have access to an electronic copy via the NU library (you'll need to sign-in and/or use the NU VPN to access it), but you may find it helpful to purchase as well.

A few other books may be useful resources while you're learning to analyze, visualize, and interpret statistical data with R. I will share some advice about these during the first class meeting:


 * Healy, Kieran. 2019. Data Visualization: A Practical Introduction. Princeton, NJ: Princeton UP. (via Healy's website)
 * Teetor, Paul. 2011. R Cookbook: Proven Recipes for Data Analysis, Statistics, and Graphics. 1 edition. Sebastopol, CA: O’Reilly Media. (Safari Proquest/NU Libraries; Various Sources; Amazon)
 * Verzani, John. 2014. Using R for Introductory Statistics, Second Edition. 2 edition. Boca Raton: Chapman and Hall/CRC. (Various Sources; Amazon)
 * Wickham, Hadley. 2010. ggplot2: Elegant Graphics for Data Analysis. 1st ed. 2009. Corr. 3rd printing 2010 edition. New York: Springer. (Springer/NU Libraries; Various Sources)

There are also some invaluable non-textbook resources:


 * Baggott's R Reference Card v2 — Print this out. Take it with you everywhere and look at it dozens of times a day. You will learn the language faster!
 * StackOverflow R Tag — Somebody already had your question about how to do X in R. They asked it, and several people have answered it, on StackOverflow. Learning to read this effectively will take time but as build up some basic familiarity with R and with StackOverflow, it will get easier. I promise.
 * Rseek — Rseek is a modified version of Google that just searches R websites online. Sometimes, R is hard to search because R is a common letter. This has become much easier over time as R has become more popular, but it can still be an issue sometimes and Rseek is a good solution.
 * ggplot2 documentation — ggplot is a powerful data visualization package for R that I recommend highly. The documentation is indispensable for learning how to use it.
 * Statistical Analysis and Reporting in R — A set of resources created and distributed by Jacob Wobbrock (University of Washington, School of Information) in conjunction with a MOOC he teaches. Contains cheatsheets, code snippets, and data to help execute commonly encountered statistical procedures in R.
 * DataCamp offers introductory R courses. Northwestern usually has some free accounts that get passed out via Research Data Services each quarter. Apparently, if you are taking or teaching relevant coursework, instructors can request free access to DataCamp for their courses from DataCamp. If folks are interested in this, I can reach out.

Computing resources:
 * If you are planning to analyze large-scale data (i.e., data that won't fit in memory on your laptop) then you will want to sign up for a research allocation on Quest, which is Northwestern's high-performance computing cluster. Instructions on how to do that are here.

Assignments
The assignments in this class focus on applied statistical research design, analysis, and interpretation. Unless otherwise noted, all assignments are due at the end of the day (i.e., 11:59pm on the day they are due).

Weekly problem sets and participation
Each week I will post a problem set incorporating three kinds of questions:


 * Statistics questions about statistical concepts, principles, and interpretation.
 * Programming challenges that you must solve using R.
 * Empirical paper questions about other assigned readings.

Some of these (usually just the statistics questions) will be taken from the textbooks and some will not. In general, we will cover the statistical concepts and principles in the first session of the week and the empirical paper questions in the second session. The programming material will likely span both sessions depending on the week. You will need to submit your solutions to the relevant questions ahead of the relevant class session. Details of exactly how this will work will be provided in the course schedule and we'll go over them during the first class.

Throughout the course you will be assigned to a (rotating) problem set group. This will be a group of 2-3 students (exact numbers will depend on the final enrollment) with whom you will meet outside of class time to discuss, complete, and/or review your problem sets. The groups will change roughly every two weeks during the quarter to ensure that you get to work with different members of the class. The main idea is to support collaborative learning, peer teaching, and accountability. While the specifics of exactly when and how you work with your problem set group will largely be up to you, the teaching team will provide a template that you can use as a starting point.

Because randomness is extremely important in statistics, I will use a small R program to randomly assign different problem set groups to share and discuss their solutions to select questions during class sessions. These assignments will be announced at least a few days ahead of time so that the group has an opportunity to prepare. The idea here is not to put people on the spot, but to ensure an equitable distribution of the responsibility to discuss questions, answers, points of confusion, and alternatives.

For the programming challenges, you should submit code for your solutions (more on how in a moment) so we can walk through the material together. If you get completely stuck on a problem, that's okay, but please share whatever code you have so that you can tell us what you did and what you were thinking.

Attendance in the synchronous portion of the class will be important to supporting your mastery of the material. Although the problem sets will not be assigned a letter grade, it is critical that you be present and able to discuss your answers to each of the questions. Your ability to do so will figure prominently in your participation grade for the course (see the section on grading and assessment below).

Research project
As a demonstration of your learning in this course, you will design and carry out a quantitative research project, start to finish. This means you will all:


 * Design and describe a plan for a study — The study you design should involve quantitative analysis and should be something you can complete at least a first pass on during this quarter.
 * Find a dataset — Very quickly, you should identify a dataset you will use to complete this project. For most of you, I suspect you will be engaging in secondary data analysis or a analysis of a previously collected dataset.
 * Engage in descriptive data analysis — Use R to calculate descriptive statistics and visualizations to describe your data.
 * Motivate and test at least one hypothesis about relationships between two or more variables — I'm happy to discuss alternatives to formal hypothesis testing procedures (even if some of them are beyond the scope of this course).
 * Report and interpret your findings — You will do this in both a short paper and a short (recorded) presentation.
 * Ensure that your work is replicable — You will need to provide code and data for your analysis in a way that makes your work replicable by other researchers.

I strongly urge you to produce a project that will further your academic career outside of the class. There are many ways that this can happen. Some obvious options are to prepare a project that you can submit for publication, use as pilot analysis that you can report in a grant or thesis proposal, and/or use to fulfill a degree requirement.

There are several intermediate milestones and deadlines to help you accomplish a successful research project. Unless otherwise noted, all deliverables should be submitted via Canvas.

Project plan and dataset identification

 * Due date: TBA
 * Maximum length: 500 words (~1-2 pages)

Early on, I want you to identify and describe your final project. Your description should be short and can be either paragraphs or bullets. It should include the following:


 * An abstract of the proposed study including the topic, research question, theoretical motivation, object(s) of study, and anticipated research contribution.
 * An identification of the dataset you will use and a description of the rows and columns or type(s) of data it will include. If you do not currently have access to these data, explain why and when you will.
 * A short (several sentences?) description of how the project will fit into your career trajectory.

Notes on finding a dataset
In order to complete your final project, you will each need a dataset. If you already have a dataset for the project you plan to conduct, great! If not, fear not! There are many datasets to draw from. Some ideas are below (please suggest others, provide updated links, or report problems). The teaching team will also be available to help you brainstorm/find resources if needed:


 * Ask your advisor for a dataset they have collected and used in previous papers. Are there other variables you could use? Other relationships you could analyze?
 * If there's an important study you loved, you can send a polite email to the author(s) asking if they are willing and able to share an archival or replication version of the dataset used in their paper. Be very polite and make it clear that this is starting as a class project, but that it might turn into a paper for publication. Make your timeline clear. In Communication and HCI, replication datasets are still very rare, so be prepared for a negative answer and/or questions about your motives in conducting the analysis.
 * Do some Google Scholar and normal internet searching for datasets in your research area. You'll probably be surprised at what's available.
 * Take a look at datasets available in the Harvard Dataverse (a very large collection of social science research data) or one of the other members of the Dataverse network.
 * Look at the collection of social scientific datasets at ICPSR at the University of Michigan (NU is a member). There are an enormous number of very rich datasets.
 * Use the ISA Explorer to find datasets. Keep in mind the large majority of datasets it will search are drawn from the natural sciences.
 * The City of Chicago has one of the best data portal sites of any municipality in the U.S. (and better than many federal agencies). There are also numerous administrative datasets released by other public entities (try searching!) that you might find inspiring.


 * FiveThirtyEight.com has published a GitHub repository and an R package with pre-processed and cleaned versions of many of the datasets they use for articles published on their website.
 * If you interested in studying online communities, there are some great resources for accessing data from Reddit, Wikipedia, and StackExchange. See pushshift for dumps of Reddit data, here for an overview of Wikipedia's data resources, and Stack Exchange's data portal.

Project planning document

 * Due date: TBA
 * Maximum length: ~5 pages

The project planning document is a basic shell/outline of an empirical quantitative research paper. Your planning document should should have the following sections: (a) Rationale, (b) Objectives; (b.1) General objectives; (b.2) Specific objectives; (c) (Null) hypotheses; (d) Conceptual diagram and explanation of the relationship(s) you plan to test; (e) Measures; (f) Dummy tables/figures; (g) anticipated finding(s) and research contribution(s). Longer descriptions of each of these planning document sections (as well as a few others) can be found on this wiki page.

I will also provide three example planning documents via our Canvas site (links to-be-updated for 2020 edition of the course):
 * One by public health researcher Mika Matsuzaki. The first planning document I ever saw and still one of the best. It's missing a measures section. It's also focused on a research context that is probably very different from yours, but try not to get bogged down by that and imagine how you might map the structure of the document to your own work.
 * One by Jim Maddock created as part of a qualifying exam early in 2019. Jim doesn't provide dummy tables or anticipated findings/contributions, but he has an especially phenomenal explanation of the conceptual relationships and processes he wants to test.
 * One provided as an appendix to Gerber and Green's excellent textbook, Field Experiments: Design, Analysis, and Interpretation (FEDAI). It's over-detailed and incredibly long for our purposes, but nevertheless an exemplary approach to planning empirical quantitative research in a careful, intentional way that is worthy of imitation.

Research paper

 * Paper due date: TBA
 * Maximum length: 6000 words (~20 pages)

I expect you to produce a short, high quality research paper that you might revise, extend, and submit for publication and/or a dissertation milestone. I do not expect the paper to be ready for publication, but it should contain polished drafts of all the necessary components of a scholarly quantitative empirical research study. In terms of the structure, please see the page on the structure of a quantitative empirical research paper.

As noted above, you should also provide data, code, and any documentation sufficient to enable the replication of all analysis and visualizations. If that is not possible/appropriate for some reason, please talk to me so that we can find another solution.

Because the emphasis in this class is on statistics and methods and because I'm probably not an expert in the substance of your research domain, I'm happy to assume that your paper, proposal, or thesis chapter has already established the relevance and significance of your study and has a comprehensive literature review, well-grounded conceptual approach, and compelling reason why this research is important. As a result, you need not focus on these elements of the work in your written submission. Instead, feel free to start with a brief summary of the purpose and importance of this research followed by an introduction of your research questions or hypotheses. If you provide more detail, that's fine, but I won't give you detailed feedback on these parts and they will not figure prominently in my assessment of the work.

I have a strong preference for you to write the paper individually, but I'm open to the idea that you may want to work with others in the class. Please contact me before you attempt to pursue a collaborative final paper.

I do not have strong preferences about the style or formatting guidelines you follow for the paper and its bibliography. However, your paper must follow a standard format (e.g., ACM SIGCHI CSCW format or APA 6th edition (Word and LaTeX templates)) that is applicable for a peer-reviewed journal or conference proceedings in which you might aim to publish the work (they all have formatting or submission guidelines published online and you should follow them). This includes the references. I also strongly recommend that you use reference management software like Zotero to handle your bibliographic sources.

Project presentation

 * Presentation due date: TBA
 * Maximum length: 7 minutes

You will also create and record a short (7-8 minute) presentation of your final project. The presentation will provide an opportunity to share a brief summary of your project and at least preliminary findings with the other members of the class. Since you will all give other research presentations throughout your career, I strongly encourage you to take the opportunity to refine your academic presentation skills. The document Creating a Successful Scholarly Presentation (file will be posted to Canvas) may be useful.

More details about the presentation goals, format suggestions, and more will be provided later in the quarter.

Human subjects research, IRB, and ethics
In general, you are responsible for making sure that you're on the right side of the IRB requirements and that your work meets applicable ethical norms and standards.

Class projects generally do not need IRB approval, but research for publications, dissertations, and sometimes even pilot studies do fall under IRB purview. You should not plan to seek IRB approval/determination retroactively. If your study may involve human subjects and you may ever publish it in any form, you will need IRB oversight of some sort.

Secondary analysis of anonymized data is generally not considered human subjects research, but I strongly suggest that you get a determination from the Northwestern IRB before you start. For work that is not considered human subjects research, this can often happen in a few hours or days. If you need to list a faculty sponsor or Principal Investigator, that should ideally be your advisor. If that doesn't make sense for some reason, please talk to me.

Research ethics are broad and complex topic. We'll talk about issues related to ethics and quantitative empirical research a bit more during class, but will likely only scratch the surface. I strongly encourage you to pursue further reading, conversation, coursework, and reflection as you consider how to understand and apply ethical principles in the context of your own research and teaching.

Grading and assessment
I will assign grades (usually a numeric value ranging from 0-10) for each of the following aspects of your performance. The percentage values in parentheses are weights that will be applied to calculate your overall grade for the course.


 * Weekly participation (includes problem sets): 40%
 * Proposal identification: 5%
 * Final project planning document: 5%
 * Final project presentation: 10%
 * Final project paper: 40%

The teaching team will jointly evaluate your participation along four dimensions: attendance, preparation, engagement, and contribution. These are quite similar to the dimensions described in the "Participation Rubric" section of Benjamin Mako Hill's assessment page and Joseph Reagle's participation assessment rubric. Exceptional participation means excelling along all four dimensions. Please note that participation ≠ talking more and I encourage all of us to seek balance in our discussions.

The teaching team's assessment of your final project proposal, planning document, presentation, and paper will reflect the clarity of the work, the effective execution and presentation of quantitative empirical analysis, as well as the quality and originality of the analysis. A more detailed assessment rubric will be provided. Throughout the quarter, we will talk about the qualities of exemplary quantitative research. In general, I expect your final project to embody these exemplary qualities.

General course policies
General policies on a wide variety of topics including classroom equity, attendance, academic integrity, accommodations, late assignments, and more are provided on my class policies page. Below are some policy statements specific to this course and quarter.

Teaching and learning in a pandemic
The Covid-19 pandemic will impact this course in various ways, some of them obvious and tangible and others harder to pin down. On the obvious and tangible front, we have things like a mix of remote and (a)synchronous instruction, the fact that many of us will not be anywhere near campus or each other this year, and the unusual academic calendar. These will reshape our collective "classroom" experience in major ways.

On the "harder to pin down" side, many of us may experience elevated levels of exhaustion, stress, uncertainty and/or distraction. We may need to provide unexpected support to family, friends, or others in our communities. I have personally experienced all of these things at various times over the past six months and I expect that some of you have too. It is a difficult time.

I believe it is important to acknowledge these realities of the situation and create the space to discuss and process them in the context of our class throughout the quarter. As your instructor and colleague, I commit to do my best to approach the course in an adaptive, generous, and empathetic way. I will try to be transparent and direct with you throughout—both with respect to the course material as well as the pandemic and the university's evolving response to it. I ask that you try to extend a similar attitude towards everyone in the course. When you have questions, feedback, or concerns, please try to share them in an appropriate way. If you require accommodations of any kind at any time (directly related to the pandemic or not), please contact the teaching team.

Syllabus revisions
This syllabus will be a dynamic document that will evolve throughout the quarter. Although the core expectations are fixed, the details will shift. As a result, please keep in mind the following:


 * 1) Assignments and readings are frozen 1 week before they are due. I will not add readings or assignments less than one week before they are due. If I forget to add something or fill in a "To Be Determined" less than one week before it's due, it is dropped. If you plan to read or work more than one week ahead, contact me first.
 * 2) Substantial changes to the syllabus or course materials will be announced. Please closely monitor your email and/or the announcements section on the course website on Canvas. When I make changes, these changes will be recorded in the edit history of this page so that you can track what has changed. I will also do my best to summarize these changes in an announcement on Canvas that will be emailed to everybody in the class.
 * 3) The course design may adapt throughout the quarter. As this is a new format for this course, I may iterate and prototype course design elements rapidly along the way. To this end, I will ask you for voluntary anonymous feedback — especially toward the beginning of the quarter. Please let me know what is working and what can be improved. In the past, I have made many adjustments based on this feedback and I expect to do so again.

Statistics and power
The subject matter of this course—statistics and statistical programming—has historical and present-day affinities with a variety of oppressive ideologies and projects, including white supremacy, discrimination on the basis of gender and sexuality, state violence, genocide, and colonialism. It has also been used to challenge and undermine these projects in various ways. I will work throughout the quarter to acknowledge and represent these legacies accurately, at the same time as I also strive to advance equity, inclusion, and justice through my teaching practice, the selection of curricular materials, and the cultivation of an inclusive classroom environment. Please see my general classroom policies for more on some of these topics.

Schedule (with all the details)
When reading the schedule below, the following key might help resolve ambiguity: §n denotes chapter n; §n.x denotes section x of chapter; §n.x-y denotes sections x through y (inclusive) of chapter n.

September 17
Required
 * Complete pre-course assessment of statistical concepts (access code TBA via email). Submission deadline: September 18, 11:00pm Chicago time
 * Confirm access to software and web-services for course (Discord, Canvas, this wiki, R, RStudio).
 * Complete problem set #0
 *  Read and work through Introduction to R and RStudio
 * Install R and RStudio, getting help, creating/saving .Rmd, weaving code and text, knitting output into html or pdf.
 *  Refactor 2019 PS1 PC1-4. Update R and Rstudio installation. Add a calculator problem. Clarify task to write code and text and knit output.

Resources
 * Verzani §1 (Getting started) and Healy §2 (Get started) provide helpful background for working with R and RStudio.

September 22
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §1.1-1.3 (Introduction to data).
 * Watch Lecture materials for §1.1-3 (Videos 1-4 in the playlist).
 * Complete problem set #1
 * SQ from OpenIntro §1: 1.6, 1.9, 1.10, 1.16, 1.21, 1.40, 1.42, 1.43

September 24
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §2.1-2 (Numerical and categorical data).
 * Review Lecture materials for §2.1 and §2.2 (Videos 6-7 in the playlist).
 * Complete problem set #2
 * OpenIntro questions:

Resources

September 29
Required
 * R lecture materials from 2019 W02
 * Complete problem set #3
 * Empirical paper/data (UCB admissions. Police stops in IL.)
 * See PS2 Programming challenges from 2019

Resources
 * UCB admissions paper
 * Stanford OpenPolicing Project

October 1
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §3.1-3; §3.4-5 (Probability).
 * Watch Probability introduction and Probability trees OpenIntro lectures (just videos 1 and 2 in the playlist).
 * Complete problem set #4
 * OpenIntro questions:

Resources
 * Seeing Theory §1-2

October 6
Required
 * Complete problem set #5

Resources

October 8
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §4.1-3 (Normal and binomial distributions).
 * Watch normal and binomial distributions OpenIntro lectures (videos 1-3 in the playlist).
 * Complete problem set #6
 * OpenIntro questions:

Resources

October 13
Required
 * Complete problem set #7

Resources

October 15
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §5 (Foundations for inference).
 * Watch foundations for inference (videos 1-3 in the playlist) OpenIntro lectures.
 * Complete Why .05? OpenIntro video/exercise.
 * Complete problem set #8
 * OpenIntro questions:

Resources
 * Kelly M., Emily Dickinson and monkeys on the stair Or: What is the significance of the 5% significance level? Significance 10:5. 2013.

October 20
Required
 * Complete problem set #9

Resources

October 22
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §6 (Inference for categorical data).
 * Watch inference for categorical data (videos 1-3 in the playlist) OpenIntro lectures.
 * Complete problem set #10
 * OpenIntro questions:

Resources
 * OpenIntro Central limit theorem for proportions demo.

October 27
Required Resources
 * Complete problem set #11

October 29
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §7.1-3 (Inference for numerical data: differences of means).
 * Watch inference for numerical data (videos 1-4 in the playlist) OpenIntro lectures (and featuring one of the textbook authors!).
 * Complete problem set #11
 * OpenIntro questions:

Resources
 * OpenIntro Central liumit theorem for means demo.

November 3
Election Day (U.S.): No class meeting today

November 5
Required
 * Read Diez, Çetinkaya-Rundel, and Barr: §7.4-5 (Inference for numerical data: power calculations, ANOVA, and multiple comparisons).
 * Watch inference for numerical data (videos 4-8 in the playlist) OpenIntro lectures (and featuring one of the textbook authors!).
 * Complete problem set #12
 * OpenIntro questions:

Resources
 * OpenIntro supplement on ANOVA calculations (useful if you think you'll be doing more ANOVAs).

November 10
Required
 * Complete problem set #13

Resources

November 12
Required Resources
 * Read Diez, Çetinkaya-Rundel, and Barr: §8 (Linear regression).

November 17
Required Resources

November 19
Required Resources

November 24
Required Resources
 * Complete post-course assessment of statistical concepts (access code TBA VIA email). Submission deadline: December 1, 11:00pm Chicago time

2019 course schedule
When reading the schedule below, the following key might help resolve ambiguity: §n denotes chapter n; §n.x denotes section x of chapter; §n.x-y denotes sections x through y of chapter n.

Week 1: Thursday April 4: Introduction, Setup, and Data and Variables

 * Statistics and Statistical Programming (Spring 2019)/Session plan: Week 1

Please complete the readings and assignment prior to class so that we can discuss them and start talking through some of the examples in R together.

Required Readings:


 * Diez, Barr, and Çetinkaya-Rundel: §1 (Introduction to data)
 * Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks. Proceedings of the National Academy of Sciences 111(24):8788–90. [Open Access]

Recommended Readings:

Assignment (Complete before class):
 * Verzani: §1 (Getting Started), §2 (Univariate data) [Available via Canvas]
 * Verzani: §A (Programming)
 * Healy: §2 (and skim the preferatory material as well as §1)


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 1

Lectures:
 * Week 1 R lecture materials (.zip file)
 * Week 1 screencast (part 1, 23 minutes) (the video should load directly in browser window)
 * Week 1 screencast (part 2, 27 minutes)

Resources:
 * Mine Çetinkaya-Rundel's OpenIntro §1 Lecture Notes
 * OpenIntro Video Lectures including some for §1

Week 2: Thursday April 11: Probability and Visualization

 * Statistics and Statistical Programming (Spring 2019)/Session plan: Week 2
 * Questions? Topics you'd like to discuss? Add them to the Canvas discussion for this week's material.

Required Readings:


 * Diez, Barr, and Çetinkaya-Rundel: §2 (Probability)
 * Shaw, Aaron and Yochai Benkler. 2012. A tale of two blogospheres: Discursive practices on the left and right. American Behavioral Scientist. 56(4): 459-487. [available via NU libraries]

Recommended Readings:
 * Verzani: §3.1-2 (Bivariate data), §4 (Multivariate data), §5 (Multivariate graphics)
 * Seeing Theory §1 (Basic Probability) and §2 (Compound Probability). (Note: this site provides a beautiful visual introduction to core concepts in probability and statistics).


 * Healy: §3.

Assignment (Complete Before Class):


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 2

Lectures:
 * Week 2 R lecture materials (.Rmd file)
 * Week 2 screencast (17 minutes)

Resources:


 * Mine Çetinkaya-Rundel's OpenIntro §2 Lecture Notes
 * Video Lectures including 2 short videos for §2

Week 3: Thursday April 18: Distributions

 * Statistics and Statistical Programming (Spring 2019)/Session plan: Week 3

Required Readings:


 * Diez, Barr, and Çetinkaya-Rundel: §3.1-3.2, §3.4: You should read the rest of the chapter (§3.3 and §3.5). I won't assign problem set questions about it but it's still important to be familiar with.

Recommended Readings:
 * Verzani: §6 (Populations)
 * Seeing Theory §3 (Probability Distributions).

Assignment (Complete Before Class):


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 3

Lectures:


 * Week 3 R lecture materials (.Rmd file)
 * Week 3 screencast (19 minutes)

Resources:


 * Mine Çetinkaya-Rundel's OpenIntro §3 Lecture Notes
 * OpenIntro Video Lectures including 2 videos for §3.1 and §3.2

Week 4: Thursday April 25: Statistical significance and hypothesis testing

 * Statistics and Statistical Programming (Spring 2019)/Session plan: Week 4

Required Readings:


 * Diez, Barr, and Çetinkaya-Rundel: §4 (Foundations for inference)

Recommended Readings:
 * Verzani: §7 (Statistical inference), §8 (Confidence intervals)
 * Seeing Theory §4 (Frequentist Inference)

Assignment (Complete Before Class):


 * Mid-quarter course evaluation survey (by Monday please!)
 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 4

Lectures:
 * Week 4 R lecture materials (.Rmd file)
 * (No screencast for this week)

Resources:


 * Mine Çetinkaya-Rundel's OpenIntro §4 Lecture Notes
 * OpenIntro Video Lectures including 7 videos for nearly all of §4

Week 5: Thursday May 2: Continuous Numeric Data & ANOVA

 * Session plan

Required Readings:


 * Diez, Barr, and Çetinkaya-Rundel: §5 (Inference for numerical data)


 * Sweetser, K. D., & Metzgar, E. (2007). Communicating during crisis: Use of blogs as a relationship management tool. Public Relations Review, 33(3), 340–342. [Available through NU Libraries]
 * Reinhart, §1

Recommended Readings:
 * Verzani: §9 (significance tests), §12 (Analysis of variance)
 * Gelman, Andrew and Hal Stern. 2006. “The Difference Between ‘Significant’ and ‘Not Significant’ Is Not Itself Statistically Significant.” The American Statistician 60(4):328–31. [Available through NU Libraries]

Assignment (Complete Before Class):


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 5

Lectures:
 * No new R material for this week.

Resources:


 * Mine Çetinkaya-Rundel's OpenIntro §5 Lecture Notes

Week 6: Thursday May 9: Categorical data
Required Readings:
 * Session plan


 * Diez, Barr, and Çetinkaya-Rundel: §6.1-6.4 (Inference for categorical data).
 * Buechley, Leah and Benjamin Mako Hill. 2010. “LilyPad in the Wild: How Hardware’s Long Tail Is Supporting New Engineering and Design Communities.” Pp. 199–207 in Proceedings of the 8th ACM Conference on Designing Interactive Systems. Aarhus, Denmark: ACM. [PDF available on Hill's personal website]
 * Reinhart, §4 and §5.

'''Recommended Readings:
 * Diez, Barr, and Çetinkaya-Rundel: §6.5-6.6 (Small samples and randomization inference)
 * Verzani: §3.4 (Bivariate categorical data); §10.1-10.2 (Goodness of fit)
 * Gelman, Andrew and Eric Loken. 2014. “The Statistical Crisis in Science Data-Dependent Analysis—a ‘garden of Forking Paths’—explains Why Many Statistically Significant Comparisons Don’t Hold Up.” American Scientist 102(6):460. [Available through NU Libraries] (This is a reworked version of this unpublished manuscript which provides a more detailed examples.)

Assignment (Complete Before Class):


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 6

Lectures:
 * Week 6 R lecture materials (.Rmd file)
 * (No screencast for this week)

Resources:
 * Mine Çetinkaya-Rundel's OpenIntro §6 Lecture Notes
 * OpenIntro Video Lectures including 4 videos for §7

Week 7: Thursday May 16: Linear Regression
Required Readings:
 * Session plan


 * Diez, Barr, and Çetinkaya-Rundel: §7 (Introduction to linear regression)
 * OpenIntro eschews a mathematical approach to correlation. Look over the Wikipedia article on correlation and dependence and pay attention to the formulas. It's tedious to compute, but you should be aware of what goes into it.
 * Lampe, Cliff, and Paul Resnick. 2004. “Slash(Dot) and Burn: Distributed Moderation in a Large Online Conversation Space.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04), 543–550. New York, NY, USA: ACM. doi:10.1145/985692.985761. [Available via NU libraries]

Recommended Readings:
 * Verzani: §11.1-2 (Linear regression).
 * Seeing Theory §5 (Regression Analysis)

Assignment (Complete Before Class):


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 7
 * Final project planning document (see details above!)

Lectures:
 * Week 7 R lecture materials

Resources:
 * Mine Çetinkaya-Rundel's OpenIntro §7 Lecture Notes
 * Mine Çetinkaya-Rundel's OpenIntro §8 Lecture Notes
 * OpenIntro Video Lectures including 4 videos for §7 and 3 videos on the sections §8.1-8.3

Week 8: Thursday May 23: Polynomial Terms, Interactions, and Logistic Regression

 * Session plan

Required Readings:
 * Diez, Barr, and Çetinkaya-Rundel: §8 (Multiple and logistic regression)
 * Lesson 8: Categorical Predictors and Lesson 9: Data Transformations from the PennState Eberly College of Science STAT 501 Regression Methods Course. There are several subparts (many quite short), please read them all carefully.
 * (Revisit) Lampe, Cliff, and Paul Resnick. 2004. “Slash(Dot) and Burn: Distributed Moderation in a Large Online Conversation Space.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04), 543–550. New York, NY, USA: ACM. doi:10.1145/985692.985761. [Available via NU libraries]
 * Reinhart, §8 and §9.

Recommended Readings:
 * Verzani: §11.3 (Linear regression), §13.1 (Logistic regression)
 * Ioannidis, John P. A. 2005. “Why Most Published Research Findings Are False.” PLoS Medicine 2(8):e124. [Open Access]
 * Head, Megan L., Luke Holman, Rob Lanfear, Andrew T. Kahn, and Michael D. Jennions. 2015. “The Extent and Consequences of P-Hacking in Science.” PLOS Biology 13(3):e1002106. [Open Access]

Assignment (Complete Before Class):


 * Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 8

Lectures:
 * Week 8 R lecture materials

Resources:


 * Mine Çetinkaya-Rundel's OpenIntro §8 Lecture Notes
 * OpenIntro Video Lectures including a video on §8.4
 * Mako Hill wrote this document which will likely be useful for many of you: Interpreting Logistic Regression Coefficients with Examples in R

Week 9: Thursday May 30: Loose ends and Final Presentations (part 1)

 * Session plan

Required readings:
 * Reinhart, §10 and §11.

Final presentations: (part 1)
 * First batch today. The rest next week.

Resources:
 * Week 9 R-lecture (we will use this in class)

Week 10: Thursday June 6: Fully reproducible research example, Replications, Final Presentations (part 2), and wrap-up

 * Fully reproducible research example.
 * Research replication study by Polly Straub-Cook (UW Comm. Ph.D. student)
 * (n.b.: cluster & heteroscedasticity robust standard errors!)


 * Final presentations: (part 2)
 * Second batch of presenters today.


 * Closing thoughts
 * What next? Beyond your final projects...
 * Class social gathering

Followed by much rejoicing!

Credit and Notes
This syllabus has, in ways that should be obvious, borrowed and built on the OpenInto Statistics curriculum. I also based most aspects of the course design on Benjamin Mako Hill's COM 521 class.