Editing Statistics and Statistical Programming (Winter 2021)/Problem set 11

Overall, the focus this week is on practical issues in applied analysis of categorical data. Specifically, issues that sometimes arise in the context of conducting <math>\chi^2</math> tests, calculating/interpreting p-values, as well as issues with multiple comparisons and base-rate fallacies. We're going to work entirely from three empirical papers (one experiment and two observational studies). You shouldn't need to read any of them particularly closely to be able to answer the following questions, but it's useful to have a sense of the specifics of each study and to be able to refer to the authors' rationales for their analyses and interpretations of their findings.

== Programming Challenges ==

For the programming challenges, we'll re-analyze data from the following (Halloween-appropriate!) study:
:: Aronow PM, Karlan D, Pinson LE. (2018). The effect of images of Michelle Obama’s face on trick-or-treaters’ dietary choices: A randomized control trial. ''PLoS ONE'' 13(1): e0189693. [https://doi.org/10.1371/journal.pone.0189693 https://doi.org/10.1371/journal.pone.0189693]

===PC1. Access and import the data ===
* [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/2NJV2P Download the dataset]. You may want to familiarize yourself with the experimental treatment and key details of the research design from the (short) article linked above.
* Import the data into R. Depending on the file format you encounter, you may need to install the <code>readstata13</code> package and identify an appropriate function with which to do so.
===PC2. Explore and cleanup the data=== 
Get to know your dataset. Take a look at the codebook if necessary and make sure you have the two columns of the dataset that correspond to the experimental treatment (being presented with Michelle Obama's face or not) and the outcome (whether or not trick-or-treaters picked up fruit). Don't worry about any of the other measures for now.
===PC3. Summarize key variables===
Create a two-way contingency table summarizing these two variables. Make sure your table has easily understandable column and row names.
===PC4. Test for differences between groups ===
Construct and perform a statistical hypothesis test to determine whether or not the two groups are dependent. State your hypotheses clearly. Report and interpret the results of your test and be prepared to discuss your findings. Please note that the paper uses a variety of techniques including linear regression and incorporates other variables, but you should use estimators and tests we read about in ''OpenIntro'' §6 last week.
===PC5. Replicate a figure===
Try to reproduce the top panel of Figure 1 using the same two columns of the dataset (by ignoring year and the other variables we are, in effect, working with the "pooled" sample). If you cannot reproduce that portion of the figure (or something like it), try to at least reproduce the values presented in it.
===PC6. Export a table===
We've used RMarkdown to handle reproducible data analysis and export thus far, but it's also often important to export tables directly into your word processor or typesetting software without cutting and pasting the contents of individual cells by hand. Write R code that exports the ''output'' of your table from PC4. There are a bunch of functions you can use to do this. I would likely use the <code>xtable</code> package to generate HTML and/or LaTeX output, but I think that the Base-R <code>write.table()</code> function for export into Excel could do the job just as well.

== Empirical Paper Questions (with Statistical Questions baked in) ==

===EQ1. LilyPad Arduino users===

These questions are for the Buechley and Hill paper on LilyPad Arduino. We'll focus on the statistical test conducted in ''Study 1'':

:: (a) What is the unit of analysis? What is the dependent variable? The independent variable? What groups are being compared in the test? Is it a one-way or two-way design?
:: (b) Why bother with the statistical test? Why is this an appropriate statistical procedure?
:: (c) What is the null hypothesis being tested? What is the alternative hypothesis? 
:: (d) Summarize/restate the results in both statistical and substantive terms. Assess the results: How convincing do you find them? What should we take away from this analysis?

===EQ2. Two blogospheres===

These questions are about the Shaw and Benkler paper on political blogs. We'll focus on Table 3 and Figure 2:
:: (a) What statistical procedure produced the p-value in Table 3? State the null/alternative hypotheses being tested.
:: (b) How convincing do you find these results? What are some reasons to be skeptical?
:: (c) Reproduce Figure 2 using the data in Table 3. One thing that's missing from that figure is error bars. Based on the reading from Reinhart §5, what type of error bars do you think you should use? Figure out how to calculate the error bars and figure out if they overlap? What does that tell us? (Bonus: figure out how to add error bars to the bar plot)
:: (d) Should we be concerned about the base rate fallacy described in Reinhart §4? Why or why not?