Statistics and Statistical Programming (Fall 2020)/pset5: Difference between revisions

From CommunityData
(Created page with "<small>← Back to Week 7</small> == Programming Challenges == For the programming ch...")
 
m (Change to PC numbering. Previously was a jump from PC3 to PC5 ~CA)
 
(2 intermediate revisions by one other user not shown)
Line 1: Line 1:
<small>[[Statistics_and_Statistical_Programming_(Fall_2020)##Week_7_.2810.2F27.2C_10.2F29.29|← Back to Week 7]]</small>
<small>[[Statistics_and_Statistical_Programming_(Fall_2020)##Week_7_.2810.2F27.2C_10.2F29.29|← Back to Week 7]]</small>
Overall, the focus this week is on practical issues in applied analysis of categorical data. Specifically, issues that sometimes arise in the context of conducting <math>\chi^2</math> tests, calculating/interpreting p-values, as well as issues with multiple comparisons and base-rate fallacies. We're going to work entirely from three empirical papers (one experiment and two observational studies). You shouldn't need to read any of them particularly closely to be able to answer the following questions, but it's useful to have a sense of the specifics of each study and to be able to refer to the authors' rationales for their analyses and interpretations of their findings.


== Programming Challenges ==
== Programming Challenges ==


For the programming challenges this week, we'll re-analyze data from the following study:
For the programming challenges, we'll re-analyze data from the following (Halloween-appropriate!) study:
:: Aronow PM, Karlan D, Pinson LE. (2018). The effect of images of Michelle Obama’s face on trick-or-treaters’ dietary choices: A randomized control trial. PLoS ONE 13(1): e0189693. [https://doi.org/10.1371/journal.pone.0189693 https://doi.org/10.1371/journal.pone.0189693]
:: Aronow PM, Karlan D, Pinson LE. (2018). The effect of images of Michelle Obama’s face on trick-or-treaters’ dietary choices: A randomized control trial. PLoS ONE 13(1): e0189693. [https://doi.org/10.1371/journal.pone.0189693 https://doi.org/10.1371/journal.pone.0189693]


: '''PC1.''' [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/2NJV2P Download the dataset]. You may want to familiarize yourself with the experimental treatment and key details of the research design from the (short) article.
===PC1. Access and import the data ===
:'''PC2.''' Import the data into R. You may need to install the <code>readstata13</code> package and identify an appropriate function with which to do so.
* [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/2NJV2P Download the dataset]. You may want to familiarize yourself with the experimental treatment and key details of the research design from the (short) article linked above.
:'''PC3.''' Get to know your dataset. Take a look at the codebook if necessary and make sure you have the two columns of the dataset that correspond to the experimental treatment (being presented with Michelle Obama's face) and the outcome (whether or not kids picked up fruit). Don't worry about any of the other measures for now.
* Import the data into R. Depending on the file format you encounter, you may need to install the <code>readstata13</code> package and identify an appropriate function with which to do so.
:'''PC4.''' Create a two-way contingency table summarizing these two variables. Make sure your table has easily understandable column and row names.
===PC2. Explore and cleanup the data===
:'''PC5.''' Conduct a test to determine whether or not the two groups are dependent. Report and interpret the results of your test and be prepared to discuss your findings. Please note that the paper uses a variety of techniques including linear regression and incorporates other variables, but you should use estimators and tests we read about in Chapter 6 last week. Do you have a sense, from your ''OpenIntro'' readings this week, why the results might be different when you use regression and incorporate the other variables?
Get to know your dataset. Take a look at the codebook if necessary and make sure you have the two columns of the dataset that correspond to the experimental treatment (being presented with Michelle Obama's face or not) and the outcome (whether or not trick-or-treaters picked up fruit). Don't worry about any of the other measures for now.
:'''PC6.''' Try to reproduce the top panel of Figure 1 using the same two columns of the dataset (by ignoring year and the other variables we are, in effect, working with the "pooled" sample). If you cannot reproduce that portion of the figure (or something like it), try to at least reproduce the values presented in it.
===PC3. Summarize key variables===
:'''PC7.''' It's very important to be able to export tables directly into your word processor or typesetting software without cutting and pasting the contents of individual cells. Can you export the ''output'' of your table from PC4? There are a bunch of functions you can use to do this. I would likely use the <code>xtable</code> package to generate HTML and/or LaTeX output, but I think that <code>write.table()</code> and Excel could do the job just as well.
Create a two-way contingency table summarizing these two variables. Make sure your table has easily understandable column and row names.
===PC4. Test for differences between groups ===
Construct and perform a statistical hypothesis test to determine whether or not the two groups are dependent. State your hypotheses clearly. Report and interpret the results of your test and be prepared to discuss your findings. Please note that the paper uses a variety of techniques including linear regression and incorporates other variables, but you should use estimators and tests we read about in ''OpenIntro'' §6 last week.
===PC5. Replicate a figure===
Try to reproduce the top panel of Figure 1 using the same two columns of the dataset (by ignoring year and the other variables we are, in effect, working with the "pooled" sample). If you cannot reproduce that portion of the figure (or something like it), try to at least reproduce the values presented in it.
===PC6. Export a table===
We've used RMarkdown to handle reproducible data analysis and export thus far, but it's also often important to export tables directly into your word processor or typesetting software without cutting and pasting the contents of individual cells by hand. Write R code that exports the ''output'' of your table from PC4. There are a bunch of functions you can use to do this. I would likely use the <code>xtable</code> package to generate HTML and/or LaTeX output, but I think that the Base-R <code>write.table()</code> function for export into Excel could do the job just as well.


== Empirical Paper Questions (with Statistical Questions baked in) ==
== Empirical Paper Questions (with Statistical Questions baked in) ==


These questions are for the Buechley and Hill paper on LilyPad Arduino:
===EQ1. LilyPad Arduino users===
 
These questions are for the Buechley and Hill paper on LilyPad Arduino. We'll focus on the statistical test conducted in ''Study 1'':


: '''EQ1.''' For ''Study 1'', let's focus on the statistical test:
:: (a) What is the unit of analysis? What is the dependent variable? The independent variable? What groups are being compared in the test? Is it a one-way or two-way design?
:: (a) What is the unit of analysis? What is the dependent variable? The independent variable? What are groups being compared in the test? Is it a one-way or two-way design?
:: (b) Why bother with the statistical test? Why is this an appropriate statistical procedure?
:: (b) Why not just summarize the results, like we did in week 2? Why bother with the statistical test? How do you decide when to use each statistical procedure?
:: (c) What is the null hypothesis being tested? What is the alternative hypothesis?  
:: (c) What is the null hypothesis being tested? What is the alternative hypothesis?  
:: (d) Summarize or restate the results in statistical terms. Explain what these results mean in substantive terms? How convincing do you find these results? What should we be taking away?
:: (d) Summarize/restate the results in both statistical and substantive terms. Assess the results: How convincing do you find them? What should we take away from this analysis?
 


And these are about the Shaw and Benkler paper on political blogs:
===EQ2. Two blogospheres===


: '''EQ2.''' Using the data from Table 3 and Figure 2:
These questions are about the Shaw and Benkler paper on political blogs. We'll focus on Table 3 and Figure 2:
:: (a) What statistical procedure produced the p-value in Table 3? What is the null hypothesis being tested?
:: (a) What statistical procedure produced the p-value in Table 3? State the null/alternative hypotheses being tested.
:: (b) How convincing do you find these results? What are some reasons to be skeptical?
:: (b) How convincing do you find these results? What are some reasons to be skeptical?
:: (c) Reproduce Figure 2 using the data in Table 3. One thing that's missing from that figure is error bars. Based on the reading from Reinhart §5, what type of error bars do you think you should use? Figure out how to calculate the error bars and figure out if they overlap? What does that tell us? (Bonus: figure out how to add error bars to the bar plot)
:: (c) Reproduce Figure 2 using the data in Table 3. One thing that's missing from that figure is error bars. Based on the reading from Reinhart §5, what type of error bars do you think you should use? Figure out how to calculate the error bars and figure out if they overlap? What does that tell us? (Bonus: figure out how to add error bars to the bar plot)
:: (d) Should we be concerned about the base rate fallacy described in Reinhart §4? Why or why not?
:: (d) Should we be concerned about the base rate fallacy described in Reinhart §4? Why or why not?

Latest revision as of 20:53, 25 October 2020

← Back to Week 7


Overall, the focus this week is on practical issues in applied analysis of categorical data. Specifically, issues that sometimes arise in the context of conducting tests, calculating/interpreting p-values, as well as issues with multiple comparisons and base-rate fallacies. We're going to work entirely from three empirical papers (one experiment and two observational studies). You shouldn't need to read any of them particularly closely to be able to answer the following questions, but it's useful to have a sense of the specifics of each study and to be able to refer to the authors' rationales for their analyses and interpretations of their findings.

Programming Challenges[edit]

For the programming challenges, we'll re-analyze data from the following (Halloween-appropriate!) study:

Aronow PM, Karlan D, Pinson LE. (2018). The effect of images of Michelle Obama’s face on trick-or-treaters’ dietary choices: A randomized control trial. PLoS ONE 13(1): e0189693. https://doi.org/10.1371/journal.pone.0189693

PC1. Access and import the data[edit]

  • Download the dataset. You may want to familiarize yourself with the experimental treatment and key details of the research design from the (short) article linked above.
  • Import the data into R. Depending on the file format you encounter, you may need to install the readstata13 package and identify an appropriate function with which to do so.

PC2. Explore and cleanup the data[edit]

Get to know your dataset. Take a look at the codebook if necessary and make sure you have the two columns of the dataset that correspond to the experimental treatment (being presented with Michelle Obama's face or not) and the outcome (whether or not trick-or-treaters picked up fruit). Don't worry about any of the other measures for now.

PC3. Summarize key variables[edit]

Create a two-way contingency table summarizing these two variables. Make sure your table has easily understandable column and row names.

PC4. Test for differences between groups[edit]

Construct and perform a statistical hypothesis test to determine whether or not the two groups are dependent. State your hypotheses clearly. Report and interpret the results of your test and be prepared to discuss your findings. Please note that the paper uses a variety of techniques including linear regression and incorporates other variables, but you should use estimators and tests we read about in OpenIntro §6 last week.

PC5. Replicate a figure[edit]

Try to reproduce the top panel of Figure 1 using the same two columns of the dataset (by ignoring year and the other variables we are, in effect, working with the "pooled" sample). If you cannot reproduce that portion of the figure (or something like it), try to at least reproduce the values presented in it.

PC6. Export a table[edit]

We've used RMarkdown to handle reproducible data analysis and export thus far, but it's also often important to export tables directly into your word processor or typesetting software without cutting and pasting the contents of individual cells by hand. Write R code that exports the output of your table from PC4. There are a bunch of functions you can use to do this. I would likely use the xtable package to generate HTML and/or LaTeX output, but I think that the Base-R write.table() function for export into Excel could do the job just as well.

Empirical Paper Questions (with Statistical Questions baked in)[edit]

EQ1. LilyPad Arduino users[edit]

These questions are for the Buechley and Hill paper on LilyPad Arduino. We'll focus on the statistical test conducted in Study 1:

(a) What is the unit of analysis? What is the dependent variable? The independent variable? What groups are being compared in the test? Is it a one-way or two-way design?
(b) Why bother with the statistical test? Why is this an appropriate statistical procedure?
(c) What is the null hypothesis being tested? What is the alternative hypothesis?
(d) Summarize/restate the results in both statistical and substantive terms. Assess the results: How convincing do you find them? What should we take away from this analysis?

EQ2. Two blogospheres[edit]

These questions are about the Shaw and Benkler paper on political blogs. We'll focus on Table 3 and Figure 2:

(a) What statistical procedure produced the p-value in Table 3? State the null/alternative hypotheses being tested.
(b) How convincing do you find these results? What are some reasons to be skeptical?
(c) Reproduce Figure 2 using the data in Table 3. One thing that's missing from that figure is error bars. Based on the reading from Reinhart §5, what type of error bars do you think you should use? Figure out how to calculate the error bars and figure out if they overlap? What does that tell us? (Bonus: figure out how to add error bars to the bar plot)
(d) Should we be concerned about the base rate fallacy described in Reinhart §4? Why or why not?