Editing Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 2

This is general advice going forward but it makes sense to include it here: My advice is to start working through the programming challenges first. The programming challenges will only include material that we covered in the readings for the previous week.

If you're having trouble loading up your dataset ('''PC2''') please find me or Jeremy for help asap because the other questions depend on doing this successfully.

== Programming Challenges ==

:'''PC0.''' Create a new project and RMarkdown script for this week's problem set.
:'''PC1.''' Run the following command (just once!) at the R console: <code> sample(x=c(1:20), size=1)</code>. The output of the command is your group number for this assignment (''Optional bonus'': Explain what this command does.).
:'''PC2.''' Navigate to the [https://communitydata.cc/~ads/teaching/2019/stats/data data repository for the course] and download the RData file in the <code>week_02</code> subdirectory with your group number from PC1 associated with it (e.g., <code>group_<output>.Rdata</code>). 
:'''PC3.''' Load that file into R. It should contain one variable. Find that variable!
:'''PC4.'''  Compute and present summary statistics for your variable. Be sure to include the minimum, maximum, mean, median, variance, standard deviation, and interquartile range.
<!---:'''PC5.''' Write your own functions in R to re-compute the mean and the median. Be ready to walk us through how your function works.--->
:'''PC5.''' Create some visualizations of your dataset: at the very least, create a boxplot and histogram.
:'''PC6.''' Some of you will have negative numbers. Whoops! Recode all negative numbers as missing (i.e. NA) in your dataset. Now compute a new mean and standard deviation. How does it change? ('''Hint:''' the <code>mean()</code> function may now produce an error. You have to include the argument <code>na.rm=TRUE</code> to work around this.) 
:'''PC7.''' Log transform your dataset (i.e., take the natural logarithm for each value). If you have very small values it may be helpful to add 1 to each value before you take the natural logarithm (this avoids nonsense values). Calculate the new mean, median, and standard deviation of the transformed variable. Also create a new histogram and boxplot.
:'''PC8.''' Briefly discuss any important differences you observe between the original variable and the new variable (i.e., the one you removed negative values from and log-transformed).
:'''PC9.''' Save your work and archive the project (i.e., in a .zip file) and [https://canvas.northwestern.edu/courses/90927/assignments/577505 upload it to canvas].

== Statistical Questions ==
=== Exercises from OpenIntro §2 ===

: '''SQ0.''' Any questions or clarifications from the OpenIntro text or lecture notes?
: '''SQ1.''' Exercise 2.12 on kids missing school 
: '''SQ2.''' Exercise 2.20 on "assortative mating"
: '''SQ3.''' Exercise 2.26 on twins (and conditional probability)
: '''SQ4.''' Exercise 2.32 on the birthday problem (This is a super famous problem! Don't look it up!) 
: '''SQ5.''' Exercise 2.38 with the example of baggage fees 
: '''SQ6.''' Exercise 2.44 on income and gender

=== Empirical Paper ===

Let's take a look at this (very dated!) paper which I published in graduate school:

: Shaw, Aaron and Yochai Benkler. 2012. A tale of two blogospheres: Discursive practices on the left and right. ''American Behavioral Scientist''. 56(4): 459-487. [[https://doi.org/10.1177%2F0002764211433793 available via NU libraries]]

Read enough of the paper to get a sense for what it's about and to answer the questions below. Many of them refer to the top part of Table 5 (about the authorship of posts). Feel free to ignore all the other stuff that's not relevant.

: '''EQ1.''' Identify (a) the population of interest and (b) the sample used in the study.
: '''EQ2.''' Given that a blog in the dataset has a left perspective, what is the probability that it is "solo" authored?
: '''EQ3.''' Given that a blog in the dataset is a "large-scale collaboration", what is the probability that it has a right perspective?
: '''EQ4.''' In substantive terms, do these probabilities provide evidence that left wing blogs are more or less collaborative? Identify a reason to support your answer as well as reason to be skeptical.
: '''EQ5.''' Given your answers to EQ1 and EQ4, do you think that your conclusion genereralizes from the sample to the population? Why or why not?