Editing Statistics and Statistical Programming (Fall 2020)/pset2
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
<small>[[Statistics_and_Statistical_Programming_(Fall_2020)#Week_4_.2810.2F6.2C_10.2F8.29|← Back to Week 4]]</small> | <small>[[Statistics_and_Statistical_Programming_(Fall_2020)#Week_4_.2810.2F6.2C_10.2F8.29|← Back to Week 4]]</small> | ||
For this problem set, the programming challenges focus on some of the more advanced fundamentals of R, including some of the new types of data import, transformation, tidying, and visualization introduced in the most recent R tutorial. These are followed by some questions about an empirical paper that focus on applying some of the concepts from the first few chapters of ''OpenIntro'' to a research context that | For this problem set, the programming challenges focus on some of the more advanced fundamentals of R, including some of the new types of data import, transformation, tidying, and visualization introduced in the most recent R tutorial materials. These are followed by some questions about an empirical paper that focus on applying some of the concepts from the first few chapters of ''OpenIntro'' to a research context that will likely be familiar. | ||
== Programming Challenges == | == Programming Challenges == | ||
The programming challenges below ask you to perform a series of fairly typical data import, exploration, tidying, and descriptive analysis steps. Once again, you'll work with some "fake" data that Aaron created to ensure consistency and illustrate some useful points. The most recent R tutorials and problem set worked solutions contain example code that should help you do almost everything asked of you here. From this point forward, | The programming challenges below ask you to perform a series of fairly typical data import, exploration, tidying, and descriptive analysis steps. Once again, you'll work with some "fake" data that Aaron created to ensure consistency and illustrate some useful points. The most recent R tutorials and problem set worked solutions contain example code that should help you do almost everything asked of you here. From this point forward, I will start to assume that you have become familiar with some of the more basic fundamental skills (e.g., creating your R Markdown script or notebook) and that you have some ideas of where to turn for help and more information when you need it. That said, you should always seek whatever help you need at any time, whether online, from your peers, or the teaching team. | ||
''Note: if you have trouble accessing or importing your dataset, please reach out for help ASAP as you will only be able to do the other challenges once you've done that one.'' | ''Note: if you have trouble accessing or importing your dataset, please reach out for help ASAP as you will only be able to do the other challenges once you've done that one.'' | ||
Line 32: | Line 30: | ||
Load your vector from [[Statistics_and_Statistical_Programming_(Fall_2020)/pset1|Problem Set #1]] (Week 3) again (you might want to give it a new name) and perform the same cleanup steps you did in PC2.5 and PC2.6 last week (recode negative values as missing and log-transform the data). Now, compare the vector <code>x</code> from Problem Set #1 with the first column (<code>x</code>) of the data you imported for this assignment (Problem Set #2, i.e., the current dataset you just imported from a .csv file). They should be similar, but are they ''exactly'' the same? Use R code to show your answer. | Load your vector from [[Statistics_and_Statistical_Programming_(Fall_2020)/pset1|Problem Set #1]] (Week 3) again (you might want to give it a new name) and perform the same cleanup steps you did in PC2.5 and PC2.6 last week (recode negative values as missing and log-transform the data). Now, compare the vector <code>x</code> from Problem Set #1 with the first column (<code>x</code>) of the data you imported for this assignment (Problem Set #2, i.e., the current dataset you just imported from a .csv file). They should be similar, but are they ''exactly'' the same? Use R code to show your answer. | ||
=== | ===PC6. Cleanup/tidy your data=== | ||
A very common step when you import and prepare for data analysis is going to be cleaning and recoding data. Some of that is needed here. It turns out that the variables <code>i</code> and <code>j</code> are really dichotomous "true/false" variables that have been coded as 0 and 1 in this dataset. Recode these columns as <code>logical</code> (i.e., "TRUE" or "FALSE" values). The variable <code>k</code> is really a categorical variable. Recode this as a factor and change the numbers so that they are replaced with the following values or levels: 0="none", 1="some", 2="lots", 3="all". The goal is to end up with a factor where those text strings are the levels of the factor. | |||
=== | ===PC6. Create a bivariate table=== | ||
Now that you have some categorical variables to work with, let's go ahead and create a bivariate table so that you can examine the distributions of some of these values. Use the <code>table()</code> command to create a cross-tabulation of the recoded versions of the <code>k</code> variable and the <code>j</code> variable. | Now that you have some categorical variables to work with, let's go ahead and create a bivariate table so that you can examine the distributions of some of these values. Use the <code>table()</code> command to create a cross-tabulation of the recoded versions of the <code>k</code> variable and the <code>j</code> variable. | ||
=== | ===PC7. Create a bivariate visualization=== | ||
Visualize two variables in the Problem Set #2 dataset using <code>ggplot2</code> and the <code>geom_point()</code> function to produce a scatterplot | Visualize two variables in the Problem Set #2 dataset using <code>ggplot2</code> and the <code>geom_point()</code> function to produce a scatterplot. First, plot <code>x</code> on the x-axis and <code>y</code> on the y-axis. Second, visualize the other variables on other dimensions (e.g., color, shape, and size seem reasonable). If you run into any issues plotting these dimensions, revisit the examples in the tutorial and the ggplot2 documentation and consider that ggplot2 can be very picky about the classes of objects... | ||
== Statistical Questions == | == Statistical Questions == | ||
===SQ1 | ===SQ1=== | ||
Return to the dataset you imported and worked with in the programming challenges above. Imagine that it comes from a year-long study of bicyclists using a combination of survey and ride-tracking data from the Divvy bikeshare members in the Chicagoland area conducted a few years ago (let's say 2018, just to pick a year). Each row in the data corresponds to a single Divvy cyclist/member and the variables correspond to the following measures: | Return to the dataset you imported and worked with in the programming challenges above. Imagine that it comes from a year-long study of bicyclists using a combination of survey and ride-tracking data from the Divvy bikeshare members in the Chicagoland area conducted a few years ago (let's say 2018, just to pick a year). Each row in the data corresponds to a single Divvy cyclist/member and the variables correspond to the following measures: | ||
* <code>x</code>: Average daily distance cycled (in miles) measured via | * <code>x</code>: Average daily distance cycled (in miles) measured via dock check-in/check-out data. | ||
* <code>j</code>: An indicator (True/False) of whether any rides were recorded between January and March. | * <code>j</code>: An indicator (True/False) of whether any rides were recorded between January and March. | ||
* <code>l</code>: An indicator (True/False) of whether the cyclist also uses vehicle rideshare provided by Lyft (the company that owns Divvy). | * <code>l</code>: An indicator (True/False) of whether the cyclist also uses vehicle rideshare provided by Lyft (the company that owns Divvy). | ||
* <code>k</code>: A measure of how frequently the cyclist rode in bad weather, with bad weather defined using a standard measure provided by the U.S. NOAA (National Oceanic and Atmospheric Administration) and the categories (none, some, a lot, all) defined in terms of | * <code>k</code>: A measure of how frequently the cyclist rode in bad weather, with bad weather defined using a standard measure provided by the U.S. NOAA (National Oceanic and Atmospheric Administration) and the categories (none, some, a lot, all) defined in terms of quartiles within the dataset. | ||
* <code>y</code>: A continuous measure of income calculated in tens of thousands of dollars and scaled so that "0" = average income for a Divvy member (i.e., a value of "5" = $50,000 more per year than an average Divvy member). | * <code>y</code>: A continuous measure of income calculated in tens of thousands of dollars and scaled so that "0" = average income for a Divvy member (i.e., a value of "5" = $50,000 more per year than an average Divvy member). | ||
# Return to the bivariate contingency table you created in PC## above. Given the information you now have about the study, how would you interpret it? | |||
# Return to the bivariate contingency table you created in | |||
# Return to the scatterplot you created in | # Return to the scatterplot you created in PC## above. Given the information you now have about the study, how would you interpret it? | ||
===SQ2=== | |||
=== | ===Optional bonus SQ3=== | ||
''' | ''In the previous session, we talked about birthdays in the context of one of the textbook exercises for Chapter 3. Here's an opportunity to apply your knowledge and extend that exercise. Note that you can absolutely use R to help calculate the solutions to both parts of this problem. That said, it's a super famous problem and answers/examples are all over the internet, so if you want to challenge yourself, don't look at them while you're working on it! The only hint I'll give you is that you may find binomial coefficients useful and the <code>choose()</code>) function can calculate them for you in R.'' | ||
# The last time I taught this course, there were 25 people in it (including the teaching team). Imagine that I offered you a choice between two bets: Bet #1 is determined by the flip of a fair coin (you can choose heads or tails and you win the bet if your choice turns out to be correct). Bet #2 is determined by whether any two members of that previous version of the class shared a birthday (if a birthday was shared I win the bet, if no shared birthdays you win the bet). Assuming you want to win the bet, which bet should you choose? | |||
# Now calculate the probability that any two members of our 7 person class share a birthday and compare this probability with the results of SQ2.1 above. | # Now calculate the probability that any two members of our 7 person class share a birthday and compare this probability with the results of SQ2.1 above. | ||