Editing Statistics and Statistical Programming (Winter 2017)/Problem Set: Week 4
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
| Latest revision | Your text | ||
| Line 3: | Line 3: | ||
We're going to be doing a bit more advanced data wrangling this week. And with real data! | We're going to be doing a bit more advanced data wrangling this week. And with real data! | ||
We're going to be using two datasets from [https://data.seattle.gov data.seattle.gov] related to online engagement. Both datasets are drawn from Google Analytics but we're going to try to merge them together to make a statement about what proportion of views we believe are coming from mobile users and if we believe this number is going up or down over time. | We're going to be using two datasets from [https://data.seattle.gov data.seattle.gov] related to online engagement. Both datasets are drawn from Google Analytics but we're going to try to merge them together to make a statement about what proportion of views we believe are coming from mobile users and if we believe this number is going up or down over time. The datasets are messy and aren't set up well to help us answer the question as we would like to. Nearly all of this challenge is going to be dealing with the messiness of real data. | ||
:'''PC1.''' Download these two datasets from data.seattle.gov, save them into your git repository [https://data.seattle.gov/City-Business/COS-Statistics-Top5000-Pages/yuhv-gvtm COS-Statistics-Top5000-Pages] [https://data.seattle.gov/City-Business/COS-Statistics-Mobile-Sessions/2u47-byfn COS-Statistics-Mobile Sessions]. Assume, for the purposes of this assignment that the first dataset on views to the top 5000 pages is ''all'' views made to the http://www.seattle.gov website | |||
:'''PC1.''' Download these two datasets from data.seattle.gov, save them into your git repository | |||
:'''PC2.''' Load both datasets into R as separate data frames. Explore the data to get a sense of the structure of the data. What are the columns, rows, missing data, etc? Write code to take (and then check/look at) several random subsamples of the data. | :'''PC2.''' Load both datasets into R as separate data frames. Explore the data to get a sense of the structure of the data. What are the columns, rows, missing data, etc? Write code to take (and then check/look at) several random subsamples of the data. | ||
:'''PC3.''' Using the top 5000 dataset, create a new data frame | :'''PC3.''' Using the top 5000 dataset, create a new data frame that one column per month described in the data and a second column is the total number of views made to all pages in the dataset over that month. | ||
:'''PC4.''' Using the mobile dataset, create a new | :'''PC4.''' Using the mobile dataset, create a new dataset where one column is each month described in the data and the second is a measure (estimate?) of the total number of views made by mobiles (all platforms) over each month described in that dataset. This will will involve at least two steps since total views are included. You'll need to first use the data there to create a measure of the total views per platform. | ||
:'''PC5.''' Merge your two datasets together into a new dataset with columns for each month, total views (across the top 5000 pages) and total mobile views. Are there are missing data? Can you tell why? | :'''PC5.''' Merge your two datasets together into a new dataset with columns for each month, total views (across the top 5000 pages) and total mobile views. Are there are missing data? Can you tell why? | ||
:'''PC6.''' Create a new column in your merged dataset that describes your best estimate of the proportion (or percentage, if you really must!) of views that comes from mobile. Be able to talk about the assumptions you've made here. Make sure that date, in this final column, is a date or datetime object in R. | :'''PC6.''' Create a new column in your merged dataset that describes your best estimate of the proportion (or percentage, if you really must!) of views that comes from mobile. Be able to talk about the assumptions you've made here. Make sure that date, in this final column, is a date or datetime object in R. | ||
:''' | :'''PC6.''' Graph this over time and be ready to describe: (a) your best estimate of the proportion of views from mobiles to the Seattle City website over time and (b) an indication of whether it's going up or down. | ||
== Statistical Questions == | == Statistical Questions == | ||
| Line 25: | Line 23: | ||
=== Empirical Paper === | === Empirical Paper === | ||
