Editing Statistics and Statistical Programming (Spring 2019)/Problem Set: Week 4

From CommunityData

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
== Programming Challenges ==
== Programming Challenges ==


This week we are going to do more advanced data wrangling. We'll use two datasets from [https://data.seattle.gov data.seattle.gov] related to online engagement with the city of Seattle's websites around 2015-2016. Both datasets are drawn from Google Analytics. We're going to merge them together in order to analyze the proportion of pageviews from mobile users and determine whether this proportion is going up or down over time.  
This week we are going to do more advanced data wrangling. We'll use two datasets from [https://data.seattle.gov data.seattle.gov] related to online engagement. Both datasets are drawn from Google Analytics. We're going to merge them together in order to analyze the proportion of pageviews from mobile users and determine whether this proportion is going up or down over time.  


The datasets are messy and aren't set up well to help us answer the question. Luckily, you know how to use R to solve these problems!
The datasets are messy and aren't set up well to help us answer the question. Luckily, you know how to use R to solve these problems!


:'''PC1.''' The two "raw" datasets come from data.seattle.gov and are available in the [https://communitydata.cc/~ads/teaching/2019/stats/data/week_04/ course data repository] as well as from these links: [https://data.seattle.gov/City-Business/COS-Statistics-Gov-Domains-Only/5nzb-2wmf COS-Statistics-Gov-Domains-Only] [https://data.seattle.gov/City-Business/COS-Statistics-Mobile-Sessions/2u47-byfn COS-Statistics-Mobile Sessions]. You may want to visit the links to read the codebook for each dataset.
:'''PC1.''' Download these two datasets from data.seattle.gov, save them into your git repository: [https://data.seattle.gov/City-Business/COS-Statistics-Top5000-Pages/yuhv-gvtm COS-Statistics-Top5000-Pages] [https://data.seattle.gov/City-Business/COS-Statistics-Mobile-Sessions/2u47-byfn COS-Statistics-Mobile Sessions]. Assume, for the purposes of this assignment that the first dataset on views to the top 5000 pages is ''all'' views made to the http://www.seattle.gov website
:'''PC2.''' Load both datasets into R as separate data frames. Assume, for the purposes of this assignment, that the two datasets include pageview data for the same population of websites. Explore the data to get a sense of the structure. What are the columns, rows, missing data, etc? Write code to take a random sample of rows and then look at them! Maybe inspect a few samples just to get more familiar.
:'''PC2.''' Load both datasets into R as separate data frames. Explore the data to get a sense of the structure of the data. What are the columns, rows, missing data, etc? Write code to take (and then check/look at) several random subsamples of the data.
:'''PC3.''' Using the gov domains data, create a new data frame where one column is each month (as described in the data) and a second column is the total number of views made to all pages in the dataset over that month.
:'''PC3.''' Using the top 5000 dataset, create a new data frame where one column is each month (as described in the data) and a second column is the total number of views made to all pages in the dataset over that month.
:'''PC4.''' Using the mobile data, create a new data frame where one column is each month described in the data and the second is a measure (estimate?) of the total number of views made by mobile devices (all platforms) over each month. This will involve at least two steps since total views are not included. You'll need to first use the data there to create a measure of the total views for each line in the dataset.
:'''PC4.''' Using the mobile dataset, create a new data frame where one column is each month described in the data and the second is a measure (estimate?) of the total number of views made by mobiles (all platforms) over each month. This will involve at least two steps since total views are included. You'll need to first use the data there to create a measure of the total views for each line in the dataset.
:'''PC5.''' Merge your two datasets together into a new dataset with columns for each month, total views (across the gov domain pages) and total mobile views. Make sure that month, in your merged dataset, is a date or datetime object in R. Are there are missing data? Can you tell why?
:'''PC5.''' Merge your two datasets together into a new dataset with columns for each month, total views (across the top 5000 pages) and total mobile views. Are there are missing data? Can you tell why?
:'''PC6.''' Create a new column in your merged dataset that describes your best estimate of the proportion of total views that come from mobile. Be able to talk about any assumptions/decisions you've made in constructing this measure.
:'''PC6.''' Create a new column in your merged dataset that describes your best estimate of the proportion (or percentage, if you really must!) of views that comes from mobile. Be able to talk about the assumptions you've made here. Make sure that date, in this final column, is a date or datetime object in R.
:'''PC7.''' Graph the proportion over time and be ready to describe: (a) your best estimate of the proportion of views from mobile devices to the Seattle City website over time and (b) an indication of whether it's going up or down.
:'''PC7.''' Graph this over time and be ready to describe: (a) your best estimate of the proportion of views from mobiles to the Seattle City website over time and (b) an indication of whether it's going up or down.


== Statistical Questions ==
== Statistical Questions ==
Line 28: Line 28:


Come to class prepared to discuss your answers to the following questions
Come to class prepared to discuss your answers to the following questions
: '''EQ1.''' Write down, in your own words, the key pairs of null/alternative hypotheses tested in the paper (hint: the four pairs that correspond to the main effects represented in the figure).
: '''EQ1.''' Describe, in your own words, the main effects estimated in this paper.
: '''EQ2.''' Describe, in your own words, the main effects estimated in the paper for these four key pairs of hypotheses.
: '''EQ2.''' The authors report ''[https://en.wikipedia.org/wiki/Effect_size#Cohen's_d Cohen's d]]'' along with their regression estimates. Look up and review the formula for ''Cohen's d.'' Discuss the ''substantive'' or ''practical'' significance of the estimates given the magnitude of ''d'' reported.
: '''EQ3.''' The authors report ''[https://en.wikipedia.org/wiki/Effect_size#Cohen's_d Cohen's d]'' along with their regression estimates of the main effects. Look up the formula for ''Cohen's d.'' Discuss the ''substantive'' or ''practical'' significance of the estimates given the magnitudes of the ''d'' values reported.
Please note that all contributions to CommunityData are considered to be released under the Attribution-Share Alike 3.0 Unported (see CommunityData:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To protect the wiki against automated edit spam, we kindly ask you to solve the following CAPTCHA:

Cancel Editing help (opens in new window)
Retrieved from "https://wiki.communitydata.science/Statistics_and_Statistical_Programming_(Spring_2019)/Problem_Set:_Week_4"
Attribution-Share Alike 3.0 Unported
Powered by MediaWiki