Statistics and Statistical Programming (Winter 2021)/Problem set 7

Programming Challenges
Do police in the United States engage in discriminatory behavior on the basis of race and ethnicity? For this problem set, you will investigate the relationship between traffic stops, vehicle searches and driver attributes (especially race as recorded by police officers conducting traffic stops). Doing so will involve some more advanced data wrangling, visualization, and analysis. We'll use data from The Stanford Open Policing Project (SOPP) that looks at records of traffic stops in Washington state between January 1, 2009 and September 30, 2018. The full SOPP dataset for Washington is about 11 million rows, so I've created a 1% random sample for us to work with here.

Overall, the dataset is well-documented and pretty "clean" (as far as these things go) but there are still a number of features that may be confusing, weird, and/or ill-organized to help answer the questions I've asked you below. Thank goodness you know how to use R to address these issues...

PC1. Create a bivariate visualization
Before we get started with policing data, lets do an "easier" attempt at visualization using.

Return the dataset we used in ../Problem set 5 and load it up the CSV file. You can just find and copy

Visualize two variables in the Problem Set #2 dataset using  and the   function to produce a scatterplot of   on the x-axis and   on the y-axis.

Optional bonus: Incorporate any of the other variables on other dimensions (e.g., color, shape, and/or size are all good options). If you run into any issues plotting these dimensions, revisit the examples in the tutorial and the ggplot2 documentation and consider that ggplot2 can be very picky about the classes of objects.

PC2. Learn about the Open Policing data
Review the project overview on the SOPP homepage, the overview of the data, the description of the standardized data, the codebook/notes for the Washington data from the data_readme.md, as well as any other ancillary materials that you can find that seem likely to help you get oriented with the data.

For the questions below we'll focus on the following measures recorded for each traffic stop in Washington 2009-2018:,  ,  ,  , and.

Record any questions or issues you might notice related to these measures as you review the information about the project and dataset.

PC2. Import, explore, clean
As I mentioned above, the full WA-SOPP dataset is over 11 million rows, so I have created a random 1% subset for us to work with in this assignment which is our Dropbox here repository here. It's about 28MB.

To get started, you'll want to import the data and explore its structure as well as key variables that we'll be focusing on in this analysis (, ,  ,  , and   ). Inspect a random sample of rows to get a sense of the data. What (if anything) is missing? You may also want to clean/recode some of the key variables. Make sure to explain and justify any data cleanup and/or recoding steps you decide to take.

PC3. Summarize outcome and predictor variables
Calculate and report appropriate summary statistics for the outcome and each of the predictor variables we care about (,  ,  , and  ). Include visual and/or tabular summaries where appropriate. Attempt, when possible, to write efficient/elegant code that avoids unnecessary repetition while also retaining clarity.

PC4. Summarize conditional relationships between outcome and predictor variables
The outcome variable we care about here is a dichotomous indicator for whether each traffic stop resulted in a police search (of either the driver or the vehicle) being conducted. Summarize the relationship between each of the predictor variables (, ,  , and  ) and the outcome variable. For continuous predictors, be sure to include visual summaries. For categorical predictors, focus on providing cross-tabulations that report conditional summary statistics within groups (for example, compare the numbers of searches conducted across the two categories of ).

PC5. Analyze relationships between driver race/ethnicity and vehicle searches over time
Summarize the relationship between the recorded  for each traffic stop and the   outcome over the time period covered by the dataset. You may do this in a variety of ways, but a good goal should be to should be to produce the following:


 * 1) A plot of the number of stops across the entire sample within each month
 * 2) A plot of the number of searches across the entire sample within each month.
 * 3) A plot of the number of stops within each   category each month.
 * 4) A plot of the number of searches within each   category each month.
 * 5) A plot of the proportion all searches accounted for within each   category each month.

You'll need to build a dataset. My suggestion is draw out the dataset you want to build. What are the rows? What are the columns?

Here's one suggestion for how you might approach this:

1. Create a new data frame that aggregates stop and search data across sub-groups of  per month. This object could include the following columns:
 * date as a month/year (should be a date or date-time object which will require a day. I set all of them just to "YYYY-MM-01" by rounding them down)
 * race/ethnicity (from the  variable)
 * number of stops (within the  group identified for the row)
 * number of searches (within the  group identified for the row)
 * total number of searches that month/year
 * proportion of total searches (within the  group identified for the row).

Note that this will result in a data frame with multiple rows per month/year (as many as one row for each  category).

If your dataset is wide, you will need to turn it into a long dataset for graphing.

2. Use  and the  layer to generate each of the plots. Note that you'll want to assign  as an aesthetic element  for some of the plots so that ggplot2 represents each category as a separate line (maybe distinguished by color?). Make sure to incorporate useful titles, axis labels, and legends for each plot you produce. Recall that the R tutorials include examples of using  with.

PC6. Calculate baseline population proportions for relevant race/ethnicity categories
To help interpret the results of the foregoing analysis of the traffic stop data, we should calculate some baseline population proportions of the same race/ethnicity categories in the state of Washington around the same time that the SOPP data comes from. Luckily, we have access to exactly the data we need to do this via our old friend, the  library!

Use the  dataset from the   library to calculate the proportions of the Washington population in 2010 in each of the categories identified in the   variable of the traffic stop dataset. You will want to use the codebook for the dataset to identify the variables you'll need. You will also need to wrangle the data a bit to produce the proportions we're looking for. Be sure to note and justify any assumptions and/or recording decisions that you make along the way.

SQ1. Interpret the Washington traffic stop analysis (PCs 3-5)
Return to the questions that motivated this analysis. Based on your results from PCs 3-5, what patterns do you observe in vehicle searches among Washington drivers between 2012-2017? Specifically, what patterns do you note from your comparisons of stops and searches across the categories identified in the  variable? What patterns do you observe when you compare across groups in aggregate (i.e., summed across the entire dataset)? What patterns do you observe in terms of either the numbers or proportions of stops and searches when you compare across groups over time?

SQ2. Contextualize traffic stop data in relation to population data
Consider the results of PCs 3-5 in relation to the results of PC 6. Do the results of PC 6 impact your interpretation of PCs 3-5 in any way? How do you relate the results of the traffic stop and vehicle search analysis to the population baseline proportions?

SQ3. Reflect on the limitations and possible extensions of your analysis
Identify and briefly explain key limitations of the data, your analysis, and the results in relation to the questions and concepts that motivated the exercise. Where possible, what additional information/data or analysis might you suggest in order to overcome these limitations and answer the questions in a more comprehensive or convincing fashion?