Editing Community Data Science Course (Sprint 2019)/Day 7 Notes

'''We will be discussing [https://data.seattle.gov/Transportation/Collisions/vac5-r8kk this data set].'''


* One of the most important qualities of the Scientific Revolution was that results were broadly shared, so new results could build on top of existing knowledge.
* Repeatability is the key to science (even data science): your results are only scientific if they are repeatable by a third party.

'''Today's Lecture'''
Let's go end to end on a data question: are there factors that predict injuries and fatalities in automobile accidents?
* Download data
* Explore the data: find missing values, identify categorical, numerical, ordinal data fields
* Transform (filter, project)
* Analyze

'''Download'''

 import requests
    
 url = 'https://data-seattlecitygis.opendata.arcgis.com/datasets/5b5c745e0f1f48e7a53acec63a0022ab_0.csv'
 response = requests.get(url)
    
 filehandle = open('~/Desktop/collisions.csv', 'w')
 filehandle.write(response.text)
 filehandle.close()


Opening a file is new. Note that "open" can open for reading or writing files. '''Be careful''' opening a file to write will erase that file. You can not get it back.

'''Explore'''
Open the file in Excel. What columns seem to be missing sometimes?

Find a categorical, numerical, and ordinal data field.


'''Read and Transform'''

In this section, we will read and transform the data.

'''Code to open a file and print the first column'''
   
 file_handle = open('collisions.csv', 'r')   # open the csv file
 for line in file_handle:                                 # loop through the file one line at a time.
     line_clean = line.strip()                            # remove the newline character at end of line
     line_clean_list = line_clean.split(',')              # split the line into parts using split
     print(line_clean_list[0])                            # print the first column of data for this row.


'''Code to open a file, select a subset of rows and columns, and write to a new file'''
Figure out what this code does!

(You'll want to open this window in a wide browser)
   
 file_handle = open('collisions.csv', 'r')   # open the csv file
 header = file_handle.readline()
 output_handle = open('collisions_transformed.csv', 'w')    # NOTE this will overwrite 
 for line in file_handle:                                 # loop through the file one line at a time.
     line_clean = line.strip()                            # remove the newline character at end of line
     line_clean_list = line_clean.split(',')              # split the line into parts using split
     if int(line_clean[17]) > 0:                          # If the integer value in columns 17 is greater than one then...
         output_handle.write(line)                        # write that line to the output.
 output_handle.close()                                    # Close the output file after the loop.


'''Analyze'''

We will answer together whether accidents involving cyclists or pedestrians are more likely to result in an injury.

We will use the '''odds ratio''' to discuss the likelihood of injury.

Be sure you understand the difference between risk ratio and odds ratio!