Editing Community Data Science Course (Spring 2023)/Week 6 coding challenges
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
This week there are three sets of questions. To answer to the first two | This week there are three sets of questions. To answer to the first two, you'll want to work closely from the notebooks I talked through in class during the [[../Week 6 lecture]]. The good news is that answering these will mostly involve modifying or adding small amounts of code (maybe even code we've done in previous assignments!) to those notebooks. | ||
Feel free to use spreadsheets for any part of this that you can but be sure to share links to the spreadsheets in the same way you've been doing. | Feel free to use spreadsheets for any part of this that you can but be sure to share links to the spreadsheets in the same way you've been doing. | ||
Line 9: | Line 9: | ||
mako-03-yelp_1.ipynb | mako-03-yelp_1.ipynb | ||
mako-05-final_project.ipynb | mako-05-final_project.ipynb | ||
== #1 MediaWiki API == | == #1 MediaWiki API == | ||
Identify a movie, television, video game, or other media property that has both (a) | Identify a movie, television, video game, or other media property that has both (a) five or more related articles on Wikipedia and (b) 5 or more other articles on the same topic on a [https://fandom.com Fandom.com] website. Any large entertainment franchise will definitely work but feel free to get creative! | ||
# First modify the code from first sets of notebooks I used in the [[../Week 6 lecture]] to download data (and metadata) about revisions to the | # First modify the code from first sets of notebooks I used in the [[../Week 6 lecture]] to download data (and metadata) about revisions to the five articles from Wikipedia. Be ready to share: | ||
## (i) what proportion of those edits were made by users without accounts | ## (i) what proportion of those edits were made by users without accounts | ||
## (ii) what proportion of those edits were marked as "minor", and | ## (ii) what proportion of those edits were marked as "minor", and | ||
## (iii) make and share a visualization of the total number of edits across those | ## (iii) make and share a visualization of the total number of edits across those five articles over time (I didn't do this in class but I made the TSV file would allow this). | ||
# Now grab data for the 5 articles | # Now grab data for the comparable set of 5 articles (ideally articles on the same topic) from the Fandom.com wiki you identified and grab revision/edit data from there. ('''''Hint:''' The fandom API endpoint for The Wire is https://thewire.fandom.com/api.php but the API, as I said in class, is the same''). Produce answers to the same three questions (i, ii, and iii) above but using this dataset. | ||
# Finally, | # Finally, create a visualization that shows five of those each of those articles (either the ones on Wikipedia or Wikia, but not need to do both) have grown in length (as measured in characters or "bytes") over time. ('''''Hint:''' you'll need to return "size" as one of the revision properties (<code>rvprop</code>) if you are not doing it already.'') | ||
== #2 Yelp API == | == #2 Yelp API == | ||
# Get | # Get setup on [https://fusion.yelp.com/ the Yelp Fusion API]. I've put some details on how to do that on the page on [[Yelp Authentication setup]] page which will likely be very useful! | ||
# Install the <code>yelpapi</code> module which is online | # Install the <code>yelpapi</code> module which is online both [https://pypi.org/project/yelpapi/ has a page] on [https://pypi.org/ the Python Package Index (PyPI) website] and has [https://github.com/lanl/yelpapi a Github page with some documentation]. As I said in class, you can either do this by (a) opening a terminal on your system and running <code>pip install yelpapi</code> or you can try running <code>%run pip install yelpapi</code> in your Jupyter notebook. Reach on out teams or in open lab sessions if you run into trouble. | ||
# Create a new <code>.py</code> file (e.g., I called mine <code>yelp_authentication.py</code>) in the same directory as | # Create a new <code>.py</code> file (e.g., I called mine <code>yelp_authentication.py</code>) in the same directory as Yelp notebooks that are using and add your API key to it. Then use the <code>import</code> command to use your API key in a notebook without having the key itself visible in the notebook! | ||
# Once you've done this, use | # Once you've done this, use this to grab a list of 50 businesses of any time (your choice!) in any city (again, your choice!) using Yelp and the <code>yelpapi</code> module. This should be easy if you modify notebook from the [[../Week 6 lecture]]. | ||
# Once you have done this, add some code so that you save the "raw" JSON output to a <code>.json</code> or <code>.jsonl</code> file (whichever is appropriate). | # Once you have done this, add some code so that you save the "raw" JSON output to a <code>.json</code> or <code>.jsonl</code> file (whichever is appropriate). | ||
# Now create a second notebook that opens up that file, reads the data, and outputs a TSV file with the the name of the business, the average rating, and at least three other pieces or metadata that are available in the Yelp API. | # Now create a second notebook that opens up that file, reads the data, and outputs a TSV file with the the name of the business, the average rating, and at least three other pieces or metadata that are available in the Yelp API. | ||
Line 36: | Line 34: | ||
Answer these questions using markdown cells in a notebook. Check out the notes I left about this in the [[../Week 5 coding challenges]] (the third paragraph) or think back to Kaylea's comments during the Week 6 assignment recap video. | Answer these questions using markdown cells in a notebook. Check out the notes I left about this in the [[../Week 5 coding challenges]] (the third paragraph) or think back to Kaylea's comments during the Week 6 assignment recap video. | ||
Let us know: | Let us know know: | ||
# What is your proposed ''unit of analysis''? In other words, if/when you end up building something like a spreadsheet, what are rows going to represent? | # What is your proposed ''unit of analysis''? In other words, if/when you end up building something like a spreadsheet, what are rows going to represent? | ||
# What specific measures associated with each unit do you want to collect? In other words, what are | # What specific measures associated with each unit do you want to collect? In other words, what are your columns in the spreadsheet going to be? | ||
# Tell us what you've learned about the API: | # Tell us what you've learned about the API: | ||
## Are you going to be able to get the data you want with one API call or many? If more than one, how many? | ## Are you going to be able to get the data you want with one API call or many? If more than one, how many? |