Community Data Science Course (Spring 2023)/Week 6 coding challenges

This week there are three sets of questions. To answer to the first two, you'll want to work closely from the notebooks I talked through in class during the Community Data Science Course (Spring 2023)/Week 6 lecture. The good news is that nswering these will mostly involve modifying or adding small amounts of code (maybe even code we've done in previous assignments!) to those notebooks.

Feel free to use spreadsheets for any part of this that you can but be sure to share links to the spreadsheets in the same way you've been doing.

#1 MediaWiki API

Identify a movie, television, video game, or other media property that has both (a) five or more related articles on Wikipedia and (b) 5 or more other articles on the same topic on a Fandom.com website. Any large entertainment franchise will definitely work but feel free to get creative!

First modify the code from first sets of notebooks I used in the Community Data Science Course (Spring 2023)/Week 6 lecture to download data (and metadata) about revisions to the five articles from Wikipedia.
1. Tell me (i) what proportion of those edits were made by users without accounts, (ii) what proportion of those edits were marked as "minor", and (iii) make and share a visualization of the total number of edits across those five articles over time (I didn't do this in class but I made the TSV file would allow this).
Now grab data for the comparable set of 5 articles (ideally articles on the same topic) from the Fandom.com wiki you identified and grab revision/edit data from there. (Hint: The fandom API endpoint for The Wire is https://thewire.fandom.com/api.php but the API, as I said in class, is the same).
1. Produce answers to the same three questions (i, ii, and iii) above but using this dataset? Do you see more or less edits over time in the Fandom or Wikipedia articles? Are there more or fewer by users without accounts? Do the number of revisions over time follow similar patterns?
Finally, create a visualization that shows five of those each of those articles (either the ones on Wikipedia or Wikia, but not need to do both) have grown in length (as measured in characters or "bytes") over time. (Hint: you'll need to return "size" as one of the revision properties (rvprop) if you are not doing it already.)

#2 Yelp API

Get setup on the Yelp Fusion API. I've put some details on how to do that on the page on Yelp Authentication setup page which will likely be very useful!
Install the yelpapi module which is online both has a page on the Python Package Index (PyPI) website and has a Github page with some documentation. As I said in class, you can either do this by (a) opening a terminal on your system and running pip install yelpapi or you can try running %run pip install yelpapi in your Jupyter notebook. Reach on out teams or in open lab sessions if you run into trouble.
Create a new .py file (e.g., I called mine yelp_authentication.py) in the same directory as Yelp notebooks that are using and add your API key to it. Then use the import</code command to use your API key in a notebook without having the key itself visible in the notebook!


Once you've done this, use this to grab a list of 50 businesses of any time (your choice!) in any city (again, your choice!) using Yelp and the yelpapi module. This should be easy if you modify  notebook from the Community Data Science Course (Spring 2023)/Week 6 lecture.
Once you have done this, add some code so that you save the "raw" JSON output to a .json or .jsonl file (whichever is appropriate).
Now create a second notebook that opens up that file, reads the data, and outputs a TSV file with the the name of the business, the average raiting, and at least three other pieces or metadata that are available in the Yelp API.

#3 Progress on your final project

Answer these quetions using markdown cells in a notebook. Check out the notes I left about this in the Community Data Science Course (Spring 2023)/Week 5 coding challenges (the third paragraph) or think back to Kaylea's comments during the Week 6 assignment recap (video [Forthcoming]).
Let us know know:

What is your proposed unit of analysis? In other words, if/when you end up building something like a spreadsheet, what are rows going to represent?
What specific measures associated with each unit do you want to collect? In other words, what are your columns in the spreadsheet going to be?
Tell us what you've learned about the API:
Are you going to be able to get the data you want with one API call or many? If more than one,

how many?

If it's more than one call, how will you know when you have collected all your data?
Make one API call and save the output to your desk in either a .json or .jsonl file. Be sure to share the code you used to do this. Be sure not to include any API keys in your notebook!
How big is the JSON file that you saved on your disk (i.e., in bytes or kilobytes)? If it is not your full dataset, what is your estimate for how much larger the full dataset will be? How big will the total dataset be? Is that a problem?