Editing Community Data Science Course (Spring 2023)/Week 5 coding challenges

From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 4: Line 4:


* [[../Week 5 lecture notes]]
* [[../Week 5 lecture notes]]
* Notebooks from the Week 5 lecture including:
* The [Week 5 lecture notebook]
** [https://github.com/kayleachampion/spr23_CDSW/blob/main/curriculum/week5/week_5_lecture_part_1-data_collection.ipynb Week 5 lecture notebook part 1] - Data Collection
* The [Week 5 lecture video]
** [https://github.com/kayleachampion/spr23_CDSW/blob/main/curriculum/week5/week_5_lecture_part_2-data_processing.ipynb Week 5 lecture notebook part 2] - Data Processing
** [https://github.com/kayleachampion/spr23_CDSW/blob/main/curriculum/week5/week-5-lecture_pre-baked_workthrough-20230424.ipynb Week 5 lecture notebook (prebaked)] — A combination with the three notebooks above with versions of the code that I wrote as notes for myself before the class.
* The [https://uw.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=8133566f-217d-4e45-b3c9-afef0152528b Week 5 lecture video]


== #1 Wikipedia Page View API ==
== #1 Wikipedia Page View API ==


# Identify a famous person who has been famous for at least a few years and that you have some personal interest in. Use the Wikimedia API to collect page view data from the English Wikipedia article on that person. Now use that data to generate a time-series visualization and include a link to it in your notebook.
# Identify a famous person that you are interested in and collect page view data on that person. Generate a time-series visualization and include a link to it in your notebook.
# Identify 2 other languages editions of Wikipedia that have articles on that person. Collect page view data on the article in other languages and create a single visualization that shows how the dynamics and similar and/or different. (Note: My approach involved creating a TSV file with multiple columns.)
# Identify 2 other languages editions of Wikipedia that have articles on that person. Collect page view data on the article in other languages and create a single visualization that shows how the dynamics and similar and/or different.
# Collect page view data on the articles about [https://en.wikipedia.org/wiki/Marvel_Comics Marvel Comics] and [https://en.wikipedia.org/wiki/DC_Comics DC Comics] in English Wikipedia. (If you'd rather replace these examples with some other comparison of popular rivals, that's just as good!)
# Collect page view data on [https://en.wikipedia.org/wiki/Marvel_Comics Marvel Comics] and [https://en.wikipedia.org/wiki/DC_Comics DC Comics] in Wikipedia. (If you'd rather replace these examples with some other comparison of popular rivals, that's fine.)
## Which has more total page views in 2022?
## Which has more total page views in 2022?
## Can you draw a visualization in a spreadsheet that shows this? (Again, provide a link.)
## Can you draw a visualization of this?
## Were there any years when 2022's more popular page was instead the less popular of the two? How many and which ones?
## Where there years since 2015 when the less viewed page was viewed more? How many and which ones?
## Were there any months was this reversal of relative popularity occurred? How many and which ones?
## Where their any months was this true? How many and which ones?
## How about any days? How many?
## How about any days? How many?
# I've made [https://github.com/kayleachampion/spr23_CDSW/blob/main/curriculum/week5/list_of_washington_alternative_rocks_bands_wikipedia-2023-04-25.jsonl this file available] which includes list of more than 100 Wikipedia articles about alternative rock bands from Washington state that I built from [https://en.wikipedia.org/wiki/Category:Alternative_rock_groups_from_Washington_(state) this category in Wikipedia].[*] It's a <code>.jsonl</code> file. Download the file (click "raw" and then save the file onto your drive). Now read it in, and request monthly page view data from all of them. If you need some help with loading it in, I've included some sample code at the bottom of this page.
# I've made this file available which a list of several hundred titles of Wikipedia articles about Harry Potter {{forthcoming}}.[*] I think it's all of them! Download this file, read it in, and request monthly page view data from all of them?
## Once you've done this, sum up all of the page views from all of the pages and print out a TSV file with these total numbers.
## Once you've done this, sum up all of the page views from all of the pages and print out a TSV file with these total numbers.
## You know the routine by now! Now, make a time series graph of these numbers and include a link in your notebook.
## Make a time series graph of these numbers and include a link in your notebook.


== #2 Starting on your projects ==
== #2 Starting on your projects ==


{{notice|If you are planning on collecting data from Reddit, please look into using the [https://pushshift.io Pushshift API] instead of the default Reddit API. The Pushshift API is not as up-to-date but it is targeted toward data scientists, not app-makers, and is likely much better suited to our needs in the class. That said, take a look at both!}}
{{notice|If you are planning on collecting data, please look into using the [https://pushshift.io Pushshift API] instead of the default Reddit API. The Pushshift API is not as up-to-date but it is targeted toward data scientists, not app-makers, and is much better suited to our needs in the class.}}


In this section, you will take your first steps towards working with your project API. Many of these questions will not involve code, so just mark down your answers in cells in your notebook.  
Many of these challenges will not involve code. Feel free to just write "markdown" code into your notebook.
 
One very useful trick is to convert cells into "markdown" mode. You can do in the menu with ''Cell→Cell Type→Markdown'' or you can just type <code>m</code> when the cell is selected but not being edited (just press <code>Esc</code> if you are editing to switch out of edit mode). Clicking <code>y</code> turns it back into code. Markdown is just normal text but if you want to do fancier stuff like links or formatting you can look at this [https://www.markdownguide.org/cheat-sheet/ Markdown Cheat Sheet].
 
Feel free to document any findings you think might be useful as you continue to work on your project; you might thank yourself later!


# Identify an API you will (or might!) want to use for your project.
# Identify an API you will (or might!) want to use for your project.
# Find documentation for that API and include links in your notebook.
# Find documentation for that API and include links
# What are the API endpoints you plan to use? What are the parameters you will need to use at that endpoint?
# What are the endpoints you plan to use? What are the parameters you will need to use?
# Is there a Python module that exists that helps make contact with the API? (See if you can you find example code on how to use it).  
# Is there a python module that exists that helps make contact with the API? (See if you can you find example code on how to use it).  
## If so, download it, install it, and import it into your notebook.  
# If so, download it, install it, and import it into your notebook.  
# Does the API require authentication? Does it need to be approved?
# Does the API require authentication? Does it need to be approved? If so, sign up for a developer account and get your keys.  
## If so, sign up for a developer account and get your keys. (Do this early because it often takes time for these accounts to be approved.)
# Does the API list rate limits?
# Does the API list rate limits? Does it make any requests about how you should use it?
# Make a single API call, either directly using requests or using the Python module you have used. It doesn't matter for what. The goal is that you can make technical contact.
# Make a single API call, either directly using requests or using the Python module you have used. It doesn't matter for what. The goal is that you can get ''something'''.
# '''IMPORTANT:''' If you have included any API keys in your notebook, ''make a copy of your notebook, delete the cell where you include the keys, and then upload the copy of the notebook.'' We'll show you some tricks for hiding this information going forward.
# '''IMPORTANT:''' If you have included any API keys in your notebook, ''make a copy of your notebook, delete the cell where you include the keys, before you upload the copy of the notebook.'' We'll show you some tricks for hiding this information going forward.


== Notes ==
== Notes ==
[*] You will probably not be shocked to hear that I collected this data from an API! I've included a Jupyter Notebook  with the code to grab that data from [https://petscan.wmflabs.org/ the PetScan API] [https://github.com/kayleachampion/spr23_CDSW/blob/main/curriculum/week5/get_washington_alternative_rock_bands_list-20230425.ipynb in the form of this Github notebook].
[*] You will probably not be shocked to hear that I collected this data from an API! I've included a Jupyter Notebook  with that API online here. {{forthcoming}}
 
If you just want to read it in the file, remember it's just a JSONL file so you can modify the code from the lecture and it should work (e.g., something with <code>open()</code> and the <code>.readlines()</code> function associated with file variables.
Please note that all contributions to CommunityData are considered to be released under the Attribution-Share Alike 3.0 Unported (see CommunityData:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To protect the wiki against automated edit spam, we kindly ask you to solve the following CAPTCHA:

Cancel Editing help (opens in new window)

Template used on this page:

Retrieved from "https://wiki.communitydata.science/Community_Data_Science_Course_(Spring_2023)/Week_5_coding_challenges"
Attribution-Share Alike 3.0 Unported
Powered by MediaWiki