Community Data Science Course (Spring 2023)/Week 6 lecture notes

Three goals for today's lecture:

1. talk about projects
2. Walk through example code that grabs data from the MediaWiki API and introduces a small number of new concepts
3. walk through example code that grabs data from the Yelp API (and uses a module and authentication)

Final Projects

Your next major milestone is May 15 and it will be a final Project proposal. I'm hoping that you have a clear description of your questions and a clear sense of how you are goint to get data to answer these.

A few points to talk through:

• What are the components of successful project proposal? (e.g., dummy figures, etc)
• A strong sense of whether your work is going to be doable.
• Class assignments will continue to shift toward project work.

Wikipedia Edit Data from the API

Walk through some code and introduce some new concepts:

• MediaWiki: The software that runs many wikis including basicaly every website on Fandom.com
• MediaWiki API with documentation in various places [1] [2]
• Walk through some example code that I've written in this notebook [Forthcoming]

This introduces a few new concepts:

• while True loops
• continuations (i.e., what do you do when you dont' nkow how much data you have?)
• time.sleep()

Yelp API

I also want to walk through an example of a package that comes from an API that is both (a) authenticated and (b) that requires interacting through Python module

• Finding new Python modules
• Installing new Python modules with %run pip install <PACKAGE>

The Yelp API is authenticated. Authenticatino can come in one of several forms including:

• keys that are embedded into your normal parameters (like {'api-key' : 'SOMETHING'})
• OAUTH authentication, bearer tokens, and so on...

Yelp is the latter kind. As it typically any API that lets you post and/or interact in ways that are non-passive.

That means you need to sign up for an API key. To do so at Yelp (and many other places) requires:

• creating a App ("wait... I'm creating an app?!")

Some things to keep in mind include:

• Keeping your API keys outside of your notebook:
  • in a JSON file in your directory!
  • e.g., in a separate python module

Now lets end by walking through two examples:

• Yelp eample notebook #1 [Forthcoming]
• Yelp example notebook #2 [Forthcoming]