Editing Community Data Science Workshops (Fall 2015)/Day 2 projects/Wikipedia
From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
[[File:Wikipedia.png|right|250px]] | [[File:Wikipedia.png|right|250px]] | ||
__NOTOC__ | __NOTOC__ | ||
== Building a Dataset using the Wikipedia API == | |||
In this project, we will explore a few ways to gather data using the Wikipedia API. Once we've done that, we will extend this to code to create our own datasets of Wikipedia edits or other data that we might be able to use to ask and answer questions in the final session. | In this project, we will explore a few ways to gather data using the Wikipedia API. Once we've done that, we will extend this to code to create our own datasets of Wikipedia edits or other data that we might be able to use to ask and answer questions in the final session. | ||
== Goals == | === Goals === | ||
* Get set up to build datasets with the Wikipedia API | * Get set up to build datasets with the Wikipedia API | ||
Line 16: | Line 17: | ||
(Estimated time: 10 minutes) | (Estimated time: 10 minutes) | ||
===Download the Wikipedia API project=== | ===Download the Wikipedia API project=== | ||
* Right click the following file, click "Save Target as..." or "Save link as...", and save it to your Desktop directory: http://mako.cc/teaching/2015/cdsw-autumn/wikipedia-data-examples.zip | * Right click the following file, click "Save Target as..." or "Save link as...", and save it to your Desktop directory: http://mako.cc/teaching/2015/cdsw-autumn/wikipedia-data-examples.zip | ||
* The ".zip" extension on the above file indicates that it is a compressed Zip archive. We need to "extract" its contents. To do this, | * The ".zip" extension on the above file indicates that it is a compressed Zip archive. We need to "extract" its contents. To do this, open Finder and navigate to your Desktop directory. Find wikpiedia-data-examples.zip on your Desktop and double-click on it to "unzip" it. That will create a folder called wikipedia-data-examples containing several files. | ||
==Test the Wikipedia API code== | |||
<div style="background-color:#CEE7DA; width:80%; padding:1.2em;"> | <div style="background-color:#CEE7DA; width:80%; padding:1.2em;"> | ||
'''On Windows | '''On Windows'' | ||
Start up PowerShell and navigate to the <code>Desktop\wikipedia-data-examples</code> directory where the Wikipedia API code lives. For example, if the Wikipedia API project is at <code>C:\Users\'''YOURUSERNAME'''\Desktop\wikipedia-data-examples</code>, | Start up PowerShell and navigate to the <code>Desktop\wikipedia-data-examples</code> directory where the Wikipedia API code lives. For example, if the Wikipedia API project is at <code>C:\Users\'''YOURUSERNAME'''\Desktop\wikipedia-data-examples</code>, | ||
Line 35: | Line 38: | ||
'''On Mac''' | '''On Mac''' | ||
</div> | |||
will change you into that directory, and | |||
ls | |||
will show you the source code files in that directory. One of the files is "<code>wikipedia1-1.py</code>", which has a "<code>.py</code>" extension indicating that it is a Python script. Type: | |||
python wikipedia1-1.py | python wikipedia1-1.py | ||
Line 47: | Line 50: | ||
at the command prompt to execute the <code>wikipedia1-1.py</code> Python script. Wait a little while while your computer connects to Wikipedia. You should see a series of data from Wikipedia run by on your screen. If you don't, let a staff member know. | at the command prompt to execute the <code>wikipedia1-1.py</code> Python script. Wait a little while while your computer connects to Wikipedia. You should see a series of data from Wikipedia run by on your screen. If you don't, let a staff member know. | ||
== Example topics we might cover in the | |||
* [[Community Data Science Workshops (Fall 2015)/Day 2 projects/Wikipedia/Windows setup|Windows]] | |||
* [[Community Data Science Workshops (Fall 2015)/Day 2 projects/Wikipedia/OS X setup|OS X]] | |||
=== Example topics we might cover in the lecture === | |||
* explain [http://www.mediawiki.org/wiki/API:Main_page MediaWiki], exists on other wikis | * explain [http://www.mediawiki.org/wiki/API:Main_page MediaWiki], exists on other wikis | ||
Line 57: | Line 64: | ||
* get the content of the main page http://en.wikipedia.org/w/api.php?format=json&action=query&titles=Main%20Page&prop=revisions&rvprop=content | * get the content of the main page http://en.wikipedia.org/w/api.php?format=json&action=query&titles=Main%20Page&prop=revisions&rvprop=content | ||
== Resources == | === Resources === | ||
* [https://en.wikipedia.org/w/api.php?action=help&modules=query API documentation for the query module] | * [https://en.wikipedia.org/w/api.php?action=help&modules=query API documentation for the query module] | ||
* [https://en.wikipedia.org/wiki/Special:ApiSandbox API Sandbox] | * [https://en.wikipedia.org/wiki/Special:ApiSandbox API Sandbox] | ||
* [[Sample | * [[Sample API queries]] | ||
[[Category: | [[Category:Fall_2015_series]] |