Editing Twitter (CDSW)
From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 3: | Line 3: | ||
__NOTOC__ | __NOTOC__ | ||
In this project, we will explore a few ways to gather data using the Twitter API. Once we've done that, we will extend the example code to create our own dataset of tweets. In the final workshop ( | In this project, we will explore a few ways to gather data using the Twitter API. Once we've done that, we will extend the example code to create our own dataset of tweets. In the final workshop (Nov. 5), we will ask and answer questions with the data we've collected. | ||
== Goals == | == Goals == | ||
Line 24: | Line 24: | ||
===Download the Twitter API project=== | ===Download the Twitter API project=== | ||
* | * Right click the following file, click "Save Target as..." or "Save link as...", and save it to your Desktop directory: https://mako.cc/teaching/2015/cdsw-autumn/twitter-api-cdsw.zip | ||
* | * The ".zip" extension on the above file indicates that it is a compressed Zip archive. We need to "extract" its contents. To do this on Windows, click on "Start", then "Computer". If you are a Mac, open Finder and navigate to your Desktop directory. Find <code>twitter-api-cdsw.zip</code> on your Desktop and double-click on it to "unzip" it. That will create a folder called <code>twitter-api-cdsw</code> containing several files. | ||
===Enter your API information=== | ===Enter your API information=== | ||
* Start | <div style="background-color:#CEE7DA; width:80%; padding:1.2em;"> | ||
* | '''On Windows''' | ||
* Start your text editor (probably Notepad++ if you [[Windows text editor|installed it following our instructions last time]]). Navigate to the directory that contains Twitter API (probably something of the form <code>C:\Users\'''YOURUSERNAME'''\Desktop\twitter-api-cdsw</code>). | |||
</div> | |||
<div style="background-color:#D8E8FF; width:80%; padding:1.2em;"> | |||
'''On Mac''' | |||
* Start your text editor (probably TextWrangler if you installed it following [[OSX text editor|our instructions]]). Navigate to the directory that contains the Twitter API project (probably something of the form <code>~/Desktop/twitter-api-cdsw</code>). | |||
</div> | |||
* Open up the file <code>twitter_authentication.py</code> in your text editor. | |||
* You will see four lines that include four variables in ALL CAPITALS that are being assigned, in the normal ways we learned about last session, to strings. At the moment, all of the strings say CHANGE_ME. | * You will see four lines that include four variables in ALL CAPITALS that are being assigned, in the normal ways we learned about last session, to strings. At the moment, all of the strings say CHANGE_ME. | ||
* Go find the four keys, tokens, and secrets you created and wrote-down when you followed the [[Community Data Science Workshops/Twitter authentication setup|Twitter authentication setup]]. Change every string that says CHANGE_ME into a string that includes the key, token, or secret you downloaded. Remember that since these are strings, we need to include quotations marks around them. Also make sure that you match up the right keys and tokens with the right variables. | * Go find the four keys, tokens, and secrets you created and wrote-down when you followed the [[Community Data Science Workshops/Twitter authentication setup|Twitter authentication setup]]. Change every string that says CHANGE_ME into a string that includes the key, token, or secret you downloaded. Remember that since these are strings, we need to include quotations marks around them. Also make sure that you match up the right keys and tokens with the right variables. | ||
Line 38: | Line 51: | ||
===Test the Twitter API code=== | ===Test the Twitter API code=== | ||
<div style="background-color:#CEE7DA; width:80%; padding:1.2em;"> | |||
'''On Windows''' | |||
Start up PowerShell and navigate to the Desktop\twitter-api-cdsw directory where the Twitter API code lives. For example, if the Twitter API project is at <code>C:\Users\'''YOURUSERNAME'''\Desktop\twitter-api-cdsw</code>, | |||
cd C:\Users\'''YOURUSERNAME'''\Desktop\twitter-api-cdsw | |||
</div> | |||
<div style="background-color:#D8E8FF; width:80%; padding:1.2em;"> | |||
'''On Mac''' | |||
Start a command prompt and navigate to the Desktop/twitter-api-cdsw directory where the Twitter API code lives. For example, if the Twitter API project is at ~/Desktop/twitter-api-cdsw, | |||
cd ~/Desktop/twitter-api-cdsw | |||
This will | </div> | ||
This will change you into the right directory. | |||
<code>ls</code> will show you the source code files in that directory. One of the files is "<code>twitter1.py</code>", which has a "<code>.py</code>" extension indicating that it is a Python script. Type: | |||
python twitter1.py | |||
at the command prompt to execute the <code>twitter1.py</code> Python script. Wait a little while while your computer connects to Twitter. You should see a series of tweets run by your screen. If you don't, let a mentor know. | |||
== Potential exercises == | == Potential exercises == | ||
'''Who are my followers?''' | |||
# Alter code example 2 (twitter2.py) to get your followers. | |||
# For each of your followers, get *their* followers (investigate time.sleep to throttle your computation) | |||
# Identify the follower you have that also follows the most of your followers. | |||
# How many handles follow you but none of your followers? | |||
# Repeat this for people you follow, rather than that follow you. | |||
'''Topics and Trends''' | '''Topics and Trends''' | ||
# Alter code example | # Alter code example 3 (twitter3.py) to produce a list of 1000 tweets about a topic. | ||
# How does twitter interpret a two word query like "data science" | # Look at those tweets. How does twitter interpret a two word query like "data science" | ||
# Eliminate retweets [hint: look at the tweet object! https://dev.twitter.com/overview/api/tweets] | # Eliminate retweets [hint: look at the tweet object! https://dev.twitter.com/overview/api/tweets] | ||
# For each tweet original tweet, list the number of times you see it retweeted. | # For each tweet original tweet, list the number of times you see it retweeted. | ||
# Get a list of the URLs that are | # Get a list of the URLs that are associated with your topic. | ||
'''Geolocation from Search API''' | '''Geolocation from Search API''' | ||
# Get the last 50 tweets from Ballard. | # Get the last 50 tweets from Ballard. | ||
Line 74: | Line 101: | ||
# A baseball game happened today (May 11) between the Seattle Mariners and the Tampa Bay Rays. Using two geo searches, see if you can tell which city hosted the game. Note: if you do this some other day, you should pick a new sporting event. | # A baseball game happened today (May 11) between the Seattle Mariners and the Tampa Bay Rays. Using two geo searches, see if you can tell which city hosted the game. Note: if you do this some other day, you should pick a new sporting event. | ||
'''Geolocation | '''Geolocation''' | ||
# Alter the streaming algorithm to include a "locations" filter. You need to use the order sw_lng, sw_lat, ne_lng, ne_lat for the four coordinates. (Recall | # Alter the streaming algorithm to include a "locations" filter. You need to use the order sw_lng, sw_lat, ne_lng, ne_lat for the four coordinates. (Recall Control C will stop an active process like the stream.) | ||
# What are people tweeting about in Times Square today? (Bonus points: set up a bounding box around TS and around NYC as a whole.) | # What are people tweeting about in Times Square today? (Bonus points: set up a bounding box around TS and around NYC as a whole.) | ||
# Can you find words that are more likely to appear in | # Can you find words that are more likely to appear in TS? | ||
# | # Boston is playing Houston in baseball right now. Set up a bounding box around the Houston and Boston. Can you identify tweets about baseball? Who tweets more about the game? Can you tell which team is the home team? (you can use <code>d = api.search(geocode='-95.355549,29.757480,5mi)</code> to get Tweets from Houston. Use Google or Bing maps to get a similar bounding box around Fenway Park). | ||
== Congratulations!!!!== | == Congratulations!!!!== |