Editing Community Data Science Workshops (Fall 2015)/Day 2 projects/Twitter
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 3: | Line 3: | ||
__NOTOC__ | __NOTOC__ | ||
== Building a Dataset using the Twitter API == | |||
== Goals == | In this project, we will explore a few ways to gather data using the Twitter API. Once we've done that done, we will extend this to code to create our own datasets of tweets that we might be able to use to ask and answer questions in the final session. | ||
=== Goals === | |||
* Get set up to build datasets with the Twitter API | * Get set up to build datasets with the Twitter API | ||
Line 12: | Line 14: | ||
* Create a few collections of Tweets to use in your projects | * Create a few collections of Tweets to use in your projects | ||
== Prerequisite == | === Prerequisite === | ||
To participate in the Twitter afternoon session, you ''must'' have registered with Twitter as a developer before the session by following the [[Twitter authentication setup|Twitter authentication setup instructions]]. If you did not do this, or if you tried but did not succeed, please attend one of the other two sessions instead. | To participate in the Twitter afternoon session, you ''must'' have registered with Twitter as a developer before the session by following the [[Twitter authentication setup|Twitter authentication setup instructions]]. If you did not do this, or if you tried but did not succeed, please attend one of the other two sessions instead. | ||
== Download and test the Twitter project == | === Download and test the Twitter project === | ||
If you are confused by these steps, go back and refresh your memory with the [[Community_Data_Science_Workshops_(Fall_2015)/Day_0_setup_and_tutorial|Day 0 setup instructions]] | If you are confused by these steps, go back and refresh your memory with the [[Community_Data_Science_Workshops_(Fall_2015)/Day_0_setup_and_tutorial|Day 0 setup instructions]] | ||
Line 24: | Line 26: | ||
===Download the Twitter API project=== | ===Download the Twitter API project=== | ||
# Right click the following file, click "Save Target as..." or "Save link as...", and save it to your Desktop directory: http://mako.cc/teaching/2015/cdsw-spring/twitter-data-examples.zip | |||
# The ".zip" extension on the above file indicates that it is a compressed Zip archive. We need to "extract" its contents. To do this, click on "Start", then "Computer", and navigate to your Desktop directory. Find twitter-data-examples.zip on your Desktop and double-click on it to "unzip" it. That will create a folder called twitter-data-examples containing several files. | |||
===Enter your API information=== | ===Enter your API information=== | ||
Line 32: | Line 34: | ||
'''On Windows''' | '''On Windows''' | ||
* Start your text editor (probably Notepad++ if you [[Windows text editor|installed it following our instructions last time]]). Navigate to the directory that contains Twitter API (probably something of the form <code>C:\Users\'''YOURUSERNAME'''\Desktop\twitter- | * Start your text editor (probably Notepad++ if you [[Windows text editor|installed it following our instructions last time]]). Navigate to the directory that contains Twitter API (probably something of the form <code>C:\Users\'''YOURUSERNAME'''\Desktop\twitter-data-examples</code>). | ||
</div> | </div> | ||
Line 39: | Line 41: | ||
'''On Mac''' | '''On Mac''' | ||
* Start your text editor (probably TextWrangler if you installed it following [[OSX text editor|our instructions]]). Navigate to the directory that contains the Twitter API project (probably something of the form <code>~/Desktop/twitter- | * Start your text editor (probably TextWrangler if you installed it following [[OSX text editor|our instructions]]). Navigate to the directory that contains the Twitter API project (probably something of the form <code>~/Desktop/twitter-data-examples</code>). | ||
</div> | </div> | ||
Line 54: | Line 56: | ||
'''On Windows''' | '''On Windows''' | ||
Start up PowerShell and navigate to the Desktop\twitter- | Start up PowerShell and navigate to the Desktop\twitter-data-examples directory where the Twitter API code lives. For example, if the Twitter API project is at <code>C:\Users\'''YOURUSERNAME'''\Desktop\twitter-data-examples</code>, | ||
cd C:\Users\'''YOURUSERNAME'''\Desktop\twitter- | cd C:\Users\'''YOURUSERNAME'''\Desktop\twitter-data-examples | ||
</div> | </div> | ||
Line 62: | Line 64: | ||
'''On Mac''' | '''On Mac''' | ||
Start a command prompt and navigate to the Desktop/twitter- | Start a command prompt and navigate to the Desktop/twitter-data-examples directory where the Twitter API code lives. For example, if the Twitter API project is at ~/Desktop/twitter-data-examples, | ||
cd ~/Desktop/twitter- | cd ~/Desktop/twitter-data-examples | ||
</div> | </div> | ||
Line 74: | Line 76: | ||
python twitter1.py | python twitter1.py | ||
at the command prompt to execute the <code>twitter1.py</code> Python script. Wait a little while while your computer connects to Twitter. You should see a series of tweets run by your screen. If you don't, let a | at the command prompt to execute the <code>twitter1.py</code> Python script. Wait a little while while your computer connects to Twitter. You should see a series of tweets run by your screen. If you don't, let a staff member know. | ||
== Potential exercises == | === Potential exercises === | ||
'''Who are my followers?''' | '''Who are my followers?''' | ||
# | # Use sample 2 to get your followers. | ||
# For each of your followers, get *their* followers (investigate time.sleep to throttle your computation) | # For each of your followers, get *their* followers (investigate time.sleep to throttle your computation) | ||
# Identify the follower you have that also follows the most of your followers. | # Identify the follower you have that also follows the most of your followers. | ||
Line 88: | Line 90: | ||
'''Topics and Trends''' | '''Topics and Trends''' | ||
# | # Use sample 3 to produce a list of 1000 tweets about a topic. | ||
# Look at those tweets. How does twitter interpret a two word query like "data science" | # Look at those tweets. How does twitter interpret a two word query like "data science" | ||
# Eliminate retweets [hint: look at the tweet object! | # Eliminate retweets [hint: look at the tweet object!] | ||
# For each tweet original tweet, list the number of times you see it retweeted. | # For each tweet original tweet, list the number of times you see it retweeted. | ||
# Get a list of the URLs that are associated with your topic. | # Get a list of the URLs that are associated with your topic. | ||
Line 96: | Line 98: | ||
'''Geolocation''' | '''Geolocation''' | ||
# Alter the streaming algorithm to include a "locations" filter. You need to use the order sw_lng, sw_lat, ne_lng, ne_lat for the four coordinates. | # Alter the streaming algorithm to include a "locations" filter. You need to use the order sw_lng, sw_lat, ne_lng, ne_lat for the four coordinates. | ||
# What are people tweeting about in Times Square today? | # What are people tweeting about in Times Square today? | ||
** Bonus points: set up a bounding box around TS and around NYC as a whole. | |||
# Can you find words that are more likely to appear in TS? | # Can you find words that are more likely to appear in TS? | ||
# UW is playing Arizona in football today. Set up a bounding box around the Arizona stadium and around UW. Can you identify tweets about football? Who tweets more about the game? (you can use <code>d = api.search(geocode='37.781157,-122.398720,1mi')</code> to do a static geo search.) | # UW is playing Arizona in football today. Set up a bounding box around the Arizona stadium and around UW. Can you identify tweets about football? Who tweets more about the game? (you can use <code>d = api.search(geocode='37.781157,-122.398720,1mi')</code> to do a static geo search.) | ||
== Congratulations!!!!== | === Congratulations!!!!=== | ||
You now know how to capture data from Twitter that you can use in your research!!! Next workshop we'll play with some fun analytical tools. In the meantime, here are [[a few words of caution about using Twitter data for science]]. | |||
You now know how to capture data from Twitter that you can use in your research!!! Next workshop we'll play with some fun analytical tools. In the meantime, here are [[ | |||
[[Category:Fall_2015_series]] | [[Category:Fall_2015_series]] |