In order to complete your project, you will each need a dataset. If you are at the stage of your career where you already have a dataset, great! If not, there are many datasets to draw from. Here are some ideas:
- If there's an author of a study you loved, you can send a polite email asking if they are able or willing to share an archival or replication version of the dataset used in their paper. Be very polite and make it clear that this is starting as a class project but that might turn into a paper for publication. Make your timeline clear. In communication, replication datasets are still very rare, so be prepared for a negative answer.
- Do some Google Scholar and normal Google searching for datasets in your research area. You'd be surprised at what's available.
- This Google Doc provides documentation and descriptions of datasets (and potential research questions) from Yelp, Data.seattle.gov, and Wikimedia.
- Take a look at datasets available in the Harvard Dataverse (the largest collection of social science research data) or one of the other members of the Dataverse network.
- Look at the collection of social scientific datasets at ICPSR (UW is a member). There are an enormous number of very rich datasets.
- Use the ISA Explorer to find datasets. Keep in mind the large majority of datasets it will search are drawn from the natural sciences.
- If you're interested in using Wikimedia/Wikipedia data, and want to know what's available, come talk to Jonathan and Oliver.
Other open online datasets
- Data.gov provides access to a variety of US Federal datasets and data sources, along with an API and online tools for searching for data.
- The Sunlight Foundation used to provide a one-stop shop for datasets and tools around political activity in the USA. The Foundation has closed down, but their website points to a variety of other organizations, datasets, and tools for accessing public civic data.
- Figshare contains many open datasets across scientific disciplines.
- The Internet Archive contains thousands of curated datasets of all types, including the complete corpus of StackExchange.com, among many others.
- r/datasets on Reddit contains links to many interesting datasets, although some of these data may not be freely licensed.
- Elite Data Science has published a list of public datasets that can be used for a variety of data science and machine learning projects.
- Kaggle also provides many data science datasets that can be explored and used. However, the license terms of these datasets are often absent or incorrect!
Dataset documentation examples
- Examples of well-documented open research projects
- Keegan, Brian. WeatherCrime. GitHub, 2014.
- Geiger, Stuart R. and Halfaker, Aaron. Operationalizing conflict and cooperation between automated software agents in Wikipedia: A replication and expansion of "Even Good Bots Fight". GitHub, 2017.
- Narayan, Sneha et al. Replication Data for: The Wikipedia Adventure: Field Evaluation of an Interactive Tutorial for New Users. Harvard Dataverse, 2017.
- Examples of not-so-well documented open research projects
- Eclarke. SWGA paper. GitHub, 2016.
- David Lefevre. Lefevre and Cox: Delayed instructional feedback may be more effective, but is this contrary to learners’ preferences? Figshare, 2016.
- Alneberg. CONCOCT Paper Data. GitHub, 2014.
