Not logged in
Talk
Contributions
Create account
Log in
Navigation
Main page
About
People
Publications
Teaching
Resources
Research Blog
Wiki Functions
Recent changes
Help
Licensing
Page
Discussion
Edit
View history
Editing
Innovation Communities (Fall 2017)
(section)
From CommunityData
Jump to:
navigation
,
search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
=== November 30: Applications: Human Computation === '''Resources:''' [Accessible through Canvas] * [https://canvas.uw.edu/courses/1115755/files/folder/reading_notes?preview=45100958 Week 9 Reading Notes] <!--* [https://canvas.uw.edu/courses/1115755/files/folder/slides?preview=45047762 Week 8 Slides — Ecological Perspectives on Innovation Communities] --> The class will focus on issues in crowdsourcing and human computation. Our discussion will emphasize [http://mturk.com/ Amazon's Mechanical Turk Marketplace] and [https://www.zooniverse.org/ Zooniverse]. '''Guest Lecture:''' :We'll have a guest lecture from [https://www.microsoft.com/en-us/research/people/justincr/ Justin Cranshaw] at Microsoft Research. He's a researchers at FUSE Labs at Microsoft Research focusing on social computing and human-computer interaction. He's going to talk about a system he built called [https://calendar.help/ calendar.help] which is a system that brings together humans and algorithms in a novel way. '''Required Readings:''' * von Ahn, Luis. [https://www.ted.com/talks/luis_von_ahn_massive_scale_online_collaboration?language=en Massive Scale Human Collaboration] (TedX video lecture). * Chris Lintott's TEDxCERN talk on [https://www.youtube.com/watch?v=kvpUiBqHoVM How to discover a planet from your sofa]. 2013. * Shaw, A. (2015). [https://canvas.uw.edu/courses/1115755/files/folder/readings?preview=45100097 Hired Hands and Dubious Guesses: Adventures in Crowdsourced Data Collection]. In E. Hargittai & C. Sandvig (Eds.), Digital Research Confidential: The Secrets of Studying Behavior Online. The MIT Press. ''[Available in Canvas]'' Additionally, I'll want you to skim these three. Although I have no expectation that you'll be finishing these, it's essential that you do so in order to complete the in class assignment we'll be doing instead of a case: * [https://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/Introduction.html Amazon Mechanical Turk Requester UI Guide] ''[Skim, but make sure you're ready to submit tasks.]'' * [https://mturkpublic.s3.amazonaws.com/docs/MTURK_BP.pdf Amazon Mechanical Turk Best Practices Guide]. ''[Skim, but make sure you're ready to submit tasks.]'' * Cranshaw, Justin, Emad Elwany, Todd Newman, Rafal Kocielnik, Bowen Yu, Sandeep Soni, Jaime Teevan, and Andrés Monroy-Hernández. 2017. “[https://doi.org/10.1145/3025453.3025780 Calendar.Help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop.]” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2382–2393. CHI ’17. New York, NY, USA: ACM. ''[Available through UW libraries]'' '''Assignment ''before'' class:''' Instead of a case, we'll be doing an activity. You'll need to complete the following things before we get to class: * Familiarize yourself and skim the two Amazon Mechanical Turk Guides in the readings above. * Create a "requester" account on [https://www.mturk.com/mturk/welcome mTurk]. Doing so may require up top 48 hours to be approved so please do that ''immediately'' so you have it ready to go in class. * Complete at least 1-2 tasks in two different projects of your choice on [https://www.zooniverse.org/ Zooniverse]. Come to class ready to talk about it. * Find and complete at least 2 "hits" as a worker on [http://mturk.com Amazon Mechnical Turk]. Note that to do this you will need to create a ''worker'' account on Mturk. ** Record (write down) details and notes about your tasks: What did you do? Who was the requester? What could you was the purpose of the task (as best you could tell)? What was the experience like? What research applications can you (not) imagine for this kind of system? ** ''If you are not a US citizen, just skip this.'' This is because working on mTurk involves getting paid and ensuring that you have authorization to work. '''In class exercise:''' * Design and deploy a small-scale research task on Mturk. Note that to do this, you will need to create a ''requester'' account on Mturk. Be sure to allow some time to get the task design the way you want it! Some ideas for study designs you might do: ** A small survey. ** Classification of texts or images (e.g., label tweets, pictures, or comments from a discussion thread). ** A small experiment (e.g., you can do a survey where you insert ''different'' images and ask the same set of questions. Check out the [https://requester.mturk.com/help/getting_started.html Mturk requester getting started guide] * Prepare to share details of your small-scale research task in class, including results (they will come fast). ''Note:'' In terms of running your task, it will cost real money and you have to put money on your Amazon account yourself. You've each got a $3 budget. Please use your credit card to put $3 on your account right away. I will pay each of you $3 in cash next week to reimburse you for the cost of running the experiment.
Summary:
Please note that all contributions to CommunityData are considered to be released under the Attribution-Share Alike 3.0 Unported (see
CommunityData:Copyrights
for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource.
Do not submit copyrighted work without permission!
To protect the wiki against automated edit spam, we kindly ask you to solve the following CAPTCHA:
Cancel
Editing help
(opens in new window)
Tools
What links here
Related changes
Special pages
Page information