Not logged in
Talk
Contributions
Create account
Log in
Navigation
Main page
About
People
Publications
Teaching
Resources
Research Blog
Wiki Functions
Recent changes
Help
Licensing
Page
Discussion
Edit
View history
Editing
Designing Internet Research (Winter 2020)/MTurk Notes
(section)
From CommunityData
Jump to:
navigation
,
search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
== The Worker/Turker side of the market == Let's spend 20-30 minutes talking about our experience on the workers side of the market. In preparation for this workshop I personally: * Spent probably 8 minutes painstakingly drawing boxes around palm fruits for $0.90 only to have my work rejected because I took more than 5 minutes to finish the task. I submitted the work and got nothing. * Did a bunch of transcription of receipts for 1Β’ and 2Β’ which was incredibly difficult, painstaking, etc. I mostly failed to get paid. One major challenge is that I did not see/read documentation until it was too late. === Turker Experience === A few things you should know about the MTurk labor market: <ul> <li>The vast majority of work on MTurk falls into a few categories: image/video/audio transcription or transcriptions; data collection</li> <li><p>Quite a lot of the work is now run in connection to AI/ML based systems. Things that a computer cannot accurately do are kicked to a person. The person often ends up training the computer.</p> <p>The vast majority of these HITs are created programmatically with an API.</p></li> <li>A substantial amount of work is sketchy. "Write a positive review of X on website B" or "collect phone numbers/emails for business" are often obviously part of spamming or astroturfing operations. These are often typically against the rules but are still incredibly common.</li> <li>There's a ton of extra work associated with finding accepting HITs that Turkers are not paid for.</li> <li>Pay is frequently extremely shitty. Sometimes near zero. This is especially for an inexperienced worker. There is also a 20-25% (minimum 1Β’) fees charged by Amazon.</li> <li>Payment can sometimes not come because payees have the options to review and accept/reject work.</li> <li>Things are particularly bad for new workers because the vast majority of tasks (especially better ones) on the labor market require "qualifications." <ul> <li>The most common is '''master''' qualification which is determined by Amazon using a secrete algorithm, can be incredibly difficult to gain. In general, it requires lots of high quality (i.e. approved) work across a range of types of tasks.</li> <li>Inexperienced users have only the dregs of MTurk available to them. The first 1,000 HITs often play very little or (in some cases) nothing.</li></ul> </li> </ul> The MTurk platform has an enormous bias toward support of the "requestor" side of the labor market. That said, Turkers are organized: * [https://www.reddit.com/r/TurkerNation/ TurkerNation] * [https://turkopticon.ucsd.edu/reports Turkopticon] (and [https://turkopticon.info/ Turkopticon 2]) (Both include browser-based scripts to allow Turkers integrate their interface with the labor market to these third-party services. * [https://turkerview.com/ TurkerView] Focused on Payment</li> * [http://www.wearedynamo.org/ We Are Dynamo] β Mostly shut down. Their wiki still has great stuff including thes [http://wiki.wearedynamo.org/index.php?title=Guidelines_for_Academic_Requesters guidelines for academic requesters] which is '''essential reading''' * You should know that Turkers will often reach out to requesters to give feedback on jobs.
Summary:
Please note that all contributions to CommunityData are considered to be released under the Attribution-Share Alike 3.0 Unported (see
CommunityData:Copyrights
for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource.
Do not submit copyrighted work without permission!
To protect the wiki against automated edit spam, we kindly ask you to solve the following CAPTCHA:
Cancel
Editing help
(opens in new window)
Tools
What links here
Related changes
Special pages
Page information