Wednesday, August 29, 2012

The Advantages of Opensource Crowdsourcing - Clockwork Raven

When using solution providers on Amazon Mechanical Turk, requesters are paying 4x the amount the workers receive. Often, requesters do not have the budget to go to CrowdSource or CrowdFlower or another middle man.

Twitter is having open discussions with potential requesters informing them on how mturk works. They are also being open about what they are paying workers.Twitter is developing an opensource application called Clockwork Raven, which will make it easy for potential requesters to publish Hits (human intelligence tasks).

Twitter is paying exceptionally well, even by real world standards. When you think about having to pay workers in the real world you have to pay for accounting, workmans comp, and a host of other costs. This makes hiring workers is a very costly expense. It could  require up to $25 an hour or more for just a single minimum wage worker.When a company can use opensource Clockwork Raven and pay a worker $10-15 an hour, they do not have to worry about the other costs. This can translate into a huge savings where both the worker and the requester wins. That is what Amazon Mechanical Turk should be, a mutually beneficial relationship.

Companies like CS and CF, are the ones making money and the requesters and workers are taking the hit. We have been actively working with requesters and getting them away from using the solution providers that are destroying mturk. On the forum we are helping requesters get the most for their money without dealing with a solution provider or having to hire coding professionals with Amazon to design and manage their Hits. These listings on Amazon show the companies which are sucking the life out of mturk . That is only a partial listing that is approved by mturk, there are probably 3x more that are not listed.

When I see people on turkernation, mturkforum, reddit, and TURKOPTICON, talking about how they are making $5 a day, it is upsetting. I make on average $15-20 an hour on Mturk. I do work my ass off for that pay, as do many other workers, and give requesters excellent work in return for fair pay. There is no reason for a company like LinkedIn/Oscar Smith to be paying workers $1.20 an hour. They are a multi-million dollar corporation paying slave wages and hiding behind the CardMunch/Oscar Smith pseudonym.

Studies have been published which state that crowdsourcing is a way to obtain cheap labor and offer far less than minimum wage. Most of these studies are flawed. Academic requesters have done studies paying workers $0.60 an hour to discover their mturk habits, and they only get workers who respond using mturk as a way to pass the time, not serious workers. There are workers on  mechanical turk with such a variety of skills, you can get any task accomplished as long as you are willing to pay fairly.

When a novice looks at the standard Hits on mturk, they only see the poor requesters offering substandard pay because the good work is qualified and hidden from the masses. Also, this low paying work sits and does not get completed, or when it does get completed, it is not completed properly and these people end up having the work done 4 and 5 times to ensure quality results instead of paying one worker to do it right the first time.

There are a lot of companies that would greatly benefit from crowdsourcing if it was only easier and more open. I think opensource is going to be a key in this evolving business model and a way for requesters to profit, workers to profit, and even Amazon to profit.  There are some requesters who are fighting for workers and understand what is going on.

Also, requesters, we have been ranking you guys for years over on the TurkerNation Hall of Fame/Shame and Turkopticon , so if you are not paying fairly and wondering why your Hits are not getting completed, you might want to look there and see what workers are saying about you.


  1. Thanks for posting this! You say twitter is being open about what they are paying. Where is that info available?

  2. I think they mean in the conferences that Twitter has been speaking at.

  3. Gгeеtings! Very helрful аdviсe in thіs paгticulаr ρoѕt! Ӏt іѕ the littlе changes that maκe the gгeаtest сhanges.
    Τhanks fοr sharіng!
    website design