Thursday, February 7, 2013

How to Lose Money Using Mturk - The Myth of the Penny Hit

Why are you publishing hits that pay $0.01 per HIT? 

There are two reasons why you should not publish penny HITs.

First, from a requester perspective, there is absolutely no benefit to making a penny hit. Why? It is simple math and fees associated with Amazon publishing that make it a bad choice. Amazon Fees show that the standard payment for HIT completion is 10% of the hit value, but this does not apply for minimum pricing. Minimum pricing shows that for every HIT requesters are charged $0.005. So for your penny hit to be completed, you are going to pay a half cent or 50%! So what is the benefit of having a penny HIT when you are going to pay 5x more than the standard rate? There is no benefit. Even if you have a simple task that requires a single click for human judgement, you are going to lose out by making it a penny HIT.

From a worker perspective, the penny HIT is to be avoided. It is almost impossible to make a fair wage working on them and the risk to worker accounts does not justify the reward. Worker Perspective
Workers know that they can at best make $4.37 per hour, on average. But workers have overhead as well as requesters. Electricity, taxes, internet, computer, banking fees, etc. are all things that workers have to pay. This overhead further reduces the amount they can make working on penny HITs.

What is the solution?

It is really simple, put more data in your hits and pay workers more to reduce your cost of publishing. Instead of having one single radio button in your hit and paying a penny, put five radio buttons and pay a nickle. This will reduce your cost and allow you to publish more HITs.

This only addresses one area of the problem though, you are still not paying workers enough to work for you. Workers are smart, they will see what you have done and they KNOW why your hits are designed this way. There is a way around it so that you can pay workers fairly and still keep your costs within reason. Qualify your workforce so that only trusted workers can work for you.
You may wonder how a qualified workforce can SAVE you money, but this is also simple. When you can trust your workforce, you do not have to publish your hits multiple times to ensure accurate results. I know requesters with unqualified workforces who pay less than $3 to have their HITs completed. BUT they are paying for the same hit to be completed 6 times! Why not triple the pay to one qualified and trusted worker to ensure that you are getting accurate results? Your workforce is happy because you are paying fairly, your overhead is reduced by not overpaying Amazon and you are getting the results you hoped for when using Amazon Mechanical Turk.

No comments:

Post a Comment