Friday, February 17, 2017

HITs not getting completed? You may be blocked!

We know that if a worker is blocked, Amazon sends a nasty gram to the worker telling them their account is in jeopardy. It is not a good practice to block workers because it will impact your future work completion times and your reputation as a requester. The best way to deal with workers who are not preforming up to standards is to add them to a qualification to disallow them from participating in your HITs in the future. 

Did you know that workers can block you as a requester? It can be difficult for turkers to find well paid work from requesters who have a good reputation. Many will use scripts to weed out the bad requesters and find well paid work. These scripts can block your requester account so that your HITs are never seen by a majority of the workforce. 

We polled 100 workers and asked them "Do you use HIT Scraper or some other script to block requesters?". Out of 100 workers 57 said Yes! That is over 50% of the work population who could never even see your work.
Using scripts to auto complete HITs is a violation of Amazons Terms of Service, but using scripts to scrape the database to find well paid work is allowed. 

There are a few scripts that will allow participants to block requesters, the photo below is of HIT Scraper. HIT Scraper will scrape Mturk at designated intervals and show work available, the number of HITs available, the Turkopticon score and the amount the work pays. 

If workers do not like your reputation, your pay, the amount of work required or for any reason, all they have to do is click one button and they will never see your requester account again.

There are a variety of scripts available to workers. Some calculate daily earnings, some ping when their favorite requesters post, some are complete packages to make Mechanical Turk function better, but all are useful tools to help workers maximize their time on Mturk.

How do you avoid being blocked by workers?
It really is easy.

1. Pay a fair wage - If you do not know what to pay a good rule of thumb is .15 per minute or $9 an hour.
2. Respond to worker emails
3. Test your HITs before you publish
4. If you must reject, provide sound reasons for rejecting and do not just upload a .csv file with "x" marked for the rejection.
5. Maintain a good qualification to prevent problematic participants from working for you in the future. This will help so that you do not end up rejecting the same people over and over.

No comments:

Post a Comment