Amazon Mechanical Turk Pays Less Than 40% of U.S. Minimum Wage, Study Finds
Findings from a study conducted by researchers in the United States and Mexico suggest that crowdsourcing workers in the Amazon Mechanical Turk marketplace, with a 75% majority based in the United States, earn as little as $ 2.83 per hour on average, or about 39.5% of hourly wages. mandatory federal minimum wage of $ 7.25 an hour, as remote workers spend one-third of their work time scanning job request lists, managing payments, and performing other types of “work”. invisible “.
The findings follow a recent report from the University of Massachusetts Amherst, which criticized the low level of pay and benefits associated with being a “turker,” and which effectively characterizes the AMT’s eco-structure as a poor quality sweatshop for subcontractors. funded university researchers.
Launched in 2005, Amazon Mechanical Turk (AMT) provides crowdworkers with ad hoc Human Intelligence (HIT) tasks, frequently exploited by machine learning researchers in “affordable” study groups for tasks such as image recognition, natural language processing (NLP) assessment, and other systematic programmatic tasks designed to help formulate or test algorithms (although the service is also widely used to generate spam).
Invisible work on MTurk
The new study, titled Quantifying invisible work in crowd work, was carried out by researchers at Northeastern University in Boston, Microsoft Research and the Universidad Nacional Autonoma de Mexico (UNAM).
The article details the extent of “invisible work” which constitutes 33% of a turker’s working time, as evidenced by a field study undertaken for the work. The greatest amount of “wasted” time was represented by the time AMT employees spend reviewing their payment details to ensure they received payment for a task. The report states that “97% of the workers in our study practiced invisible work by visiting the earnings section on their workers’ dashboard (perhaps to make sure they had been paid fairly)”.
The second biggest waste of time for turkers was the “hyper-vigilance” needed to actually get work, where workers refreshed applicant profile pages and looked for new tasks advertised in AMT HIT listings sections.
Surveillance of the Turks
The researchers developed a Chrome extension, which they have since made available on GitHub, based on previous work that investigated the attention habits of AMT workers, but that did not address the assessment of invisible work. The extension was installed on the systems of 100 participating Mechanical Turk workers, who collectively undertook 40,903 HITs over a one-week period.
The main task of the plugin is to assess the actual time a worker spends on a HIT task, and to this end, it monitors the status of AMT-related pages by scratching and parsing the HTML content of the plugin. ‘a page the turker is currently focusing on. , in order to understand the worker’s interactions with it. The extension also has a time-based background process that logs browser events on the MTurk domain and the state of each page (i.e. active, inactive, closed).
Time-consuming aspects monitored under this system include workers checking the status of “skills” (to see if recently completed work has “improved” them, giving them access to better HITs); “Patrol and sort” through an avalanche of HIT ads with search filters to identify a possible next job and exclude unwanted work; and filter “fraudulent” HITS and generally manage / monitor queued HITs.
Delays and streaks
The system also recognizes “delay” events and “false starts”, such as when a worker returns a previously accepted task to the HIT pool because it turned out to be different from their expectations; the time taken to “preview” the HITs or to read the information about the HITs; send messages to the sender of the HIT in order to better understand the requirements of the task; and read general platform help from the Mechanical Turk help pages.
In the very time-consuming category of ‘payments’, research found that AMT workers were frequently at their dashboards to check if they had been paid, as there is a possibility that completed work could be rejected, unpaid, for issues. reasons that the worker will never know. to be told; view their earnings, which have a dedicated section of the MTurk dashboard, as opposed to announcements of a new payout; and perform HITs that ‘expire’ – when a worker fails to complete an HIT within the allotted time and receives no payment for the work performed (a category that researchers believe should be in the ‘payments’ section ).
In addition to the report’s findings that invisible work reduces the previously estimated median wage ($ 3.76) of an AMT worker by 39.5% when factored into a work session, it also notes that the Workers’ perception of the time they spent on unpaid work during a session is significantly lower than what follow-up sessions reveal.
The authors note that they rounded all estimates in their methodological conclusions in order to ensure a rigorously “conservative” approach to the work and to avoid accusations of exaggeration of the conclusions. They conclude:
“Considering that the median hourly wage of workers is only $ 3.76 (not counting invisible work), it is clear that workers in droves still need a dramatic increase in their wages before we can. consider this labor fair. However, this is not only something for applicants to consider, but also something for platforms, workers and even policy makers.
The state of Turkish mechanical workers
Other authors and organizations estimate that the average AMT worker actually earns less than what’s estimated in the new report – as little as $ 1 an hour.
Advocates of AMT benefits argue, with relative frequency, that the controversies surrounding the poor pay of AMT workers are overblown. A 2020 study interviewed part of MTurk workers in the United States, concluding that ” the financial situation of people on MTurk reflects the general population‘, and that the Turks “Wouldn’t trade MTurk’s flexibility for less than $ 25 an hour”.
This work was conducted by employees of CloudResearch (formerly TurkPrime), whose services include “tools that enable researchers to conduct studies on Mechanical Turk”.
Since AMT workers are considered independent contractors, they are generally not covered by labor laws in the United States. In a 2016 survey by the Pew Research Center, more than half of AMT workers (52%) reported earning less than $ 4.99 an hour, which – assuming a full-time job without vacation or sick leave – equals income of $ 10,379.20 per year. year.
For many workers, the survey found that Mechanical Turk made up “all or most” of their income, and this category of Turkish people were typically younger, less educated, and lived in low-income households.
However, according to the survey, Turks were generally more educated than average American workers, with 51% reporting having at least a college degree, compared to 36% of adult workers in the United States – with just 12% of AMT workers reporting no have only a high school diploma or less.
The right thing?
The Pew Report also indicated that typical AMT workers may be the exact opposite of the “average person” that most academic studies seek to exploit, with implications for the extent to which their unusual responses could potentially bias or degrade performance. quality of machine learning projects. who depend on the Turks.
The findings of the Pew report indicate that the Turks “are not representative of all Americans – or even working adults”.
He also notes research from an earlier study (2014), which indicated that AMT workers are subjected to so many experiments and investigations that the amount of this type of work “Could condition workers to certain types of answers or cause them to become too familiar with the types of questions they will be asked” – indeed, that the meticulous work of the AMT will encourage the Turks to become experts in order to “please the system” rather than providing an authentic and useful contribution on the academic (or even clinical) level.