Mechanical Turk Use in Psychological Experiments and the Courts

With contributions by Teresa Kline

In this post, we’ll look at some examples of how Mechanical Turk (“MTurk”) has been used in academia and litigation. In the academic realm, MTurk has often been used for surveys, psychological experiments, and behavioral experiments.As we discussed in the first post of this series, MTurk has a large, diverse respondent pool that doesn’t have the same limitations as the traditional university-student subject pools. In fact, MTurk has been found to be more representative of the population and provide the same quality results as the university-student subject pool. Researchers have replicated several classic experiments in behavioral economics using MTurk, including the following:

• The Prisoner’s Dilemma: Participants imagine they are one of two prisoners in solitary confinement with no means of communicating with the other. Prosecutors lack sufficient evidence to convict the pair on the principal charge, but they have enough to convict both on a lesser charge; they offer each prisoner a bargain. Each prisoner is given the opportunity either to betray the other by testifying that the other committed the crime or to cooperate with the other by remaining silent. The outcome of these options is that the prisoner chooses to protect him- or herself at the expense of the other prisoner.

• Public-Goods Game: Participants are asked to contribute funds to a public pool, which is augmented by some value based on the amount contributed. The augmented funds are then distributed evenly among all participants. Participants are more likely to increase their initial contributions if their neighbors were also high contributors and more likely to decrease their initial contributions if their neighbors were low contributors.

• The Everest Experiment: Participants are asked to estimate the height of Mount Everest as greater or less than a series of measurements. For instance, some are asked whether the mountain is greater or less than 2,000 feet, while others are asked if it is greater or less than 45,000 feet. People who are given the higher anchor (45,000 feet) tend to provide larger estimates than those who are given the lower anchor (2,000 feet).

• FTC v. National Urological Group
• Brighton Collectibles v. Believe Production
• Ebin v. Kangadis Food
• Fagan v. Neutrogena
• Saint-Jean v. Emigrant Mortgage
• Wikimedia Foundation v. National Security Agency