MTurk Attention Check based on the time spent on the survey

We obtained two samples via the online crowdsourcing marketplace, Amazon
Mechanical Turk (MTurk). The first round of data collection occurred in the fall of 2013 (n =
215) and the second in spring of 2014 (n = 606); both included U.S. individuals only.
Participants in 2013 received $0.20 for participating in a 5-10 minute survey and participants in
2014 received $1.50 for participating in a 25-minute survey, which included additional measures
in addition to those used in 2013. Twenty-three individuals participated in both rounds of data
collection; thus, their later entry was removed. Seven participants (3.2% of 215) of 2013 who
worked on the study for less than 90 seconds (average time taken was 195.44 seconds SD =
108.98) were excluded as inattention to research materials might undermine data quality (Rogge
& Maniaci, 2014). Similarly, thirteen participants (2.2% of 583) of 2014 who worked on the
study for less than three minutes (n = 6) and more than 40 minutes (n = 7) were excluded
(average time taken was 804.82 seconds SD = 422.65, or approximately 13 minutes). We then
removed an additional 37 individuals who failed the manipulation check. In total we excluded 57
participants (7.1%) of the overall sample. Note that 35% of all participants who fell outside of
the reasonable time window failed the manipulation check, whereas this was only the case for
4.8% of participants who completed the study within the expected time.



Maniaci, M. R., & Rogge, R. D. (2014). Caring about carelessness: Participant inattention and its effects on research. Journal of Research in Personality, 48, 61-83.