Similar to how Amazon Mechanical Turk was used to help us learn about concept recognition in the context of crowdsourcing, our lab has been utilizing the CrowdFlower platform to learn about relationship extraction. Unfortunately, we found a small subset of task participants (henceforth workers) who we suspected to be cheating and not actually doing the requested task. With his Ipython Notebook post, Toby demonstrates how crowdflower data could be utilized in order to discern which CrowdFlower workers were likely to be cheating.
See the data, code, analysis, and results all in one post here:
http://nbviewer.ipython.org/github/noxequo/crowdsourcing_cheaters/blob/master/cheater_detection.ipynb