As a new distributed computing model, crowdsourcing lets people leverage a crowds intelligence and wisdom toward solving problems. This thesis aims to characterize various dimensions of worker quality control, which is a crucial component of crowdsourcing systems, typically occupying a large fraction of the time and money invested in crowdsourcing.

In this thesis, we implement several state-of-the-art works related to worker error rate estimates in crowdsourcing using three large open datasets, by which we have a discussion about the efficiency and accuracy of different methods under various circumstances. Besides that, we propose to evaluate the crowd with confidence. This provides a confidence interval in addition to a simple numeric worker error rate and further leads to va...[ Read more ]

As a new distributed computing model, crowdsourcing lets people leverage a crowds intelligence and wisdom toward solving problems. This thesis aims to characterize various dimensions of worker quality control, which is a crucial component of crowdsourcing systems, typically occupying a large fraction of the time and money invested in crowdsourcing.

In this thesis, we implement several state-of-the-art works related to worker error rate estimates in crowdsourcing using three large open datasets, by which we have a discussion about the efficiency and accuracy of different methods under various circumstances. Besides that, we propose to evaluate the crowd with confidence. This provides a confidence interval in addition to a simple numeric worker error rate and further leads to various potential applications.