Materials and Resources.

CREX

CREX (CReate, Enrich, eXtend) is a framework allowing the creation the extension and the enrichment of crowdsourcing datasets. CREX allows a clustering based tasks selection and the generation of crowdsourcing campaign sites. Code is in Python for the computational parts and in Javascript for the campaign generation tool.


CREX
CREX-D
CREX-C
Docs
V.0.1

CrowdED

CrowdED is a crowdsourcing evaluation dataset. It consists of 300K+ contributions collected from 400 workers for 1000+ questions distributed over 500+ tasks. It also contains a declarative profiles for each worker, a self evaluation (1-5 rating) in different knowledge domains and a crowdsource consistency relevance of this profile. Tasks belong to various domains such as sport, fashion, economy, politics etc., and have different types, relevance judgment, data annotation, image labelling etc.


CrowdED
Statistics

CAWS

CAWS (Context-Aware Worker Selection) is a framework that learns during an offline stage the relation between the various types of tasks and the declarative profiles of reliable workers. The task types are determined through a content-based clustering process. The learned models are then used in an online stage to select reliable workers within the available crowd.
CAWS