Cost forecasting allows the crowdsourcer apply efficient algorithms to crowd-generated microtasks, which lets the crowdsourcer get high quality results even when the set of tasks is open-ended or if the budget is cut short unexpectedly.

University of Rochester

- GRI: Request a new task when the expected cost of a new task is less than the minimum cost of any existing task.
- GRII: Request a new task when the expected cost of a new task is less than the median cost of the set of already-existing tasks.
- GRII is included in case of outliers that would make the overall minimum an inappropriate choice.

Growth Rules to Test Cost Forecasting (Efficient crowdsourcing of crowd-generated microtasks)

1. Efficient algorithms applied to crowdsourcing
2. Higher accuracy for smaller budgets
3. Useful for dynamic costs

Advantages of Cost Forecasting

Hotaling, A., &amp; Bagrow, J. P. (2020). Efficient crowdsourcing of crowd-generated microtasks. PLOS ONE, 15(12). https://doi.org/10.1371/journal.pone.0244245 

Efficient crowdsourcing of crowd-generated microtasks

Because the data for both Growth Rules fit a power-law distribution, cost forecasting is a bursty process. This means that algorithm alternates between requesting new tasks in short bursts and receiving responses to existing tasks for quite some time.

Cost Forecasting Can Be Modeled as a Bursty Process

1. The maximum number of responses allowed
2. Larger seed sizes
3. Smaller seed sizes
4. Brief inaccuracy

Constraints on Cost Forecasting

Cost forecasting can result in higher accuracy for smaller budgets (≤ ≈5%) because it dynamically determines growth rate with past and current state of crowdsourcing.

Advantages of Cost Forecasting: Higher Accuracy for Smaller Budgets

Even though cost forecasting assumes each task is equally costly, it can still be helpful when that is not the case.

Learn Before

Related