Cost forecasting can result in higher accuracy for smaller budgets (≤ ≈5%) because it dynamically determines growth rate with past and current state of crowdsourcing.

University of Rochester

1. Efficient algorithms applied to crowdsourcing
2. Higher accuracy for smaller budgets
3. Useful for dynamic costs

Advantages of Cost Forecasting

- GRI: Request a new task when the expected cost of a new task is less than the minimum cost of any existing task.
- GRII: Request a new task when the expected cost of a new task is less than the median cost of the set of already-existing tasks.
- GRII is included in case of outliers that would make the overall minimum an inappropriate choice.

Growth Rules to Test Cost Forecasting (Efficient crowdsourcing of crowd-generated microtasks)

Hotaling, A., &amp; Bagrow, J. P. (2020). Efficient crowdsourcing of crowd-generated microtasks. PLOS ONE, 15(12). https://doi.org/10.1371/journal.pone.0244245 

Efficient crowdsourcing of crowd-generated microtasks

Cost forecasting allows the crowdsourcer apply efficient algorithms to crowd-generated microtasks, which lets the crowdsourcer get high quality results even when the set of tasks is open-ended or if the budget is cut short unexpectedly.

Advantages of Cost Forecasting: Efficient Algorithms Applied to Crowdsourcing

Even though cost forecasting assumes each task is equally costly, it can still be helpful when that is not the case.

Advantages of Cost Forecasting: Useful for Dynamic Costs

Because the data for both Growth Rules fit a power-law distribution, cost forecasting is a bursty process. This means that algorithm alternates between requesting new tasks in short bursts and receiving responses to existing tasks for quite some time.

Cost Forecasting Can Be Modeled as a Bursty Process

1. The maximum number of responses allowed
2. Larger seed sizes
3. Smaller seed sizes
4. Brief inaccuracy

Learn Before

Related