1. Cost Forecasting responds to the current state of the crowdsourcing, trading off expected costs from responses to current tasks with the potential cost of a new unseen task.
2. The "spike trains" at the top indicate that cost forecasting is not a memoryless process; i.e. it works with the context of the past actions and states of the crowdsouring.

Implications of Cost Forecasting as a Bursty Process

Because the data for both Growth Rules fit a power-law distribution, cost forecasting is a bursty process. This means that algorithm alternates between requesting new tasks in short bursts and receiving responses to existing tasks for quite some time.

University of Rochester

Cost forecasting is a technique that allows the crowdsourcer to decide between having the crowd generate more tasks or letting the crowd complete some of the already-generated tasks to more efficiently balance resources in a situation where crowdworkers are both proposing new tasks and completing them.

(Proposed in the paper Efficient crowdsouring of crowd-generated microtasks.)

Cost Forecasting

A bursty process is characterized by long pauses of nonactivity interrupted by sudden bursts of activity. Bursty processes tend to follow power-law distributions.

Bursty Processes (needs a new parent)

- GRI: Request a new task when the expected cost of a new task is less than the minimum cost of any existing task.
- GRII: Request a new task when the expected cost of a new task is less than the median cost of the set of already-existing tasks.
- GRII is included in case of outliers that would make the overall minimum an inappropriate choice.

Growth Rules to Test Cost Forecasting (Efficient crowdsourcing of crowd-generated microtasks)

Hotaling, A., &amp; Bagrow, J. P. (2020). Efficient crowdsourcing of crowd-generated microtasks. PLOS ONE, 15(12). https://doi.org/10.1371/journal.pone.0244245 

Efficient crowdsourcing of crowd-generated microtasks

1. The maximum number of responses allowed
2. Larger seed sizes
3. Smaller seed sizes
4. Brief inaccuracy

Constraints on Cost Forecasting

1. Efficient algorithms applied to crowdsourcing
2. Higher accuracy for smaller budgets
3. Useful for dynamic costs

Advantages of Cost Forecasting

Cost forecasting allows the crowdsourcer apply efficient algorithms to crowd-generated microtasks, which lets the crowdsourcer get high quality results even when the set of tasks is open-ended or if the budget is cut short unexpectedly.

Advantages of Cost Forecasting: Efficient Algorithms Applied to Crowdsourcing

Cost forecasting can result in higher accuracy for smaller budgets (≤ ≈5%) because it dynamically determines growth rate with past and current state of crowdsourcing.

Learn Before

Related

Learn After