Google

Weak-to-Strong Performance, denoted as Pweak→strong, is a metric that measures the performance of a strong model on a test set after it has been fine-tuned using supervision from a weaker model. This metric is used to evaluate the effectiveness of the weak-to-strong fine-tuning process.

Weak-to-Strong Performance (Pweak→strong)

Performance Gap Recovered (PGR) is a metric used to evaluate the effectiveness of weak-to-strong generalization. It quantifies the extent to which the performance gap between a weak model's baseline and a strong model's theoretical maximum performance (the ceiling) is closed after the strong model is supervised by the weak one.

Performance Gap Recovered (PGR)

Based on the scenario, describe the specific steps and data required to calculate the 'Weak-to-Strong Performance' (Pweak→strong) metric for Model-B.

Measuring Weak-to-Strong Generalization

To measure the weak-to-strong performance (Pweak→strong) of a powerful model, a specific sequence of actions must be followed. Arrange the core steps below into the correct chronological order.

A research team trains a large, powerful model by fine-tuning it on a dataset labeled by a smaller, less accurate model. After this training process, they evaluate the powerful model on a held-out test set and find its performance is 85%. This 85% figure represents the weak-to-strong performance (Pweak→strong). What is the most accurate interpretation of this result?

Learn Before

Related