Google

Several strategies can be employed to mitigate overfitting and catastrophic forgetting during Supervised Fine-Tuning. Common techniques include using regularization and early stopping, applying a smaller learning rate to make gentle adjustments to the model's weights, and incorporating data from diverse sources and problem domains to improve robustness.

Strategies to Mitigate Overfitting and Catastrophic Forgetting in SFT

Based on the scenario provided, identify the two primary issues the fine-tuned model is exhibiting. Then, propose one specific strategy the team could implement in their next training run to address these issues and explain why this strategy would be effective.

Diagnosing and Correcting a Fine-Tuning Process

A development team is fine-tuning a general-purpose language model to specialize in generating code for a specific programming language. To achieve high performance quickly, they use a large, homogenous dataset of code from a single project, set a high learning rate, and train for a large number of epochs. Critique this strategy. Identify two major risks associated with this approach and, for each risk, recommend a specific technique to mitigate it, explaining how your recommendation would lead to a better-performing final model.

Evaluating a Flawed Fine-Tuning Strategy

A machine learning engineer is fine-tuning a pre-trained language model and observes several undesirable behaviors. Match each observed behavior with the most appropriate mitigation strategy.

A development team is fine-tuning a large, pre-trained language model on a specialized dataset of medical research papers. They observe that while the model's performance on medical queries is excellent, it has started to perform poorly on simple, general-knowledge questions it could previously answer correctly. Which of the following adjustments to the fine-tuning process is the most direct and effective strategy to address this degradation in general capabilities?

Learn Before

Related