Google

Supervised Fine-Tuning is conceptualized as a distinct training phase that follows the initial pre-training of a model. Its purpose is to make specific adjustments and incorporate new capabilities while aiming to preserve the valuable, general knowledge gained during the pre-training stage.

SFT as a Post-Training Phase

A company has a powerful, general-purpose language model that was initially trained on a vast dataset of internet text, giving it a broad understanding of language and world knowledge. The company now wants to create a specialized chatbot for its e-commerce website to handle customer queries about product details, shipping, and returns. Two strategies are proposed:

**Strategy 1:** Train a new model from scratch using only the company's internal data (product descriptions, shipping policies, return guidelines).

**Strategy 2:** Take the existing general-purpose model and conduct a second phase of training using only the company's internal data.

Evaluate the two proposed strategies. Which strategy is more likely to result in a more capable and helpful customer service chatbot, and why? Justify your reasoning by considering the knowledge base and conversational abilities of the final model in each scenario.

Chatbot Development Strategy

A large language model first undergoes a pre-training phase on a massive, diverse dataset, followed by a supervised fine-tuning phase on a smaller, more specialized dataset. Analyze the distinct contribution of each training phase to the model's final abilities. In your analysis, discuss the primary challenge that arises when trying to add new, specific capabilities during the second phase without diminishing the broad knowledge gained in the first.

Balancing General and Specific Knowledge in Model Training

A development team starts with a large language model that has been pre-trained on a vast corpus of text from the internet, giving it a broad base of general knowledge. To make it a better customer service assistant, they then fine-tune it on a specific dataset of support chat logs. After this fine-tuning, they observe that while the model excels at customer service conversations, its performance on general trivia questions has noticeably degraded. What does this outcome most directly illustrate

Learn Before

Related