Google

To diversify fine-tuning data and improve a model's generalization, the same underlying task, such as a binary classification problem, can be described using multiple different instructions. This approach exposes the model to various ways a task can be framed, helping it become more robust and less sensitive to specific phrasing.

Using Varied Instructions for a Single Task to Enhance Data Diversity

A development team has fine-tuned a language model to perform a single task: summarizing news articles. They created a high-quality dataset where every example was paired with the exact same instruction: 'Write a one-paragraph summary of the following article.' The model performs exceptionally well on this specific prompt. However, when beta testers use slightly different phrasing like 'Summarize this for me' or 'What are the main points of this text?', the model's performance drops significantly, often generating irrelevant or poorly structured responses. Based on this scenario, identify the most likely cause of this performance gap and propose a specific change to the fine-tuning dataset to resolve it.

Diagnosing a Fine-Tuning Data Issue

You are preparing a dataset to fine-tune a language model for a single-sentence summarization task. To make the model more robust and flexible, you need to create varied instructions for this same task. Provide three distinct instructions you would use in your dataset. Each instruction should prompt the model to summarize a given text into one sentence, but be phrased differently.

Generating Diverse Instructions for a Summarization Task

A grammaticality judgment prompt can be designed by first providing the sentence to be assessed, followed by a related question, and ending with clear multiple-choice options for the answer. For example, the sentence 'LLMs are powerful models but are expensive to build.' is presented at the beginning. This is followed by asking 'Does this sentence make sense grammatically?' and supplying an 'Answer Option' list that explicitly states the choices '- Yes' and '- No', along with the expected final response 'Yes'.

Example of a Sentence-First Prompt for Grammaticality Judgment with Answer Options

When evaluating grammaticality, a prompt can be organized by stating strict constraints on the answer format and meanings before presenting the target sentence. For instance, the prompt might begin with the rules: 'Yes or No? Yes = the following sentence is grammatically correct. No = it contains grammatical errors.' Right after these explicit instructions, the sentence to be judged is provided: 'LLMs are powerful models but are expensive to build.', followed by the desired response 'Yes'.

Example of a Constraint-First Prompt for Grammaticality Judgment

A team is fine-tuning a language model for a single, specific task: extracting the main sentiment (positive, negative, or neutral) from customer reviews. To ensure the final model is robust and can handle the varied ways users might phrase this request, which of the following training data strategies is the most effective?

Learn Before

Related