Google

A key characteristic of Large Language Models is their capacity to adhere to the prompts given by users to execute a wide range of tasks. This capability is commonly referred to as the instruction-following ability of the model.

Instruction-Following Ability of LLMs

Analyze the provided model output in response to the user's prompt. Based on the model's ability to follow the given instructions, evaluate its performance and identify which specific constraints were not met.

Evaluating an LLM's Instruction-Following Performance

A user provided the following prompt to a language model:

**Prompt:**
'Describe three key features of a modern smartphone. Present your answer as a numbered list. Each point must be a complete sentence and contain no more than 12 words. Do not mention the brand of any phone.'

**Model's Response:**
1. High-resolution cameras for stunning photos.
2. The iPhone has a powerful processor for fast performance.
3. Long battery life and fast charging capabilities.

Identify **one** specific instruction from the prompt that the model failed to follow. Then, rewrite the corresponding part of the model's response to make it compliant with that instruction.

Analyzing and Correcting Instruction-Following Failures

A language model was given the following text and a specific set of instructions. Analyze the four responses below and identify which one demonstrates the best ability to follow *all* the given instructions.

**Text:**
'Solar energy is a renewable power source derived from the sun's radiation. It offers a significant advantage by reducing greenhouse gas emissions, which is crucial for combating climate change. Furthermore, solar panels can be installed on rooftops, making use of otherwise unused

Learn Before

Related