Google

Instruction alignment and human preference alignment are driven by distinct goals. The primary motivation for instruction alignment is to make a model generate outputs that adhere closely to explicit user commands, whereas human preference alignment is motivated by the need to train a model based on broader, often implicit, human feedback.

Differing Motivations of Instruction and Human Preference Alignment

A research team is refining a language model using two distinct methods. In Method A, they train the model on a large dataset of specific commands paired with ideal, human-written responses that perfectly execute those commands (e.g., Command: 'List three benefits of solar power.' Ideal Response: A list of exactly three benefits). In Method B, they show human raters two different model-generated responses to the same open-ended prompt (e.g., 'Write a short, encouraging note') and ask the raters 

Based on the chatbot's behavior described below, which of the two primary alignment motivations—adherence to explicit commands or alignment with broader human preferences—is well-addressed, and which is deficient? Justify your answer by connecting specific examples from the case study to the core goal of each motivation.

Diagnosing Chatbot Performance Issues

A development team is working on two separate improvement goals for their language model. Match each goal with the alignment methodology it primarily represents.

Learn Before

Related