Google

Instruction alignment and human preference alignment are driven by distinct goals. The primary motivation for instruction alignment is to make a model generate outputs that adhere closely to explicit user commands, whereas human preference alignment is motivated by the need to train a model based on broader, often implicit, human feedback.

Differing Motivations of Instruction and Human Preference Alignment

Based on the chatbot's behavior described below, which of the two primary alignment motivations—adherence to explicit commands or alignment with broader human preferences—is well-addressed, and which is deficient? Justify your answer by connecting specific examples from the case study to the core goal of each motivation.

Diagnosing Chatbot Performance Issues

A development team is working on two separate improvement goals for their language model. Match each goal with the alignment methodology it primarily represents.

A research team is refining a language model using two distinct methods. In Method A, they train the model on a large dataset of specific commands paired with ideal, human-written responses that perfectly execute those commands (e.g., Command: 'List three benefits of solar power.' Ideal Response: A list of exactly three benefits). In Method B, they show human raters two different model-generated responses to the same open-ended prompt (e.g., 'Write a short, encouraging note') and ask the raters 

Learn Before

Related