Google

An example of a user prompt that an LLM might receive at the beginning of the Reinforcement Learning from Human Feedback (RLHF) process is: 'How can I live a more environmentally friendly life?' This type of input is used to generate multiple responses from the model, which are then evaluated by humans.

Example of a User Prompt in RLHF

A development team is refining an AI assistant designed to help users brainstorm creative story ideas. Their refinement process involves giving the AI a single starting instruction, generating several different story concepts based on it, and then having human writers rank these concepts from best to worst. The team's primary goal is to teach the AI how to generate more imaginative and diverse narratives.

The team is considering two potential starting instructions:
1. "Write a story about a detective who solves a crime."
2. "What is the definition of the 'hero's journey' story structure?"

Analyze both options and determine which one is more suitable for the team's refinement process. Explain your reasoning, focusing on why your chosen option is better aligned with the goal of teaching the AI to be more imaginative and diverse.

Optimizing an AI for Creative Brainstorming

Consider two types of questions you could give to an AI model during a training phase where human reviewers will rank its multiple generated answers:
1. 'What is the capital of France?'
2. 'What are some fun and educational activities to do in Paris?'

Explain why one of these questions is significantly less effective than the other for this specific training purpose.

Evaluating Prompts for AI Training

Imagine you are part of a team training a new AI assistant. A key step in this process involves providing the AI with a single question, generating multiple different responses to it, and then having human reviewers rank these responses from best to worst. This helps the AI learn what constitutes a high-quality answer. Which of the following questions would be most effective for this specific training step?

Learn Before

Related