Google

Long sequence modeling problems can be broadly categorized into three main types based on the relative lengths of the input context and the output text. Using the conditional text generation probability notation $$\Pr(\mathbf{y}|\mathbf{x})$$, where $$\mathbf{x}$$ represents the context and $$\mathbf{y}$$ represents the generated text, these problems involve handling extended token sequences in either the input, the output, or both.

Classification of Long Sequence Modeling Problems

This category of long sequence modeling involves tasks where a language model produces text based on an input context that is an extensive sequence. In terms of the text generation probability notation $$\Pr(\mathbf{y}|\mathbf{x})$$, this corresponds to scenarios where the context $$\mathbf{x}$$ is a long sequence, while the generated text $$\mathbf{y}$$ is typically shorter.

Text Generation Based on Long Context

This category of long sequence modeling focuses on tasks where a language model's output is an extensive sequence of text, often generated from a brief initial input. Using the text generation probability notation $$\Pr(\mathbf{y}|\mathbf{x})$$, this describes scenarios where the generated text $$\mathbf{y}$$ is a long sequence.

Long Text Generation

This category of long sequence modeling involves complex tasks where both the input context and the resulting generated output are extensive token sequences. Represented by the text generation probability notation $$\Pr(\mathbf{y}|\mathbf{x})$$, this describes scenarios where both the context $$\mathbf{x}$$ and the generated text $$\mathbf{y}$$ are long sequences.

Long Text Generation from a Long Context

Match each text generation task with the description that best represents the relationship between the length of its input context (x) and its generated output (y).

Based on the relationship between the input and output sequence lengths described in the scenario, how would you classify this task? Justify your reasoning.

Classifying a Code Refactoring Task

An AI-powered assistant is tasked with summarizing a 200-page research paper into a single, concise paragraph. In the context of text generation probability, represented as Pr(y|x), how would this task be classified based on the relative lengths of the input and output sequences?

Learn Before

Related