Google

The BART (Bidirectional and Auto-Regressive Transformers) model employs two specific corruption strategies that are designed for sequences containing multiple sentences.

BART Model's Corruption Methods for Multi-Sentence Sequences

Sentence reordering is a data corruption technique applied to multi-sentence inputs where the original order of sentences within a document is randomly shuffled. The model is then trained on this permuted sequence with the objective of learning to restore the sentences to their correct, logical order.

Sentence Reordering as an Input Corruption Method

Analyze why this specific data corruption technique is particularly well-suited for the engineer's stated goal. What fundamental aspect of language is the model forced to learn through this process?

Choosing a Pre-training Strategy

A model is pre-trained using corruption techniques that operate on the structure of multi-sentence documents. Match each technique with its correct description.

A pre-training process is applied to the following two-sentence input: 'The team celebrated their victory. They had trained hard for months.' The input is transformed into: 'The team [MASK] hard for months.' The model is then tasked with reconstructing the original, complete text from this transformed input. Which specific data corruption technique, designed for handling sequences of text, does this process exemplify?

Learn Before

Related