Google

The `[CLS]` token, a convention established by BERT models, can be used as the special start symbol for a sequence in language modeling. This serves as a concrete example of an initial token used to begin the probability calculation for a sequence.

[CLS] Token as a Start Symbol

The `[CLS]` token, when used in a language model, is exclusively for aggregating sequence information for downstream classification tasks and does not serve as the initial token required to start the sequence's probability calculation.

A language model calculates the probability of a sequence of tokens by breaking it down into a series of conditional probabilities. Explain the specific, essential function of a special start symbol, such as `[CLS]`, in initiating this calculation.

The Role of the Initial Token in Sequence Probability

A researcher is preparing the sentence 'Language models are powerful.' for input into a specific type of model. This model architecture requires a special token, denoted as `[CLS]`, to be placed at the very beginning of every input sequence to mark its start. Which of the following options correctly formats the sentence for this model?

Learn Before

Related