Google

Span prediction is an NLP task focused on identifying a continuous segment of text. A prominent example is reading comprehension, where a model is given a query (e.g., $$x_1...x_m$$) and a context passage (e.g., $$y_1...y_n$$) and must locate the span within the context that answers the query. This task can be conceptualized as a form of sequence labeling, where the objective is to predict for each token $$y_j$$ in the context whether it marks the beginning or the end of the desired span.

Span Prediction in NLP

Describe how you would formulate this problem as a span prediction task. Specifically, what would the model need to predict for the example text provided to correctly identify the target span?

Framing a Clinical Information Extraction Task

A developer is building a system to extract information from product reviews. One task is to extract the 'product name' mentioned in the review. A second task is to extract all 'mentioned features' of the product, which might be scattered throughout the review text. Explain why a standard span prediction model, which identifies a single start and end position in the text, would be suitable for the first task but would fundamentally fail at the second.

Applicability of Span Prediction

A common architecture for span prediction tasks utilizes BERT by concatenating the query and context text into a single input sequence. To identify the optimal answer span, two distinct prediction networks are placed on top of BERT's output layer. For each token $$y_j$$ within the context text, the first network generates the probability that it marks the start of the answer span (denoted by $$p_j^{\mathrm{beg}}$$), while the second network calculates the probability that it represents the end of the span (denoted by $$p_j^{\mathrm{end}}$$). These prediction networks are exclusively applied to the outputs corresponding to the context text.

BERT-based Architecture for Span Prediction

A language model is tasked with answering a question by identifying the correct text span from a given context. The model works by calculating a probability for each token being the 'start' of the answer and a separate probability for each token being the 'end' of the answer. Consider the following scenario:

**Context:** 'The first modern Olympic Games were held in Athens, Greece, in 1896. The International Olympic Committee (IOC) was founded in 1894 by Pierre de Coubertin.'
**Question:** 'When

Learn Before

Related