Google

In many common scenarios, mapping the textual output of a Large Language Model to a specific label is a straightforward task. This can be accomplished using simple heuristics, such as searching for the presence of predefined label words (e.g., 'positive', 'negative') within the generated sentence.

Heuristic-based Label Mapping for LLM Outputs

A developer is using a Large Language Model to classify customer feedback into two categories: `POSITIVE` or `NEGATIVE`. To map the model's sentence-level output to a final label, they implemented a simple rule: if the word 'positive' appears in the output sentence, assign the `POSITIVE` label. Analyze the following set of model outputs and identify which one would be misclassified by this rule. Explain the fundamental flaw in the rule-based approach that causes this error.

Debugging a Rule-Based Label Mapper

A sentiment analysis system uses a simple rule to map a language model's output to either a `POSITIVE` or `NEGATIVE` label: it checks for the presence of the word 'positive' or 'negative' in the output sentence. Explain why this rule would fail for the output sentence: 'The review is not at all positive.'

Identifying Limitations of Simple Keyword Mapping

A developer is building a system to categorize customer support tickets into one of three classes: `URGENT`, `ROUTINE`, or `INFO_REQUEST`. The language model they are using produces a full sentence as its output, such as 'This ticket appears to be an urgent matter.' Which of the following is the most direct, rule-based approach to map this sentence to the correct class label?

Learn Before

Related