Google

The output $$\mathbf{o}$$ from a general sequence model, which is generated by a neural network $$g(\cdot; \theta)$$, can differ based on the specific problem being addressed. For token prediction problems (such as language modeling), the output $$\mathbf{o}$$ is typically a probability distribution over a defined vocabulary. Conversely, for sequence encoding problems, the output $$\mathbf{o}$$ serves as a representation of the input sequence, commonly expressed as a sequence of real-valued vectors.

Output Variation in Sequence Models

Consider two different applications for a sequence model. Task A is to autocomplete a user's sentence by predicting the most likely next word. Task B is to classify an entire email as either 'spam' or 'not spam'. Compare and contrast the expected structure of the final output from the sequence model for Task A versus Task B. Explain the reasoning behind these structural differences based on the goal of each task.

Analysis of Sequence Model Outputs for Different Tasks

A data scientist is developing two different systems. Match each system's primary task with the most likely structure of its underlying model's direct output.

A data scientist is working with a sequence model, `g(·; θ)`, for two different projects. In Project A, the goal is to predict the most likely next word in a user's search query. In Project B, the goal is to classify an entire product review as either 'positive' or 'negative'. How would the nature of the model's direct output, `o`, most likely differ between these two projects?

Learn Before

Related