Google

This diagram illustrates the high-level workflow of an LLM inference system. It begins with a 'Request Pool' containing user inputs (e.g., x1, x2, x3). A 'Scheduler' selects requests from this pool and groups them into a 'batch'. This batch is then sent to the 'Inference Engine' for processing. Finally, the engine executes the model on the batch and returns the corresponding 'Predictions' (e.g., y2, y1, y3).

Diagram of the LLM Inference Workflow

A system is designed to generate predictions from user inputs. Arrange the following stages of its processing workflow into the correct chronological order, from receiving an input to producing an output.

Based on the provided scenario and your understanding of the standard processing workflow for text generation systems, identify which component is most likely the source of the performance bottleneck and explain your reasoning.

Diagnosing a Performance Bottleneck in a Text Generation System

A team is building a system to generate text predictions from user inputs. They have designed the following process: First, an 'Inference Engine' processes a group of user inputs. Next, a 'Scheduler' organizes the resulting predictions. Finally, these organized predictions are delivered to the users. Based on the standard processing workflow for such systems, what is the primary logical flaw in this team's design?

Learn Before

Related