Prefilling-Decoding Frameworks

Search (Decoding) Algorithms for LLM Inference

Evaluation Metrics for LLM Inference Performance

Methods for Improving LLM Inference Efficiency

Purpose of Defining Notation for LLM Inference

Interdisciplinary Nature of Efficient LLM Inference

Inference-Time Scaling

A technology company is deploying a large language model for a customer service chatbot. They face two distinct challenges: 1) The time and computational power required to generate a response for each user is too high, leading to slow reply times and expensive server costs. 2) The generated responses, while fluent, are often too generic and repetitive. Which two distinct areas of inference study are most relevant for solving challenge #1 and challenge #2, respectively?

Optimizing an LLM for a Code Generation Application