Learn Before
IO-Aware Self-Attention Implementations
An example of a hardware-aware optimization for Transformers is the use of IO-aware implementations of the self-attention function. This technique is particularly effective on modern GPUs for improving the model's efficiency.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Optimizing Model Inference on GPUs
A development team is deploying a large Transformer model on a new, custom-designed hardware accelerator. They observe that the model's inference speed is significantly slower than expected. Profiling reveals that the primary bottleneck is not the raw computational speed of the accelerator, but the time spent moving data between different levels of its unique memory hierarchy. Which of the following strategies represents a hardware-aware optimization approach to directly address this specific da
Differentiating Optimization Strategies