1Cademy - Under the $q^*$ execution policy, what behavior does the system dynamically degenerate to as the host bandwidth approaches PCIe bandwidth ($B_H \to B

Learn Before

Optimal MoE Decode Cache-Fill Count ( $q^*$ Policy)

Short Answer

Under the $q^*$ execution policy, what behavior does the system dynamically degenerate to as the host bandwidth approaches PCIe bandwidth ( $B_H \to B_P$ )?

Updated 2026-09-07

Contributors are:

AI Agent

🏆 1

Who are from:

University of Michigan - Ann Arbor

🏆 1

Tags

Prep Sessions

Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor

Ch.3 Adaptive Runtime Policies and Device Execution - Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor

Bandwidth-Adaptive Decode and the q* Execution Policy - Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor

A system encounters $m = 12$ missing experts during decode, with PCIe bandwidth $B_P = 32\text{ GB/s}$ and host bandwidth $B_H = 128\text{ GB/s}$ . Based on the optimal execution policy, how many missing experts should be routed to the concurrent PCIe cache-fill branch?
In the runtime implementation of the $q^*$ execution policy, a minimum of at least one cache fill is retained to ensure continuous cache warming.
Explain the mathematical derivation used to determine the optimal expert cache-fill count $q^*$ for serving $m$ missing experts of size $S$ . Include the expressions for both concurrent branches, how they are equated, and the resulting closed-form policy.

Learn Before

Related