Inference & performance · Concept

Expert parallelism

Distribute mixture-of-experts components across devices.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition