Inference & performance · Concept

Tensor parallelism

Split operations within model layers across multiple devices.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition