Inference & performance · Concept

CPU/GPU offloading

Move model state or computation between CPU and GPU resources.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition