Inference & performance · Concept

Model routing

Choose a model based on task needs, constraints, policy, or observed difficulty.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition