vLLM

A model inference and serving engine focused on efficient execution and throughput.

The distinction that matters

Product capabilities and compatibility change. Check the linked project documentation before adopting or upgrading.

Reference definition