vLLM project (originated at UC Berkeley Sky Computing Lab) · Local LLM inference & serving
In-depth entry
High-throughput LLM inference and serving engine with PagedAttention, continuous batching, quantization and an OpenAI-compatible server for GPU and accelerator clusters
ML experiment tracking, sweeps, artifacts and model registry plus Weave for LLM tracing and evaluation, now sold by CoreWeave within its Forge platform with a free academic tier
OpenAI's open-source multilingual speech recognition model family with MIT-licensed code and weights, a Python package and CLI for transcription, translation and language identification