Directory

Python web framework for building HTTP APIs from type-annotated functions, with automatic validation through Pydantic and generated OpenAPI documentation

Hosting platform for open models, datasets and Spaces demo apps built on git repositories and Xet storage, with Inference Providers, Endpoints, Jobs and an open-source Python client and CLI

Open-source Python SDK and self-hosted AI gateway that calls 100+ LLM providers in the OpenAI format, with routing, fallbacks, virtual keys, budgets, spend tracking and logging callbacks

C/C++ inference engine for LLMs and vision-language models with GGUF quantized weights, Metal, CUDA, Vulkan and other backends, a CLI and an OpenAI-compatible server

High-throughput LLM inference and serving engine with PagedAttention, continuous batching, quantization and an OpenAI-compatible server for GPU and accelerator clusters