Directory

Amazon's public cloud with hundreds of usage-billed services spanning compute (EC2, Lambda), storage (S3), databases (RDS), networking, AI (Bedrock) and analytics

Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX

Google's public cloud with Compute Engine, GKE, Cloud Run, BigQuery, Cloud Storage and the Vertex AI platform for Gemini and other models

Low-latency hosted inference API for open-weight models on Groq's LPU hardware, OpenAI-compatible, with speech-to-text, text-to-speech, tool use and batch tiers

Hosting platform for open models, datasets and Spaces demo apps built on git repositories and Xet storage, with Inference Providers, Endpoints, Jobs and an open-source Python client and CLI

Open-source Python SDK and self-hosted AI gateway that calls 100+ LLM providers in the OpenAI format, with routing, fallbacks, virtual keys, budgets, spend tracking and logging callbacks

Microsoft's public cloud with virtual machines, AKS, Azure Functions, Cosmos DB, Azure Database for PostgreSQL, Entra ID identity and Microsoft Foundry for hosted AI models

Open-source Python framework for distributing tasks, actors and ML workloads across a cluster, with libraries for data processing, training, tuning, serving and reinforcement learning

Cloud platform for open-source models: serverless and dedicated inference, fine-tuning, GPU clusters, batch jobs and code sandboxes behind an OpenAI-compatible API