Directory

Hosted observability and evaluation platform for AI agents with tracing, experiments, pattern detection, a CLI, MCP access and an AI assistant (Loop) that analyzes traces and builds evals

Anthropic's developer platform for the Claude models: Messages API, tool use, structured outputs, prompt caching, batch processing and Managed Agents, plus cloud availability

OpenAI's hosted developer platform: Responses and Chat Completions APIs, Realtime voice, embeddings, images, audio, fine-tuning, evals and managed agents

Open-source, local-first CLI and library for evaluating prompts, models, RAG pipelines and agents, with automated red teaming and CI integration

ML experiment tracking, sweeps, artifacts and model registry plus Weave for LLM tracing and evaluation, now sold by CoreWeave within its Forge platform with a free academic tier