Directory

Python framework for programming language-model pipelines with typed signatures and modules, then optimizing prompts and weights automatically against a metric

Open-source LLM observability and evaluation platform: tracing, LLM-as-judge and human evals, prompt management, datasets and cost dashboards, self-hostable or as Langfuse Cloud

Open-source, local-first CLI and library for evaluating prompts, models, RAG pipelines and agents, with automated red teaming and CI integration

Type-safe Python agent framework from the Pydantic team with validated structured outputs, dependency injection, swappable model providers, MCP, durable execution and evals

Open-source Python library for evaluating LLM and RAG applications with LLM-driven and custom metrics, experiments, dataset management and result tracking