Evidence-Based LLM Development

Building LLM applications that can be measured: model APIs, retrieval components, and the evaluation and observability layer that turns prompt tinkering into something closer to engineering.

13 tools · updated Nov 2, 2026 · compare the first four

Tools in this collection

1. 1
   CP

   Claude Platform

   Anthropic · Model providers & APIs

   In-depth entry

   Anthropic's API and developer tooling.

   ProprietaryUsage-basedEnterpriseNov 2, 2026
2. 2
   OA

   OpenAI API

   OpenAI · Model providers & APIs

   In-depth entry

   OpenAI's API and developer platform.

   ProprietaryUsage-basedEnterpriseNov 2, 2026
3. 3
   O

   OpenRouter

   OpenRouter, Inc. · Model providers & APIs

   Many providers behind one API, useful for model comparisons.

   ProprietaryFreemiumUsage-basedBroken
4. 4
   ST

   Sentence Transformers

   Hugging Face · Embeddings, reranking & retrieval

   Embedding and reranking models you can run yourself.

   Apache-2.0Open sourceFreeLocalSelf-hostedNov 2, 2026
5. 5
   VA

   Voyage AI

   MongoDB (Voyage AI) · Embeddings, reranking & retrieval

   Hosted embeddings and rerankers.

   ProprietaryFreemiumUsage-basedNov 2, 2026
6. 6
   P

   pgvector

   Andrew Kane (pgvector project) · Vector databases

   In-depth entry

   Vector similarity inside PostgreSQL, often enough instead of a separate store.

   PostgreSQLOpen sourceFreeSelf-hostedNov 2, 2026
7. 7
   Q

   Qdrant

   Qdrant Solutions GmbH · Vector databases

   In-depth entry

   Dedicated vector database when filtering and scale demand it.

   Apache-2.0Open sourceFreemiumLocalSelf-hostedNov 2, 2026
8. 8
   R

   Ragas

   Vibrant Labs (formerly Exploding Gradients) · Prompting, evaluation & observability

   Metrics for retrieval-augmented generation pipelines.

   Apache-2.0Open sourceFreeLocalBroken
9. 9
   P

   promptfoo

   Promptfoo, Inc. · Prompting, evaluation & observability

   Declarative test suites for prompts, models and agents.

   MITOpen sourceFreeLocalSelf-hostedBroken
10. 10
    L

    Langfuse

    Langfuse (part of ClickHouse, Inc. since January 2026) · Prompting, evaluation & observability

    In-depth entry

    Tracing, datasets and evaluations, self-hostable.

    MITOpen sourceFreemiumSelf-hostedNov 2, 2026
11. 11
    B

    Braintrust

    Braintrust Data, Inc. · Prompting, evaluation & observability

    Hosted evaluation and logging platform.

    MixedFreemiumPaidNov 2, 2026
12. 12
    D

    DSPy

    Stanford NLP and the DSPy community · Prompting, evaluation & observability

    Programmatic prompt optimization against a metric.

    MITOpen sourceFreeLocalSelf-hostedNov 2, 2026
13. 13
    I

    Instructor

    Jason Liu and contributors (567 Labs) · Agent frameworks & orchestration

    Structured, validated outputs from any model.

    MITOpen sourceFreeLocalNov 2, 2026