Ragas
EvaluationApache-2.0
Ragas evaluates LLM applications with model-based and traditional metrics and can generate test datasets from existing data. It is a Python package installed from PyPI or from source.
The README gives installation from PyPI or from source and a ragas quickstart command that creates example evaluation projects from templates, such as a RAG evaluation project. The repository record now sits under the vibrantlabsai organization on GitHub.

Pinout
- MCP server: not checked yet
- HTTP API: not checked yet
- command line: supported, verified
- Python SDK: not checked yet
- TypeScript SDK: not checked yet
- OpenAPI description: not checked yet
| Interface | Status | Source we read | Read on |
|---|---|---|---|
| MCP server | Not checked yet | — | — |
| HTTP API | Not checked yet | — | — |
| Command line | Supported, verified | README, section "Quickstart" (ragas quickstart) | |
| Python SDK | Not checked yet | — | — |
| TypeScript SDK | Not checked yet | — | — |
| OpenAPI description | Not checked yet | — | — |
Facts
| License | Apache-2.0. GitHub API, repository license field, read . |
|---|---|
| Main language | Python (as detected by GitHub). GitHub API, repository record, read . |
| Publisher | vibrantlabsai (GitHub organization). GitHub API, repository record, read . |
| Latest GitHub release | v0.4.4, published 2026-11-14. GitHub API, latest release, read . |
| Quickstart | ragas quickstart lists templates; ragas quickstart rag_eval creates a RAG evaluation project. README in the repository, read . |
| Last commit | . GitHub API, latest commit on the default branch of vibrantlabsai/ragas, read . |
Limits
- The repository moved from the explodinggradients organization to vibrantlabsai, so older links use the former owner name.
Strengths
- A quickstart command scaffolds an evaluation project from a template, per the README.
Alternatives to Ragas
All evaluation tools for agents
Readymade evaluators that score an agent's trajectory of steps.

Readymade evaluators for LLM applications in Python and TypeScript.

Open-core platform to trace, test and evaluate agents and coding assistants.

Framework for evaluating, testing and monitoring ML and LLM systems.
Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check
Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.