Evidently
EvaluationApache-2.0
Evidently evaluates, tests and monitors machine learning and LLM systems, from tabular data to generative AI outputs. It is a Python framework from Evidently AI that runs locally or on Evidently Cloud.
The README shows installation from PyPI, a locally hosted Evidently UI that opens with demo projects, a self-hosted open-source version with a live demo, and the hosted Evidently Cloud with extra features such as alerting and dataset management.

Pinout
- MCP server: not checked yet
- HTTP API: not checked yet
- command line: not checked yet
- Python SDK: supported, verified
- TypeScript SDK: not checked yet
- OpenAPI description: not checked yet
| Interface | Status | Source we read | Read on |
|---|---|---|---|
| MCP server | Not checked yet | — | — |
| HTTP API | Not checked yet | — | — |
| Command line | Not checked yet | — | — |
| Python SDK | Supported, verified | README, install (pip install evidently) | |
| TypeScript SDK | Not checked yet | — | — |
| OpenAPI description | Not checked yet | — | — |
Facts
| License | Apache-2.0. GitHub API, repository license field, read . |
|---|---|
| Main language | Jupyter Notebook (as detected by GitHub). GitHub API, repository record, read . |
| Publisher | evidentlyai (GitHub organization). GitHub API, repository record, read . |
| Latest GitHub release | v0.7.23, published 2026-09-11. GitHub API, latest release, read . |
| Self-hosting | Open-source version with a local UI; live demo at demo.evidentlyai.com. README in the repository, read . |
| Hosted option | Evidently Cloud. README in the repository, read . |
| Last commit | . GitHub API, latest commit on the default branch of evidentlyai/evidently, read . |
Limits
- Alerting, user management and no-code evals are listed in the README as features of Evidently Cloud rather than of the open-source version.
Strengths
- The open-source version includes a local web interface, started after a pip install, per the README.
Alternatives to Evidently
All evaluation tools for agents
Open-core platform to trace, test and evaluate agents and coding assistants.

Observability and evaluation platform for tracing and improving AI agents.

Python framework for testing LLM outputs and agents with metrics.

Open-source library for testing and scanning LLM agents, in Python.
Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check
Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.