Evidently

EvaluationApache-2.0

Evidently evaluates, tests and monitors machine learning and LLM systems, from tabular data to generative AI outputs. It is a Python framework from Evidently AI that runs locally or on Evidently Cloud.

The README shows installation from PyPI, a locally hosted Evidently UI that opens with demo projects, a self-hosted open-source version with a live demo, and the hosted Evidently Cloud with extra features such as alerting and dataset management.

Pinout

  • MCP server: not checked yet
  • HTTP API: not checked yet
  • command line: not checked yet
  • Python SDK: supported, verified
  • TypeScript SDK: not checked yet
  • OpenAPI description: not checked yet
Where each pin comes from
InterfaceStatusSource we readRead on
MCP serverNot checked yet——
HTTP APINot checked yet——
Command lineNot checked yet——
Python SDKSupported, verifiedREADME, install (pip install evidently)
TypeScript SDKNot checked yet——
OpenAPI descriptionNot checked yet——

Facts

Facts about Evidently
LicenseApache-2.0. GitHub API, repository license field, read .
Main languageJupyter Notebook (as detected by GitHub). GitHub API, repository record, read .
Publisherevidentlyai (GitHub organization). GitHub API, repository record, read .
Latest GitHub releasev0.7.23, published 2026-09-11. GitHub API, latest release, read .
Self-hostingOpen-source version with a local UI; live demo at demo.evidentlyai.com. README in the repository, read .
Hosted optionEvidently Cloud. README in the repository, read .
Last commit. GitHub API, latest commit on the default branch of evidentlyai/evidently, read .

Limits

  • Alerting, user management and no-code evals are listed in the README as features of Evidently Cloud rather than of the open-source version.

Strengths

  • The open-source version includes a local web interface, started after a pip install, per the README.

Alternatives to Evidently

All evaluation tools for agents
  1. Open-core platform to trace, test and evaluate agents and coding assistants.

    Evaluation
  2. Observability and evaluation platform for tracing and improving AI agents.

    Evaluation
  3. Python framework for testing LLM outputs and agents with metrics.

    Evaluation
  4. Open-source library for testing and scanning LLM agents, in Python.

    Evaluation

Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check

Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.