Inspect
EvaluationMIT
Inspect is a Python framework for evaluating large language models, with components for prompts, tool use, multi-turn dialog and model-graded scoring. It is published by the UK AI Security Institute.
The README links a set of ready-made evaluations that can be run against any model, a documentation index published as llms.txt for coding agents, and a development setup based on uv with a checked-in lockfile.

Pinout
- MCP server: not checked yet
- HTTP API: not checked yet
- command line: supported, verified
- Python SDK: not checked yet
- TypeScript SDK: not checked yet
- OpenAPI description: not checked yet
| Interface | Status | Source we read | Read on |
|---|---|---|---|
| MCP server | Not checked yet | — | — |
| HTTP API | Not checked yet | — | — |
| Command line | Supported, verified | Documentation home page, Hello, Inspect (inspect eval) | |
| Python SDK | Not checked yet | — | — |
| TypeScript SDK | Not checked yet | — | — |
| OpenAPI description | Not checked yet | — | — |
Facts
| License | MIT. GitHub API, repository license field, read . |
|---|---|
| Main language | Python (as detected by GitHub). GitHub API, repository record, read . |
| Publisher | UKGovernmentBEIS (GitHub organization). GitHub API, repository record, read . |
| Latest GitHub release | 0.3.290, published 2026-11-14. GitHub API, latest release, read . |
| Documentation for agents | https://inspect.aisi.org.uk/llms.txt. README in the repository, read . |
| Platforms | Docker sandboxes, Kubernetes sandboxes, Modal sandboxes. Documentation home page, sandboxes, read . |
| Last commit | . GitHub API, latest commit on the default branch of UKGovernmentBEIS/inspect_ai, read . |
Call example
inspect eval simpleqa.py --model openai/gpt-6-lunaLimits
- Working on the web interface needs a separate TypeScript setup in a git submodule, per the README.
Strengths
- The documentation is also published as plain text for coding agents.
Alternatives to Inspect
All evaluation tools for agents
Open-core platform to trace, test and evaluate agents and coding assistants.

Framework for evaluating, testing and monitoring ML and LLM systems.

Open-source library for testing and scanning LLM agents, in Python.

Command line tool and library to evaluate and red-team LLM apps and agents.
Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check
Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.