Inspect

EvaluationMIT

Inspect is a Python framework for evaluating large language models, with components for prompts, tool use, multi-turn dialog and model-graded scoring. It is published by the UK AI Security Institute.

The README links a set of ready-made evaluations that can be run against any model, a documentation index published as llms.txt for coding agents, and a development setup based on uv with a checked-in lockfile.

Pinout

  • MCP server: not checked yet
  • HTTP API: not checked yet
  • command line: supported, verified
  • Python SDK: not checked yet
  • TypeScript SDK: not checked yet
  • OpenAPI description: not checked yet
Where each pin comes from
InterfaceStatusSource we readRead on
MCP serverNot checked yet——
HTTP APINot checked yet——
Command lineSupported, verifiedDocumentation home page, Hello, Inspect (inspect eval)
Python SDKNot checked yet——
TypeScript SDKNot checked yet——
OpenAPI descriptionNot checked yet——

Facts

Facts about Inspect
LicenseMIT. GitHub API, repository license field, read .
Main languagePython (as detected by GitHub). GitHub API, repository record, read .
PublisherUKGovernmentBEIS (GitHub organization). GitHub API, repository record, read .
Latest GitHub release0.3.290, published 2026-11-14. GitHub API, latest release, read .
Documentation for agentshttps://inspect.aisi.org.uk/llms.txt. README in the repository, read .
PlatformsDocker sandboxes, Kubernetes sandboxes, Modal sandboxes. Documentation home page, sandboxes, read .
Last commit. GitHub API, latest commit on the default branch of UKGovernmentBEIS/inspect_ai, read .

Call example

inspect eval simpleqa.py --model openai/gpt-6-luna

Limits

  • Working on the web interface needs a separate TypeScript setup in a git submodule, per the README.

Strengths

  • The documentation is also published as plain text for coding agents.

Alternatives to Inspect

All evaluation tools for agents
  1. Open-core platform to trace, test and evaluate agents and coding assistants.

    Evaluation
  2. Framework for evaluating, testing and monitoring ML and LLM systems.

    Evaluation
  3. Open-source library for testing and scanning LLM agents, in Python.

    Evaluation
  4. Command line tool and library to evaluate and red-team LLM apps and agents.

    Evaluation

Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check

Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.