Promptfoo
EvaluationMIT
Promptfoo runs test cases against prompts, agents and RAG pipelines, compares model outputs and scans for security problems through red teaming. It is a command line tool and library configured with declarative files.
The README gives installation through npm, Homebrew or pip, and running any command with npx without installing. It states that Promptfoo is now part of OpenAI and remains open source under the MIT license.

Pinout
- MCP server: supported, verified
- HTTP API: not checked yet
- command line: supported, verified
- Python SDK: not checked yet
- TypeScript SDK: supported, verified
- OpenAPI description: not checked yet
| Interface | Status | Source we read | Read on |
|---|---|---|---|
| MCP server | Supported, verified | Promptfoo MCP Server | |
| HTTP API | Not checked yet | — | — |
| Command line | Supported, verified | README, "promptfoo is a CLI and library" | |
| Python SDK | Not checked yet | — | — |
| TypeScript SDK | Supported, verified | npm package promptfoo | |
| OpenAPI description | Not checked yet | — | — |
Facts
| Pricing | Community plan free, with red teaming capped at 10,000 probes a month; Enterprise and On-Premise priced on request. promptfoo pricing page, read . |
|---|---|
| License | MIT. GitHub API, repository license field, read . |
| Main language | TypeScript (as detected by GitHub). GitHub API, repository record, read . |
| Publisher | promptfoo (GitHub organization). GitHub API, repository record, read . |
| Latest GitHub release | 0.125.1, published 2026-10-25. GitHub API, latest release, read . |
| npm package | npm install promptfoo (version 0.125.1). npm registry, latest version record, read . |
| Install options | npm install -g promptfoo, brew install promptfoo, pip install promptfoo, or npx promptfoo@latest. README in the repository, read . |
| Company | Part of OpenAI, per the README. README in the repository, read . |
| Platforms | Mac, Linux (install with npm, npx or Homebrew). Installation docs; pricing page (deployment options), read . |
| Last commit | . GitHub API, latest commit on the default branch of promptfoo/promptfoo, read . |
Call example
npx promptfoo@latest mcp --transport stdioLimits
- The README notes the change of ownership to OpenAI, so the company behind the tool is not the one that started it.
Strengths
- The same command line tool covers evaluation and red teaming, according to the README.
Pricing
As written on Promptfoo's own page on the read date. Prices change: check there before you pay.
Alternatives to Promptfoo
All evaluation tools for agents
Observability and evaluation platform for tracing and improving AI agents.

Open-core platform to trace, test and evaluate agents and coding assistants.

Framework for evaluating, testing and monitoring ML and LLM systems.

Open-source library for testing and scanning LLM agents, in Python.
Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check
Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.