Promptfoo

EvaluationMIT

Promptfoo runs test cases against prompts, agents and RAG pipelines, compares model outputs and scans for security problems through red teaming. It is a command line tool and library configured with declarative files.

The README gives installation through npm, Homebrew or pip, and running any command with npx without installing. It states that Promptfoo is now part of OpenAI and remains open source under the MIT license.

Pinout

  • MCP server: supported, verified
  • HTTP API: not checked yet
  • command line: supported, verified
  • Python SDK: not checked yet
  • TypeScript SDK: supported, verified
  • OpenAPI description: not checked yet
Where each pin comes from
InterfaceStatusSource we readRead on
MCP serverSupported, verifiedPromptfoo MCP Server
HTTP APINot checked yet——
Command lineSupported, verifiedREADME, "promptfoo is a CLI and library"
Python SDKNot checked yet——
TypeScript SDKSupported, verifiednpm package promptfoo
OpenAPI descriptionNot checked yet——

Facts

Facts about Promptfoo
PricingCommunity plan free, with red teaming capped at 10,000 probes a month; Enterprise and On-Premise priced on request. promptfoo pricing page, read .
LicenseMIT. GitHub API, repository license field, read .
Main languageTypeScript (as detected by GitHub). GitHub API, repository record, read .
Publisherpromptfoo (GitHub organization). GitHub API, repository record, read .
Latest GitHub release0.125.1, published 2026-10-25. GitHub API, latest release, read .
npm packagenpm install promptfoo (version 0.125.1). npm registry, latest version record, read .
Install optionsnpm install -g promptfoo, brew install promptfoo, pip install promptfoo, or npx promptfoo@latest. README in the repository, read .
CompanyPart of OpenAI, per the README. README in the repository, read .
PlatformsMac, Linux (install with npm, npx or Homebrew). Installation docs; pricing page (deployment options), read .
Last commit. GitHub API, latest commit on the default branch of promptfoo/promptfoo, read .

Call example

npx promptfoo@latest mcp --transport stdio

Limits

  • The README notes the change of ownership to OpenAI, so the company behind the tool is not the one that started it.

Strengths

  • The same command line tool covers evaluation and red teaming, according to the README.

Pricing

PricingCommunity plan free, with red teaming capped at 10,000 probes a month; Enterprise and On-Premise priced on requestpromptfoo pricing page, read 16 Nov 2026
npm packagenpm install promptfoo (version 0.125.1)Read 15 Nov 2026

As written on Promptfoo's own page on the read date. Prices change: check there before you pay.

Alternatives to Promptfoo

All evaluation tools for agents
  1. Observability and evaluation platform for tracing and improving AI agents.

    Evaluation
  2. Open-core platform to trace, test and evaluate agents and coding assistants.

    Evaluation
  3. Framework for evaluating, testing and monitoring ML and LLM systems.

    Evaluation
  4. Open-source library for testing and scanning LLM agents, in Python.

    Evaluation

Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check

Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.