promptfoo
Pilot it+854 this week★ 24,628 · 2,239 forks
AI model testing and evaluation framework
Worth a timeboxed spike before you bet on it.
Who it's for
small AI and ML teams
What it replaces
manual model testing and evaluation
The catch
steep learning curve and dependency on maintainers
Your first hour
Clone the repo and run example configs
The numbers
Stars24,628
Forks2,239
Stars added (7d)+854
Open issues536
LanguageTypeScript
LicenceMIT
Last pushUpdated today
Project age3 years old
Maintainers describe it as: “Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.”
cici-cdcicdevaluationevaluation-frameworkllmllm-evalllm-evaluationllm-evaluation-frameworkllmops
promptfoo, in short
- Should a small team use promptfoo?
- Pilot it. Worth a timeboxed spike before you bet on it. small AI and ML teams
- What does promptfoo actually do?
- AI model testing and evaluation framework
- What does promptfoo replace?
- manual model testing and evaluation
- What is the downside of promptfoo?
- steep learning curve and dependency on maintainers
- Can promptfoo be used in a commercial product?
- Its licence is MIT, which is permissive and generally fine for commercial use. Confirm against the LICENSE file in the repository.