Documentation
Everything for running local, artifact-based agent evals with evalctl — grouped into guides, concepts, and project notes.
Start here
| If you want to… | Read |
|---|---|
| Install evalctl | install |
| Run your first suite | quickstart |
| The command surface | commands |
| Drive it from an agent | agent-guide |
| Decode an error code | errors |
| How it compares | comparisons |
Buckets
Guides
Install, run your first suite, the command surface, and the agent workflow.
Concepts
How durable runs, replay, command scorers, and the spoolctl queue work.
Project
How evalctl compares, what changed, and the security posture.
- How evalctl compares
- Changelog
- Tools for evaluating AI agents, compared
- evalctl and Harbor
- evalctl and DeepEval
- evalctl and OpenAI Evals
- evalctl and Inspect
- evalctl and pytest
- evalctl and LLM-as-judge
- evalctl and GitHub Actions
- evalctl and LangSmith
- evalctl and Braintrust
- evalctl and Patronus
- evalctl and Langfuse
- evalctl and Arize Phoenix
- evalctl and W&B Weave
- evalctl and TruLens
- evalctl and SWE-bench
- evalctl and lm-evaluation-harness
- evalctl and Harbor ATIF
- evalctl and Letta Trajectory
- evalctl and Guardrails (Guardrails AI, NeMo Guardrails)
- Security posture