✓evalctl
Local-first evals for agents, not just prompts.
evalctl scores what an agent does — not just what it says. The files it writes, the diffs it produces, the commands it runs, the exit codes it returns.
Cases are files
Eval cases are filesystem fixtures. Scorers are shell commands. Nothing to configure in a dashboard.
Runs are artifacts
Every run is a sealed, rerunnable directory you can diff, commit, and hand to the next agent — no gateway, no login.
Deterministic first
Deterministic scorers ship first; LLM judges are optional, never required. You decide what counts as a pass.
Composes with the toolchain
Delegate execution to spoolctl for crash-safe async runs: bounded concurrency, retries, and resume-where-it-died across a crash. Model routing can delegate toinferctl, recorded per case.
Start with evalctl init.
Pre-release. Source and install atgithub.com/evalctl/evalctl.