BenchKeeper
Short rules, a workflow skill that sizes each change, and Ponytail for Claude Code and Codex, installed by an app that shows each change first and can undo the run. Each repository adds its dev container, AGENTS.md and stack skills.
One loop, sized to the change
The workflow skill sorts each task into a software change, a docs pass or research, and gives a software change a tier: how much process it gets. Quick: branch, build, verify, ship. Full: spec, tickets, tests first, review. Several tasks on one branch run their long checks and reviews once at the end. A skill that matches the task loads in any tier, and each change type adds its own check.
1
Understand
Read the request, AGENTS.md and the code the change touches, and pick the kind (software change, docs pass, research) and the tier. With Matt Pocock's skills installed, his on-ramps start here: /triage for issues you did not write, diagnosing-bugs for a broken behavior, /wayfinder for work too big to see the way yet. Standard and full tiers: the agent asks the open questions in rounds, each with a recommendation, explores in plan mode, then drafts the issue (title, goal, acceptance criteria, tasks) and creates it after your approval. Requests come from you, GitHub issues, Jira, Linear and other trackers, bug reports, reviews and Maintain findings. Only GitHub issues are automated.
2
Specify
Standard tier: a plan of a few lines that names the test seams. Full tier: the spec in the issue (deployment target, data storage, module boundaries, core types), with terms and decisions recorded (domain-modeling with Matt Pocock's skills). Work for several sessions: one ticket per independently testable slice; with his skills the agent suggests /to-spec and /to-tickets and waits for you to run them.
3
Build
Every tier: a branch <type>/<issue>-<slug>, or <type>/<slug> without an issue, never the default branch unless you say so. Standard and full tiers: the existing tests that cover the touched code run first, then tests first at the agreed seams (tdd with Matt Pocock's skills), ticket by ticket. Ponytail full for the smallest complete change. Full tier: when the build drifts from the plan, the agent stops and asks before it discards the branch and restarts.
4
Verify
Every tier: the project's build, types, lint and tests, run where AGENTS.md says (the dev container by default), each reported with its command and result. UI changes also in a browser through Playwright MCP, in addition to the UI tests, never instead of them. Long checks run once, after the last edit. Full tier: the branch is reviewed against the spec (code-review with Matt Pocock's skills).
5
Ship
Standard, full and risky work: /code-review before git push (correctness), risky changes also /security-review. Draft pull request with Closes #<issue> and the key decisions, checks, ready. Merge only on your instruction. Several tasks on one branch get one review and one pull request at the end.
6
Maintain
After a session that went sideways: /retro (Matt Pocock's skills). On request (/maintaining-repos): Ponytail's audit and debt, cleanup tools (unused code, duplicates), architecture checks (cycles, forwarding modules), a test audit by mutation testing, dependency audit, version pins, docs drift, /doctor prompt-audit, /skill-doctor, repository hygiene, and with Matt Pocock's skills /improve-codebase-architecture and /triage. benchkeeper doctor reports setup drift on the machine and in the project. Findings become new requests.
What gets installed
Global, every session, full level
Secrets kept out, writing without filler, sources for claims. The repository rules (ask before assuming, checks never weakened, load the workflow skill) are in rules/project.md, part of each project's block.
Global, before repository work
The task kind and tier, the playbooks for software changes, docs passes and research, the check per change type, batch mode with long checks at the end.
Optional, his plugin, global in both agents
Grilling, spec and tickets, TDD, bug diagnosis, code review, pull request text. They read the issue tracker and domain docs in docs/agents/.
Global, level full
The smallest complete change: reuse before new code, no unrequested options, a test for new logic, shortcut: comments for known limits.
On request
npx benchkeeper@6.25.0 sets up the machine and each repository, shows every change first and records each run for undo. Its Dashboard reports drift, Fix everything applies the global plan and updates every project, and the Sessions board tracks tier, cost and outcome per session. /benchkeeper:setting-up-projects adds stack skills and the dev container, /benchkeeper:maintaining-repos runs a maintenance pass, /benchkeeper:orchestrating-projects coordinates workers on labeled issues across projects.
Per repository
Installs, builds, tests and the browser run in the container as a non-root user. Domain skills come from the BenchKeeper library, stack skills from find-skills, at most two per technology.
Three sizes
Start with the plugin and add what you need. Each level contains the one before.
Lite
the BenchKeeper plugin: the workflow, tier, setting-up-projects, maintaining-repos and orchestrating-projects skills
Writes: nothing outside the plugin install, besides the app's run record for undo.
npx benchkeeper@6.25.0 apply --level lite
Standard
lite plus the local project block: CLAUDE.local.md, AGENTS.override.md and the git exclude list
Writes: local files in each project you run project apply in.
npx benchkeeper@6.25.0 apply --level standard && npx benchkeeper@6.25.0 project apply
Full
standard plus the global rules, Ponytail, Claude Code and Codex settings, packs and the app
Writes: global rule files, settings and packs.
npx benchkeeper@6.25.0 apply --level full
Frameworks you already use
Project setup asks first: it shows the list and installs nothing you did not choose. A framework that owns the workflow is a singleton: at most one per project, and choosing it takes BenchKeeper's own workflow skill out of that project. Parts sit beside it. Everything installs inside the project; what cannot is labelled.
Superpowers
Installs a Claude Code plugin enabled for this project only; its plugin files stay in the user's plugin cache. Evaluation
Matt Pocock's skills
Installs the matt-pocock pack; its plugin installs for the user, outside the project. Evaluation
ECC
Installs three single skills from the default branch, no plugin and no hooks. Evaluation
agency-agents
Installs a pick of agent files fetched at a pinned commit. Evaluation
Quick start
Node.js 22 and at least one agent, Claude Code or Codex. Full list of tools. Choose the App to review each change in the browser. Choose the CLI in a terminal-only session or in a script. The app, view by view, with screenshots.
App
npx benchkeeper@6.25.0
Opens the browser on the Dashboard with the doctor findings. Select Fix everything, review the changes and press Apply; nothing is written before.
CLI
npx benchkeeper@6.25.0 plan
Lists each change and writes nothing.
npx benchkeeper@6.25.0 apply
Asks per change. --yes applies the default selection without questions.
After either path, restart the agents. npx benchkeeper@6.25.0 history lists what changed, npx benchkeeper@6.25.0 undo <run> reverses a run.
Each repository, updates and sessions
1
Project setup, in each repository root
npx benchkeeper@6.25.0 project apply --agents claude,codex
--agents names the agents the repository uses. Where commands run and the default tier, library packs with --pack, and .benchkeeper/state.json. It stays on this machine: CLAUDE.local.md, AGENTS.override.md and the skills, all in .git/info/exclude. --share commits it to the repository instead. project plan shows the changes first.
2
Stack skills and the dev container, as a recorded session
npx benchkeeper@6.25.0 record -- claude '/benchkeeper:setting-up-projects'
Codex: the same with codex '$benchkeeper:setting-up-projects'. With the matt-pocock pack, also /mattpocock-skills:setup-matt-pocock-skills. Commit the result.
3
Update later
npx benchkeeper@6.25.0 apply
With the version of the newest release. The plan lists what changed, including the plugin. Projects: npx benchkeeper@6.25.0 update-all, or Fix everything in the browser, updates each local setup in place; a shared one, or a setup still committed from before, gets a pull request.
4
Track sessions
npx benchkeeper@6.25.0 board
Writes a static page, board.html in the BenchKeeper config folder, from the Claude Code and Codex transcripts on this machine: per session the tier, the setup installed at the time, skills used, the last test run, cost, time and pull request state. The Sessions view in the browser app shows the same board.
Why this stack
- Quality: in BenchKeeper's mistreevous brownfield run only ECC scored measurably above plain Claude Code; Matt Pocock's skills, Ponytail and BenchKeeper stayed within the round-to-round spread (medians 84.1 to 90.1, Sonnet 5.5, k = 3).
- Time: it depends on the version: setup-v4.0 with Matt Pocock's skills took a median 13.1 minutes against 22.7 for plain Claude Code, setup-v5.0 took 24.9. setup-v6.0 is not measured yet.
- Cost: Ponytail alone cost 0.58 times plain Claude Code in the mistreevous run (k = 3); external results range from 10% less to 55% more.
- Skills by level: the full flow runs only for full-tier work. Small changes load one targeted skill.
- One install: one machine setup covers every repository. Each repository adds only its tracker, docs and stack.
For details, check the full data: evaluations, benchmark results.
Where things run
By default the agent runs inside the repository's dev container, together with the project commands. With --env host-agent it stays on the host and sends project commands through devcontainer exec.
- Rules and the plugins: BenchKeeper, Ponytail, optionally Matt Pocock's skills
- Git and gh: branches, commits, pull requests
- Dev Containers CLI and Docker or Podman
- The agent, with --env host-agent
devcontainer exec
- The agent (default), started with devcontainer exec
- Project toolchain at pinned versions
- Installs, builds, lint, tests
- Playwright browser
- Non-root user, project mount only