roxy

Roxy decides whether an agent's work is done. The agent proposes, and Roxy answers from checks it ran itself.

A project lists its claims in roxy.toml. Each claim has one check per platform or entry point it must hold on. Roxy records a check as evidence only after three runs:

1. The check passes on the code as it is now.
2. The check fails when Roxy applies a patch that breaks the code.
3. The check passes again once Roxy has restored the code.

A check with no evidence shows as unverified, so a platform nobody checked is visible in the report.

Install

Roxy needs git and sh on the path, and Rust 1.89 or newer to build.

cargo install --git https://github.com/SkyliveLLC/roxy

Claims file

[claims.image-upload]
description = "A signed-in user can upload an image"

[claims.image-upload.checks.web]
run = "bun test apps/web/upload.e2e.ts"
covers = ["apps/web/src/upload/**", "apps/server/src/upload/**"]
mutation = "mutations/upload-skip-content-type.patch"

[claims.image-upload.checks.ios]
run = "scripts/check-upload-ios.sh"
covers = ["apps/mobile/src/upload/**", "apps/server/src/upload/**"]
mutation = "mutations/upload-reject-heic.patch"

• run is a shell command, run with sh -c. Exit code 0 means the claim holds. A simulator run, a browser run, a property test, a Kani proof and a TLA+ model check all fit. The command must be repeatable: it gives the same result each time it runs on the same code, and it does not edit the files it covers.
• covers lists the files the check proves something about, as globs. * stays inside one directory and ** crosses directories. Only regular files count. Files that git ignores, symlinks and roxy.lock never do. A glob that matches none of those files is an error.
• mutation is a patch in git apply format that breaks the covered code. It may only edit the contents of covered files. Roxy rejects a patch that creates, deletes or renames a file or changes a file mode. A patch that git diff writes for an edited text file qualifies, as long as git uses its default a/ and b/ prefixes. For a binary file, use git diff --binary.

roxy.toml must sit at the top level of a git repository, and all paths are relative to it. Every claim needs at least one check.

Commands

Command	What it does	Exit code
roxy status	Prints every check as verified, stale or unverified.	0
roxy verify	Runs the checks that lack fresh evidence and writes roxy.lock.	1 if Roxy rejected any check
roxy verify --all	Reruns every check and ignores roxy.lock.	1 if Roxy rejected any check
roxy gate	Prints the same report as status.	1 unless every check is verified

Every command exits with code 2 when Roxy itself cannot run. Examples are a roxy.toml that does not parse, a glob that matches no file, and a second Roxy command started in the same repository while one is running. roxy verify also exits with code 2 when git cannot read a mutation patch.

How evidence goes stale

roxy.lock stores one fingerprint per check. The fingerprint is a SHA-256 hash of the run command, the mutation patch, and the path and contents of every covered file. Editing any of them changes the fingerprint, and the check shows as stale until roxy verify runs it again. Commit roxy.lock with the code.

File modes are not part of the fingerprint, so removing the executable bit from a covered script keeps the evidence fresh. The fingerprint also only sees covered files. If a check depends on a file that covers leaves out, a change to that file keeps the evidence fresh. roxy verify --all reruns the check anyway.

How the mutation runs

Roxy copies the files the patch changes into the git directory, applies the patch to the working tree, runs the check, and copies the files back. It runs in place so the check sees the same dependencies and build output as the clean run. The third run, on the restored code, rebuilds whatever the mutated run compiled or installed. Roxy skips the third run when the check passes with the mutation applied, so build output from the broken code can remain after that rejection.

Roxy restores from its copies and never reverses the patch, so the files come back byte for byte even if the check overwrote them. An edit made to those files while the check runs, by you or by another agent, is discarded when Roxy copies them back.

If Roxy is killed during the mutated run, the working tree holds the broken code until the next Roxy command. That command copies the saved files back before it does anything else, which also discards any edit made to those files in between. After an interrupted roxy verify, run roxy status before you touch the code.

Use with an agent

Add this to the project's AGENTS.md:

Before you report work as done, run `roxy gate`. If it fails, run `roxy verify` and fix what it rejects.
If `roxy verify` is interrupted or times out, run `roxy status` before anything else. It restores the files the run had changed.
Never edit `roxy.lock` by hand. If you change a check or its mutation in `roxy.toml`, say so in your report.

Use in CI

Run roxy verify --all. An agent could write a matching fingerprint into roxy.lock without running anything, so CI must not trust the lockfile from a pull request. roxy gate is the fast local answer, and roxy verify --all on a machine the agent does not control is the one to trust.

What Roxy does not do

• It does not judge whether a check or a mutation is a good one. A check can be written to detect the mutation run and fail on purpose, and a pull request can weaken a claim, remove a check or swap in an easier mutation. A reviewer has to read the diff of roxy.toml and the mutation patches.
• It does not write checks or mutations. A person or an agent writes them.
• It has no timeout. Wrap a slow check in timeout 10m yourself.
• It runs checks one at a time, and only one Roxy command per repository at a time.

Develop

mise install
cargo test
cargo run -- status

Roxy checks itself. Its own roxy.toml has one claim, backed by the test suite and a mutation that disables the rule that a check must fail on broken code.

License

MIT