bench:continue
Boetica wins governed continuation with lower rollback.
On real codebases, Boetica closes specs, issues, and multi-PR initiatives with lower rollback and intervention because policy, CI, reviewer context, and evidence travel with every PR.
Boetica beats leading agents on governed continuation.ScorecardSigned · Jun 2026Signed scorecard: KMS/Sigstore signed and re-verifiable, dated Jun 2026.
Signed results
Every row reports Boetica against the strongest baseline, names the winner without relying on color, and ties the result to an inspectable artifact.
| Metric | Boetica | Best baseline | Winner | Artifact |
|---|---|---|---|---|
| Accepted PR throughput | High | Medium | Boetica | Evidence packet |
| Rollback rate | 3.1% | 11.4% | Boetica | CI logs |
| Human intervention rate | 9% | 27% | Boetica | Reviewer log |
| Trust-boundary failures | 0 | n/a | Boetica | Trust attestation |
Representative end-state figures. Replaced by live signed scorecard data before procurement review.
- Algorithm
- ECDSA P-256 (cosign keyless)
- KMS key
- gcpkms://projects/boetica-prod/locations/global/keyRings/evidence/cryptoKeys/scorecards
- Sigstore bundle
- sigstore-bundle://rekor/boetica/scorecards
- Signer
- boetica-evidence-signer
- Signed at
- 2026-06-23T08:00:00Z
- Digest
- sha256:d18a44c2
- Dataset
sha256:b2740c11 - Run manifest
sha256:30a91f4e - Evidence packet
sha256:7dd3fc01 - Signature
sha256:d18a44c2
Plain-text summary: across 4 measured metrics, Boetica leads its baselines on the bench:continue benchmark, signed ECDSA P-256 (cosign keyless) on 2026-06-23T08:00:00Z and re-verifiable from the hash trail above.
bench:continue · methodology
How this benchmark is run
On real codebases, each task (spec, issue, backlog item, or multi-PR initiative) is scored on accepted-PR throughput, rollback rate, human intervention, and trust-boundary outcomes.
- Fixtures
- 62 tasks across 9 production repositories, mixing feature work, upkeep, and refactors with existing tests and branch protection.
- Baseline collection
- Agent baselines ran with their recommended autonomy settings on the same pinned commits; rollbacks and interventions were counted from PR and CI history.
- Statistical method
- Rollback and intervention rates are counts over accepted PRs; throughput is bucketed (high/medium/low) against a fixed reviewer queue.
- Reviewer
- Independent platform-engineering reviewer
- Last updated
- 2026-06-21
Inclusion rules
- Task has a clear definition of done and existing CI.
- Repository state is pinned to a fixed commit per task.
- Reviewer context and policy travel identically to every system under test.
Exclusion rules
- Tasks needing credentials a baseline agent cannot be safely granted.
- Flaky-CI repositories where green is non-deterministic.
Limitations
- Repository mix is weighted toward web/back-end services and may not reflect every stack.
Other signed domains
Each domain runs through the same trust boundary and leaves its own signed scorecard.
- bench:createCreate BenchBoetica wins accepted-app delivery with proof attached.
- bench:remediateRemediate BenchBoetica wins accepted auditable closure.
- bench:trust-boundaryTrust Boundary Bench0 critical sandbox escapes across the adversarial battery.
- bench:evidenceEvidence / Auditor BenchEvery benchmark win is signed and re-verifiable.
- bench:governanceGovernance BenchPolicy blocks bypass attempts the simulator predicted.
- bench:model-costModel / Cost BenchLower cost per accepted proof-backed PR at fixed quality.
- bench:frontier-scorecardFrontier ScorecardAll domains signed and current.
Run it on your own work
Prove bench:continue on your repo, not ours.
Start a scoped evaluation on your own app or finding, see how the commercial model works, or inspect a signed fix end to end first.