修复流水线Repair Pipeline
可回滚的沙箱修复 AgentA rollback-safe repair agent
项目计时Project clock
能专注时再开始。计时与截止保存在本机浏览器;离开时记得暂停。Start when you can focus. The clock and deadline persist in this browser; pause whenever you leave.
项目简报Brief
一个中型仓库的 CI 连续失败。你要让 Agent 在隔离工作区里定位失败、改代码、跑测试,并由独立 verifier 决定是否接受;任何越界写入、危险命令或回归都必须被拒绝,失败时回滚到基线。
A mid-sized repository has repeated CI failures. The agent must locate failures in an isolated workspace, patch code, run tests, and let an independent verifier decide whether to accept. Out-of-scope writes, dangerous commands and regressions must be rejected, with rollback to baseline on failure.
可重复的修复流水线:基线快照、修复循环、权限审计、回滚演练、5 个种子 bug 的评测报告,以及一份能复现的失败分类。
A reproducible repair pipeline: baseline snapshot, repair loop, permission audit, rollback rehearsal, a five-seed-bug evaluation, and a reproducible failure taxonomy.
产物契约Artifact contract
先冻结产物形状,再写 Agent。后续里程碑与评测都读取这份契约。Freeze the shape before writing the agent. Every later milestone and evaluation reads this contract.
{
"bug_id": "B-03",
"baseline_tests": "12 passed, 2 failed",
"attempts": [
{"patch": "normalize_slug", "tests": "14 passed", "verifier": "accepted"}
],
"files_changed": ["src/slugify.py"],
"rollback": "git restore --source baseline",
"status": "accepted"
}
里程碑Milestones
每条验收成立后再勾选里程碑。小时数是预算,不是秒表目标。Check each milestone only when its acceptance holds. Hours are a budget, not a stopwatch target.
-
复制仓库到隔离目录;先跑基线测试并解析失败;记录文件哈希,作为回滚锚点。
Copy the repo into an isolated directory; run baseline tests and parse failures; record file hashes as the rollback anchor.
验收Acceptance基线命令可复现;失败测试列表稳定;工作区外没有任何写入。
The baseline command reproduces; the failing list is stable; nothing outside the workspace is written.
-
plan → edit → test → verify;最多三次修复尝试;verifier 独立判断,失败反馈进入下一轮。
Plan → edit → test → verify; at most three repair attempts; the verifier judges independently and failures feed the next round.
验收Acceptance5 个种子 bug 至少修复 4 个;没有无限循环;每次尝试都有测试证据。
At least four of five seeded bugs are fixed; no infinite loop; every attempt carries test evidence.
-
限制写入在工作区内;屏蔽危险 shell;敏感文件 deny;每次阻止都记录规则与原因。
Confine writes to the workspace; block dangerous shell; deny sensitive files; record the rule and reason for every block.
验收Acceptance越界写入、
rm -rf、读取.env的尝试全部被拒绝并有日志。Out-of-scope writes, rm -rf and .env reads are all blocked and logged.
-
失败修复必须恢复基线并重跑全套测试;成功修复也不能引入回归。
A failed repair must restore baseline and rerun the full suite; a successful repair must not add regressions.
验收Acceptance回滚后文件哈希与基线一致;回归测试全绿;无残留进程/临时文件。
File hashes match baseline after rollback; regression tests are green; no leftover processes or temp files.
-
跑完 5 个种子 bug,统计修复率、误修复率与平均尝试次数;补齐 README。
Run all five seeded bugs; measure fix rate, false-fix rate and average attempts; finish the README.
验收Acceptance修复率 ≥ 80%,误修复 0,回滚 100%;README 可复现。
Fix rate ≥ 80%, zero false fixes, 100% rollback; the README reproduces.
交付物Deliverables
每项交付物必须真实存在且可复现。完成门槛会检查全部勾选,再要求复盘。Every deliverable must exist on disk and be reproducible. The finish gate checks all boxes, then asks for a reflection.
-
针对种子 bug 的最小补丁;包含改动文件与最终测试状态。
A minimal patch for the seeded bug, with changed files and final test status.
-
基线与修复后的测试命令、通过/失败清单、耗时;两次运行一致。
Baseline and post-fix commands, pass/fail lists and timing; two runs agree.
-
每次工具调用、权限裁决、验证器结论;被拒绝的越界尝试也要留下记录。
Every tool call, permission decision and verifier verdict; blocked out-of-scope attempts must be recorded too.
-
快照位置、恢复命令、如何验证工作区与基线一致。
Snapshot location, restore command and how to prove the workspace matches baseline.
-
5 个种子 bug:至少修复 4 个,误修复为 0,回滚全部成功。
Five seeded bugs: at least four fixed, zero false fixes, all rollbacks successful.
-
环境要求、运行命令、权限模型、失败分类与已知限制。
Environment requirements, commands, permission model, failure taxonomy and known limits.
评分表Rubric
宣告完成前诚实自评。低于 80 分意味着该迭代,而不是该交付。Grade yourself honestly before calling it done. Anything below 80 is a signal to iterate, not to ship.
| 评分项Criterion | 分值Points | 达标标准What good looks like |
|---|---|---|
| 修复正确性Correctness | 30 | 补丁最小、测试通过、无回归;修复理由能对上失败根因。 Minimal patch, tests passing, no regressions; the fix rationale matches the root cause. |
| 权限与安全Permissions and safety | 25 | 工作区边界、危险命令、敏感文件三层防护;阻止有审计。 Workspace boundary, dangerous commands and sensitive files are all guarded; blocks are audited. |
| 循环设计Loop design | 20 | 尝试上限、失败反馈、验证器独立;不靠提示词祈祷停止。 Attempt cap, failure feedback, independent verifier; never rely on prompt wording to stop. |
| 回滚与回归Rollback and regression | 15 | 失败可恢复、哈希可验证、无残留副作用。 Failures are recoverable, hashes verify, and no side effects remain. |
| 评测与文档Evaluation and docs | 10 | 5 个 bug 的结果可复算;README 说清限制。 Results for five bugs are recomputable; the README states limits. |
| 合计Total | 100 |
双轨实现Build on your track
交付物不绑定语言。按团队需要选择轨道;完成记录会保存你交付所用的轨道。The deliverable is language-agnostic. Follow the track that fits the team; the finish record stores the track you shipped on.
用 Read/Write/Bash 绑定工作区,用 PermissionEngine 挡住越界与敏感文件;验证器和修复器分开,失败必须回滚。
Bind Read/Write/Bash to the workspace and use PermissionEngine to block escapes and secrets; separate verifier from fixer, and roll back on failure.
工作区工具Workspace tools
Bash(cwd=root)、Read、Write 显式绑定根目录;不要依赖进程 cwd。
Bind Bash(cwd=root), Read and Write to the root explicitly; never rely on process cwd.
权限矩阵Permission matrix
DEFAULT 先询问;EXPLORE 拒绝写入;deny rule 精确挡住 **/.env 与危险命令,且优先级高于 BYPASS。
Start from DEFAULT; EXPLORE denies writes; deny rules block **/.env and dangerous commands and outrank BYPASS.
回滚与验证Rollback and verify
每轮开始前记录哈希/快照;失败后恢复并重跑基线;ToolResultEndEvent.state == error 必须视为失败。
Record hashes/snapshots before each attempt; restore and rerun baseline on failure; treat ToolResultEndEvent.state == error as failure.
toolkit = Toolkit(tools=[Read(), Write(), Bash(cwd=root)])
guarded = PermissionEngine(PermissionContext(
mode=PermissionMode.BYPASS, # sandbox only
deny_rules={
"Write": [PermissionRule(tool_name="Write", rule_content="**/.env",
behavior=PermissionBehavior.DENY, source="lab")],
"Bash": [PermissionRule(tool_name="Bash", rule_content="*rm -rf*",
behavior=PermissionBehavior.DENY, source="lab")],
},
))
blocked = await guarded.check_permission(Write(), {"file_path": str(root / ".env")})
assert blocked.behavior == PermissionBehavior.DENY
agentscope.tool.Bashagentscope.tool.Readagentscope.tool.Writeagentscope.tool.Toolkitagentscope.permission.PermissionEngineagentscope.event.ToolResultEndEventBYPass 模式被当成安全边界:它只是跳过询问,deny rule 才真正挡住敏感文件;生产必须从 DEFAULT 起步。
Treating BYPASS as a security boundary: it only skips prompts; deny rules do the blocking. Start production from DEFAULT.
用 ReadFileTool/WriteFileTool/ShellCommandTool + 根目录与 CommandValidator;verifier 与 fixer 分离,状态与回滚自己掌控。
Use ReadFileTool/WriteFileTool/ShellCommandTool with a root and CommandValidator; separate verifier and fixer, and own state and rollback.
内置工具Built-in tools
三个内置工具都需要根目录;shell 还要 CommandValidator,Windows 用 WindowsCommandValidator。
All three built-ins take a root; the shell tool also needs a CommandValidator, or WindowsCommandValidator on Windows.
权限Permissions
PermissionMode 不是边界;再加显式拒绝规则,并审计每次阻止。
PermissionMode is not the boundary; add explicit deny rules and audit every block.
回滚Rollback
每轮前快照工作区;失败后用文件哈希验证恢复;结构化 verdict 驱动状态机。
Snapshot the workspace before each attempt; verify restoration by hash; drive the state machine with a structured verdict.
Toolkit toolkit = Toolkit.builder()
.tool(new ReadFileTool(root.toString()))
.tool(new WriteFileTool(root.toString()))
.tool(new ShellCommandTool(root.toString(), new WindowsCommandValidator()))
.build();
Verdict verdict = verifier
.call(List.of(Msg.user(reviewPrompt(attempt))), Verdict.class, ctx)
.block();
if (!verdict.passed()) restoreSnapshot(root); // hash-check afterwards
io.agentscope.core.tool.file.ReadFileToolio.agentscope.core.tool.file.WriteFileToolio.agentscope.core.tool.shell.ShellCommandToolio.agentscope.core.permission.PermissionModeio.agentscope.core.ReActAgent只跑一次测试就宣布修好:隐藏的回归会在评审或下一轮暴露。成功与失败都要跑全套回归,并用哈希证明回滚。
Declaring victory after one test run: hidden regressions surface later. Run the full suite for success and failure, and prove rollback with hashes.
进阶与资源Stretch goals and resources
加入“故意失败”的 verifier 场景:验证器反复拒绝时,系统在预算耗尽后安全回滚。
Add an always-reject verifier drill: the system safely rolls back after the retry budget is exhausted.
支持多 bug 排队:一个仓库连续修复 5 个问题,每个补丁独立提交与回滚。
Queue multiple bugs: repair five issues in one repo with independently committable and reversible patches.