ai-agent-bench

Compare Claude Code and Codex on the same real code-change task with isolated worktrees, identical gates, transcripts, time, and cost.

执行命令读文件写文件联网严重 1 · 高危 18hashgraph-online/awesome-codex-plugins

它会碰到什么

扫了多少7 个文本文件，136 KB

它会碰到什么执行命令读文件写文件联网

命中总数29 处

命中统计严重 1 · 高 18 · 中 9 · 低 1

逐条看命中（19 条严重或高危）

严重 scripts/parse_transcript.py:1303instruction-harmful-additive
```
parser.add_argument(
```
高 scripts/run_evals.py:327exec-spawn
```
tracked = subprocess.run(
```
高 scripts/run_evals.py:334exec-spawn
```
untracked = subprocess.run(
```
高 scripts/run_evals.py:456exec-spawn
```
process = subprocess.run(
```
高 scripts/run_evals.py:500exec-spawn
```
process = subprocess.Popen(
```
高 scripts/run_evals.py:655exec-spawn
```
tracked = subprocess.run(
```
高 scripts/run_evals.py:662exec-spawn
```
untracked = subprocess.run(
```

高 scripts/run_evals.py:895exec-spawn

subprocess.run(["git", "init", "-q"], cwd=path, check=True)

高 scripts/run_evals.py:900exec-spawn

subprocess.run(["git", "add", "."], cwd=path, check=True)

高 scripts/run_evals.py:901exec-spawn
```
subprocess.run(
```
高 scripts/run_trial.py:42exec-spawn
```
p = subprocess.run(
```

高 scripts/run_trial.py:43exec-shell-true

cmd, cwd=str(cwd) if cwd else None, shell=shell, capture_output=True, text=True

高 scripts/run_trial.py:144exec-spawn

p = subprocess.run(cmd, shell=True, cwd=str(cwd), stdout=log, stderr=log)

高 scripts/run_trial.py:144exec-shell-true

p = subprocess.run(cmd, shell=True, cwd=str(cwd), stdout=log, stderr=log)

高 scripts/run_trial.py:187exec-spawn

p = subprocess.run(cmd, cwd=str(worktree), stdout=out, stderr=err)

高 scripts/run_trial.py:191exec-spawn

def spawn_monitor(run_dir: Path) -> subprocess.Popen | None:

高 scripts/run_trial.py:196exec-spawn

return subprocess.Popen([sys.executable, str(monitor), "--run-dir", str(run_dir)])

高 scripts/run_trial.py:199exec-spawn

def kill_monitor(proc: subprocess.Popen | None) -> None:

高 scripts/run_trial.py:583exec-spawn
```
p = subprocess.run(
```

这一栏是扫描器报的事实，不是结论。命中多不等于有毒（安全工具、规则库、示例脚本本来就会包含危险写法），命中少也不等于干净。它和你手上的凭据、文件、网络有什么关系，需要你自己看。

技能内容

AI agent bench

Compare agents only with the same task, starting commit, and outcome check. The harness preserves result branches and removes temporary worktrees.

Create <repo>/.agent-bench.toml:

prompt = "prompts/task.md"
start_branch = "main"              # or start_commit
agents = ["claude", "codex"]
outer_check = "./scripts/full_check.sh"
inner_check = "pytest tests/integration/test_x.py -q"

outer_check proves the real outcome before and after, and measures wall time. inner_check gives agents fast feedback.

Require a clean repo, available CLIs, and a passing outer_check. Confirm agents and run ID, then run trials sequentially to avoid load-biased timing:

python <skill>/scripts/run_trial.py --repo "$REPO" --config "$REPO/.agent-bench.toml" --agent "$AGENT" --run "$RUN_ID"

Results go to eval-results/<task>/<agent>/run-<id>-<timestamp>/. Record unexpected behavior in ai-agent-bench-anomalies.md per [anomalies](references/anomalies.md).

Aggregate with scripts/parse_transcript.py --aggregate <run-dirs> --output comparison.json --render-report comparison.md. Report gates, branches, time delta, tokens, and cost. Never rank a failed trial.

For plugin behavior rather than a real code task, use the bounded Pydantic runner documented by eval-regression and scripts/run_evals.py.

Never commit on the user's branch. A repeated run creates a new timestamped result and preserves prior evidence.

想直接用这个技能？

本站把开放许可（MIT / Apache 等）的技能按仓库打包整理到网盘，点一下转存到你自己的网盘，不用一个个从 GitHub 拉。许可未声明的技能只给原始仓库链接，不打包。

去原始仓库

它属于哪个仓库

仓库hashgraph-online/awesome-codex-plugins

星标★ 1,027

本站分层T1

该仓技能数1910

原文件路径plugins/reidemeister94/development-skills/skills/ai-agent-bench/SKILL.md

同一个仓库里的其他技能

看这个仓库的全部 1910 个技能