add harness framework

2026-08-28 00:34:51 +09:00
parent cec7b9bd48
commit 0adb9c26a9
36 changed files with 3500 additions and 0 deletions
@@ -0,0 +1,106 @@
+---
+name: auditing-agents-md
+description: Use when a user asks to audit, check, review, improve, update, or fix AGENTS.md or AGENTS.override.md files, or asks whether Codex repository guidance is accurate and current.
+---
+
+# Audit AGENTS.md
+
+## Overview
+
+Audit Codex guidance against the repository that it governs. Treat instruction
+precedence, factual accuracy, scope, and conciseness as observable properties:
+report evidence before proposing a change, and propose before editing.
+
+## Workflow
+
+1. **Resolve scope.** Identify the project root, current working directory, and
+   requested audit scope. Resolve Codex home from `CODEX_HOME`, falling back to
+   `~/.codex`; inspect global guidance only when it is in scope.
+2. **Discover guidance.** Find `AGENTS.override.md` and `AGENTS.md` while
+   excluding VCS metadata, dependencies, caches, and generated output. At each
+   directory, an override is selected before the standard file. Map the active
+   chain from project root to the working directory and label files that govern
+   only other subtrees.
+3. **Inspect evidence.** Read each selected file completely. Check commands,
+   paths, architecture claims, and conventions against manifests, source,
+   tests, config, and version control. Run only safe, relevant verification;
+   do not deploy, mutate external state, or expose credentials to score a file.
+4. **Score every file.** Read
+   [quality-criteria.md](references/quality-criteria.md). Use its six weighted
+   criteria, 100-point total, and grade bands. Check conflicts, duplication,
+   misplaced scope, override behavior, and the combined 32 KiB default
+   instruction budget.
+5. **Report before changing.** Emit the output contract below even when the
+   guidance is already strong.
+6. **Propose targeted diffs.** Read
+   [update-guidelines.md](references/update-guidelines.md). Include only claims
+   supported by repository evidence. Use
+   [templates.md](references/templates.md) only when a missing structure is
+   genuinely useful.
+7. **Request approval.** Stop after the report and proposed diffs. Ask which
+   changes the user wants applied.
+8. **Apply and verify.** Edit only approved files, preserve their organization,
+   and re-check commands, paths, conflicts, active-chain order, and byte budget.
+
+## Required Report
+
+```markdown
+## AGENTS.md Quality Report
+
+### Summary
+- Files found: N
+- Active chain: path -> path
+- Average score: N/100
+- Files needing update: N
+- Combined active-chain size: N bytes (default limit: 32 KiB)
+
+### File-by-File Assessment
+#### 1. path (scope; active/inactive)
+**Score: N/100 (Grade: A-F)**
+
+| Criterion | Score | Evidence |
+| --- | ---: | --- |
+| Commands and workflows | N/20 | ... |
+| Repository map and instruction scope | N/20 | ... |
+| Non-obvious constraints and gotchas | N/15 | ... |
+| Conciseness and duplication | N/15 | ... |
+| Currency against the codebase | N/15 | ... |
+| Actionability and verifiability | N/15 | ... |
+
+**Issues**
+- ...
+
+**Recommended changes**
+- ...
+```
+
+After all assessments, show each proposed diff under `### Update: <path>` with
+a one-line `Why:`. End with an explicit approval question. Do not edit in the
+same turn unless the user already approved the exact diff.
+
+## Quick Reference
+
+| Question | Evidence |
+| --- | --- |
+| Which file applies? | Global selection, then one selected file per directory from root to working directory |
+| Is a command current? | Manifest plus a safe command run when practical |
+| Is a path accurate? | Filesystem and repository search |
+| Is guidance misplaced? | Narrowest subtree where the rule is true |
+| Is the chain too large? | Combined selected-file bytes versus configured/default limit |
+| Should content be added? | Project-specific, durable, non-obvious, and actionable |
+
+## Example
+
+If `AGENTS.md` requires `npm test` but `package.json` contains only
+`test:unit`, score currency and actionability down, cite `package.json`, and
+propose the verified command. Do not invent an aggregate script or silently
+edit the file.
+
+## Common Mistakes
+
+- Scoring the repository once instead of scoring every guidance file.
+- Treating a nested override as deleting unrelated root guidance.
+- Claiming a file is an entry point because its name looks plausible.
+- Recommending generic best practices or facts obvious from code.
+- Running destructive, deployment, or credential-sensitive commands to audit.
+- Applying recommendations before the user approves the displayed diff.