add harness framework

2026-08-28 00:34:51 +09:00
parent cec7b9bd48
commit 0adb9c26a9
36 changed files with 3500 additions and 0 deletions
@@ -0,0 +1,87 @@
+# AGENTS.md Quality Criteria
+
+## Assessment Order
+
+1. Read each selected instruction file completely.
+2. Map its scope and place in the active chain.
+3. Cross-reference every factual claim with repository evidence.
+4. Score all six criteria independently.
+5. Add the scores, assign a grade, and list evidence-backed issues.
+
+## Scoring Rubric
+
+### Commands and Workflows - 20 points
+
+- **20:** Essential setup, build, test, lint, and handoff commands are current,
+  scoped, and copy-paste ready.
+- **15:** Most essential commands are accurate; minor context is missing.
+- **10:** Only basic commands are present or important workflow context is
+  absent.
+- **5:** Several commands are missing or stale.
+- **0:** No usable commands are documented.
+
+### Repository Map and Instruction Scope - 20 points
+
+- **20:** Important directories and relationships are accurate; root, nested,
+  and override rules are placed at the narrowest correct scope.
+- **15:** The map and scope are useful with minor omissions.
+- **10:** A directory list exists but relationships or scope are unclear.
+- **5:** Paths or precedence are substantially misleading.
+- **0:** No usable map exists or the file governs the wrong content.
+
+### Non-Obvious Constraints and Gotchas - 15 points
+
+- **15:** Recurring quirks, prerequisites, ordering constraints, and reliable
+  workarounds are concise and evidenced.
+- **10:** Some important non-obvious constraints are documented.
+- **5:** Only minimal or weakly supported constraints are present.
+- **0:** Important recurring gotchas are absent.
+
+### Conciseness and Duplication - 15 points
+
+- **15:** Every line earns prompt space; no filler, obvious facts, or duplicate
+  rules exist across the active chain.
+- **10:** Mostly concise with limited repetition.
+- **5:** Noticeable verbosity, duplication, or misplaced detail.
+- **0:** Mostly filler or repeated content.
+
+Also report the combined selected-file byte count. The default
+`project_doc_max_bytes` is 32 KiB; use the configured value when repository or
+user config proves a different limit.
+
+### Currency Against the Codebase - 15 points
+
+- **15:** Commands, paths, versions, architecture, and tool names match current
+  repository evidence.
+- **10:** Mostly current with minor staleness.
+- **5:** Several claims are outdated or unverified.
+- **0:** Guidance is severely stale or misleading.
+
+### Actionability and Verifiability - 15 points
+
+- **15:** Instructions are concrete, safely verifiable, and include clear
+  completion expectations.
+- **10:** Mostly actionable with small ambiguities.
+- **5:** Several vague or untestable rules remain.
+- **0:** Guidance is theoretical or unusable.
+
+## Grades
+
+| Score | Grade | Meaning |
+| ---: | :---: | --- |
+| 90-100 | A | Comprehensive, current, concise, and actionable |
+| 70-89 | B | Good guidance with minor gaps |
+| 50-69 | C | Useful basics with important omissions |
+| 30-49 | D | Sparse, stale, conflicting, or poorly scoped |
+| 0-29 | F | Missing or severely misleading |
+
+## Red Flags
+
+- A command missing from the relevant manifest or task runner.
+- A reference to a deleted or moved path.
+- Root and nested rules that conflict without an intentional override.
+- The same rule repeated across the active chain.
+- A broad file containing rules true only for one subtree.
+- An override used when additive nested guidance was intended.
+- Combined guidance near or beyond the configured byte limit.
+- Templates copied without real paths, commands, or repository details.
@@ -0,0 +1,69 @@
+# AGENTS.md Patterns
+
+Use only sections supported by repository evidence. Prefer a short accurate
+file over a comprehensive-looking template.
+
+## Repository Root
+
+```markdown
+# Repository guidance
+
+## Commands
+
+- `npm ci` - Install the locked dependency set.
+- `npm run test:unit` - Run unit tests.
+- `npm run lint` - Check formatting and static rules.
+
+## Architecture
+
+- `src/server.ts` - HTTP entry point.
+- `packages/payments/` - Payment domain package.
+
+## Verification
+
+- Run the smallest relevant test during iteration.
+- Before handoff, run `npm run lint` and `npm run test:unit`.
+```
+
+## Nested Additive Guidance
+
+Use nested `AGENTS.md` when local rules add to broader guidance.
+
+```markdown
+# Payments package guidance
+
+## Commands
+
+- `npm run test:payments` - Run payment-domain tests from the repository root.
+
+## Gotchas
+
+- Payment fixtures use UTC timestamps; do not compare local-time strings.
+```
+
+## Nested Override
+
+Use `AGENTS.override.md` only when same-directory standard guidance must be
+replaced. Broader parent guidance still remains earlier in the chain.
+
+```markdown
+# Generated client override
+
+## Editing
+
+- Do not hand-edit files under `src/generated/`.
+- Regenerate them with `npm run generate:client`.
+
+## Verification
+
+- Run `npm run check:generated`.
+```
+
+## Scope Rules
+
+- Put team-wide repository rules at the root.
+- Put subtree-only commands and constraints in the closest nested directory.
+- Put personal cross-repository preferences in Codex-home guidance only when
+  the user explicitly requests global scope.
+- Avoid overrides for ordinary additive instructions.
+
@@ -0,0 +1,62 @@
+# AGENTS.md Update Guidelines
+
+## Evidence Contract
+
+Every proposed line must be:
+
+- specific to the repository or explicitly requested global scope;
+- supported by a file, manifest, safe command result, or user-confirmed fact;
+- durable enough to help future sessions;
+- placed at the narrowest scope where it is true;
+- expressed in the shortest actionable form that preserves meaning.
+
+Never add secrets, credential values, personal data, unverified claims,
+session narrative, generic advice, facts obvious from names, or one-off fixes.
+
+## Safe Verification
+
+Prefer read-only inspection. Run a command only when it is relevant, safe, and
+unlikely to change external state. Do not deploy, publish, migrate data, write
+credentials, or invoke destructive cleanup merely to validate guidance.
+
+If a command cannot safely be run, label it unverified rather than treating it
+as current.
+
+## Precedence and Size
+
+- Select `AGENTS.override.md` before `AGENTS.md` at each directory.
+- Build the active project chain from root to current working directory.
+- Later, closer guidance wins on conflict.
+- Parent guidance remains active unless a later instruction contradicts it.
+- Flag duplicate or conflicting rules across selected files.
+- Compare combined selected-file bytes with configured
+  `project_doc_max_bytes`, using 32 KiB when no override is evidenced.
+
+## Diff Contract
+
+For every proposed change:
+
+```markdown
+### Update: path/to/AGENTS.md
+
+**Why:** `package.json` defines `test:unit`, while the documented `npm test`
+command does not exist.
+
+```diff
+- Run `npm test` before handoff.
+ Run `npm run test:unit` before handoff.
+```
+```
+
+Group diffs by destination. After all diffs, ask which changes the user wants
+applied. Do not edit before that approval.
+
+## Post-Edit Validation
+
+- Re-read every changed file.
+- Rebuild the active chain for the affected subtree.
+- Check for conflicts and duplication.
+- Verify changed commands and paths.
+- Recalculate the combined byte count.
+- Report exactly what was changed and what could not be verified.
+