From 0e785154d4b11eb5f7ba276edd1ed9bdafdd742a Mon Sep 17 00:00:00 2001 From: "KOKO\\Mimi" Date: Wed, 12 Aug 2026 03:14:26 +0900 Subject: [PATCH] docs: design independent reference validation policy --- ...independent-reference-validation-policy.md | 129 +++++++++++++ ...dent-reference-validation-policy-design.md | 178 ++++++++++++++++++ 2 files changed, 307 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-12-independent-reference-validation-policy.md create mode 100644 docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md diff --git a/docs/superpowers/plans/2026-08-12-independent-reference-validation-policy.md b/docs/superpowers/plans/2026-08-12-independent-reference-validation-policy.md new file mode 100644 index 0000000..2635cc1 --- /dev/null +++ b/docs/superpowers/plans/2026-08-12-independent-reference-validation-policy.md @@ -0,0 +1,129 @@ +# Independent Reference Validation Policy Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Align FESA project policy and the MITC4 upstream contracts with independent black-box Abaqus reference validation and prepare the implementation-planning agent to create approved Harness Steps. + +**Architecture:** Keep FESA HDF5 as authoritative solver output and retain deterministic source-node/component comparison, while reducing the reference phase to a lightweight inventory of existing `.inp` and required CSV paths. Remove metadata, naming, portfolio, drilling-calibration, and geometry/director-calibration gates from MITC4 without changing reference artifacts or implementing C++. + +**Tech Stack:** Markdown, TOML agent prompts, project-local Codex `SKILL.md`, PowerShell static validation, Git. + +## Global Constraints + +- Do not modify files under `reference/`. +- Do not run Abaqus or another reference solver. +- Do not modify C++ source, tests, or CMake. +- Preserve HDF5 as authoritative FESA output. +- Use the B33 mixed tolerance exactly for MITC4 U; U blocks and UR warns only. +- Retain fixed numerical drilling stabilization but exclude calibration and drilling evidence outputs. +- Remove `NR-O03` and `NR-O04` from MITC4 implementation readiness. +- Implementation Planning may create Harness phase files only after user approval of the Step draft. + +--- + +### Task 1: Align project-wide policy documents + +**Files:** +- Modify: `AGENTS.md` +- Modify: `docs/PRD.md` +- Modify: `docs/ADR.md` +- Modify: `docs/ARCHITECTURE.md` +- Modify: `docs/SOLVER_AGENT_DESIGN.md` +- Modify: `docs/SOLVER_SKILL_DESIGN.md` + +**Interfaces:** +- Consumes: approved design in `docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md` +- Produces: project-wide independent-solver and minimal-reference policy used by every downstream agent + +- [ ] Replace canonical/legacy/provenance bundle governance with the minimal reference-case contract. +- [ ] Separate Abaqus input syntax and external comparison from FESA formulation behavior. +- [ ] Record the new decision as a superseding ADR while preserving historical decisions. +- [ ] State the MITC4 B33 tolerance reuse and fixed drilling scope without claiming implementation exists. +- [ ] Verify changed policy terms with targeted `rg` checks and `git diff --check`. +- [ ] Commit with `docs: simplify FESA reference validation policy`. + +### Task 2: Align agent and skill workflow contracts + +**Files:** +- Modify: affected `.codex/agents/*.toml` +- Modify: affected `.codex/skills/*/SKILL.md` +- Modify: affected workflow README templates under `docs/` + +**Interfaces:** +- Consumes: Task 1 project policy +- Produces: specialist prompts that no longer recreate removed artifact gates + +- [ ] Update Requirement, Numerical Review, I/O, Reference Model, Reference Verification, Coordinator, Implementation Planning, Physics, and Release boundaries. +- [ ] Require Implementation Planning Agent to invoke `$harness`, produce a self-contained multi-Step draft, request user approval, and only then write `phases/` files. +- [ ] Keep actual Harness execution outside the planning agent unless explicitly requested. +- [ ] Validate TOML syntax, skill YAML frontmatter, required sections, and prohibited stale terms. +- [ ] Forward-test the revised skills with an independent agent prompt. +- [ ] Commit with `docs: align FESA agents with minimal reference cases`. + +### Task 3: Revise MITC4 requirements and formulation + +**Files:** +- Modify: `docs/requirements/linear-static-mitc4-shell.md` +- Modify: `docs/research/linear-static-mitc4-shell-research.md` +- Modify: `docs/formulations/mitc4-shell-formulation.md` + +**Interfaces:** +- Consumes: Tasks 1-2 policy and approved fixed drilling rule +- Produces: implementation-ready feature scope without removed calibration gates + +- [ ] Correct the observed S4/S4R reference inventory and adopt the B33 U/UR mixed tolerance. +- [ ] Replace drilling candidate/sweep language with the fixed positive rotational-diagonal rule. +- [ ] Remove drilling calibration, drilling-energy warning, `NR-O03`, `NR-O04`, and expanded portfolio requirements. +- [ ] Preserve the future nonlinear derivation as documentation-only. +- [ ] Add a research decision note without rewriting source findings as if they changed. +- [ ] Verify requirement IDs and formulation traceability contain no contradictory blocker. +- [ ] Commit with `docs: simplify MITC4 verification and drilling scope`. + +### Task 4: Revise MITC4 I/O and reference-case inventory + +**Files:** +- Modify: `docs/io-definitions/linear-static-mitc4-shell-io.md` +- Create: `docs/reference-models/linear-static-mitc4-shell-reference-models.md` + +**Interfaces:** +- Consumes: Task 3 requirements/formulation +- Produces: exact lightweight S4/S4R input-to-displacement-CSV pairing and HDF5 comparison mapping + +- [ ] Remove canonical/legacy/metadata/provenance/schema-version readiness requirements. +- [ ] Remove mandatory drilling stiffness, ratio, and energy output evidence. +- [ ] Keep minimal source-node plus U/UR component matching and finite/unique-row checks. +- [ ] Inventory the exact existing `reference/shell` and `reference/shellR` paths read-only. +- [ ] Verify no file under `reference/` changed. +- [ ] Commit with `docs: define lightweight MITC4 reference cases`. + +### Task 5: Rerun MITC4 numerical review under the approved policy + +**Files:** +- Modify: `docs/numerical-reviews/linear-static-mitc4-shell-review.md` + +**Interfaces:** +- Consumes: Tasks 1-4 current source-of-truth documents +- Produces: a fresh formulation-only implementation-planning verdict + +- [ ] Reclassify removed artifact, calibration, `NR-O03`, and `NR-O04` blockers explicitly. +- [ ] Review the fixed rotational-diagonal drilling rule for dimensions, symmetry, positivity, and separation from physical recovery. +- [ ] Confirm U/UR tolerance and current S4/S4R reference pairing are downstream comparison inputs, not formulation blockers. +- [ ] Issue the verdict supported by the remaining confirmed formulation findings only. +- [ ] Run Markdown structure, targeted policy consistency, `git diff --check`, and no-reference-mutation checks. +- [ ] Commit with `docs: rerun MITC4 numerical review under minimal reference policy`. + +### Task 6: Final repository policy verification + +**Files:** +- Verify: all files changed by Tasks 1-5 + +**Interfaces:** +- Consumes: all preceding tasks +- Produces: clean, internally consistent documentation and agent/skill configuration evidence + +- [ ] Run `uv run --with pytest python -m pytest -v -rs` because project-local agent/skill workflow documents changed. +- [ ] Parse every changed TOML file and validate every changed skill frontmatter. +- [ ] Run repository-wide targeted searches for stale mandatory provenance, canonical, README, drilling sweep, `NR-O03`, and `NR-O04` language in active contracts. +- [ ] Run `git diff --check` and inspect `git status --short`. +- [ ] Confirm `git diff -- reference` is empty. +- [ ] Report any historical documents intentionally left unchanged. diff --git a/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md b/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md new file mode 100644 index 0000000..f3f8426 --- /dev/null +++ b/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md @@ -0,0 +1,178 @@ +# Independent Reference Validation Policy Design + +## Status + +- date: `2026-08-12` +- status: `approved` +- scope: FESA project policy, agents, skills, and `linear-static-mitc4-shell` upstream contracts +- implementation_code: out of scope +- reference_artifact_mutation: prohibited + +## Decision + +FESA is an independent finite element solver. It accepts an approved subset of the +Abaqus `.inp` syntax, but it does not reproduce or claim equivalence with Abaqus +element formulations, integration rules, stabilization, internal state, or result +generation procedures. Abaqus CSV values are external numerical reference values. +Exact equality is allowed but is neither required nor evidence of identical internal +behavior. + +A feature passes reference verification when FESA solves the declared input and its +required comparison quantities satisfy the feature-approved tolerance. Reference +artifact governance that does not participate in that comparison must not block +implementation planning or verification. + +## Minimal Reference Case Contract + +A reference case requires only: + +- the declared `.inp` file used by both solvers; +- the declared Abaqus CSV file for each blocking or warning-only quantity; +- the FESA `results.h5` produced from that `.inp`; +- a deterministic mapping of the compared source identity and components; +- the approved tolerance. + +The following are not readiness requirements: + +- canonical directory or filename conventions; +- legacy-alias approval; +- bundle `README.md`; +- `metadata.json`; +- Abaqus version or generation provenance; +- duplicated unit, coordinate, model, step, final-frame, material, thickness, or + element-type metadata; +- a reference CSV schema version. + +Reference paths and filenames are consumed as declared without rename or repair. +Reference artifacts remain read-only. A comparator must still reject ambiguous or +unsafe comparisons: missing files, missing required source IDs/components, duplicate +IDs, unmatched required rows, and nonfinite values. This is an executable matching +contract, not a metadata or bundle-governance contract. + +For a feature with one supported static step and one final frame, the input/CSV pair +implicitly identifies that result state. Model properties and source element types are +read from the `.inp` rather than copied into another metadata file. + +## Reference Model Phase + +The Reference Model phase remains in the workflow as a lightweight reference-case +inventory. It records: + +- the purpose of each existing case; +- exact input and required CSV paths; +- blocking and warning-only quantities; +- FESA HDF5 projection and source-ID/component matching; +- the approved tolerance; +- whether the required files are present and readable. + +It does not design a broad benchmark portfolio unless the feature requirements +explicitly request one. It does not gate on naming, README, metadata, provenance, or +duplicated descriptive fields. + +## MITC4 Reference Acceptance + +Both source element types map to the same internal `FESA-MITC4` formulation: + +- `reference/shell/shell.inp` and `reference/shell/shell displacements.csv` cover + source `S4`; +- `reference/shellR/shellR.inp` and + `reference/shellR/shellR displacements.csv` cover source `S4R`. + +Only matched global `U1/U2/U3` rows are blocking. `UR1/UR2/UR3` rows are compared and +reported as warning-only evidence. Reactions and stresses may be inspected but are not +reference pass/fail quantities. + +For each case and component `c`: + +```text +reference_scale_c = max_i(abs(abaqus_value_i,c)) +tolerance_c = 1e-9 + 1e-6 * reference_scale_c +``` + +The `1e-9` absolute floor is expressed in the model's user-consistent length unit for +translation and as a dimensionless rotation value for warning-only UR. Values are not +zero-clamped. Missing, duplicate, unmatched, or nonfinite required rows fail before +numeric tolerance evaluation. U tolerance failure fails reference verification; UR +tolerance exceedance emits a deterministic warning only. + +No additional locking, convergence, distorted, curved, director-angle, or invalid +geometry reference portfolio is required for this MITC4 implementation acceptance. +The previously open `NR-O03` and `NR-O04` calibration/inspection requirements are +removed from the feature gate. + +## MITC4 Drilling Boundary + +Drilling calibration is excluded, but a fixed numerical stabilization remains required +to make the external six-DOF shell system solvable. The feature does not implement a +physical drilling strain or accept director-parallel nodal moments. + +Use one element-level scalar computed from the positive diagonal entries of the +physical rotational stiffness block: + +```text +k_d = 1e-3 * min(K_phys[i,i] | i is a physical rotational DOF and K_phys[i,i] > 0) +K_drill_local = k_d * I4 +K_drill_global = T_d^T * K_drill_local * T_d +``` + +Restricting the minimum to rotational entries preserves the `force*length` dimension. +The literal minimum over mixed translational and rotational diagonals is not approved. +The stabilization must be finite, symmetric, positive, deterministic, and excluded +from physical generalized strain, resultant, and stress recovery. + +The following are out of scope: + +- reference-family selection and `rho_d` normalization; +- coefficient sweeps, stable plateaus, rank/conditioning calibration, and adjacent + decade sensitivity; +- drilling-to-physical energy thresholds or warning policies; +- mandatory drilling stiffness, ratio, or energy result datasets. + +Basic unit tests may verify construction, symmetry, positivity, deterministic behavior, +and removal of the four nonphysical drilling null modes. Current reference cases contain +no drilling-direction load; their blocking safeguard against unacceptable physical +response contamination is the approved U comparison. + +## Numerical Review Boundary + +Numerical Review decides whether the documented formulation is internally complete and +consistent enough to plan implementation. It must not fail because a later Reference +Model document, canonical bundle, README, metadata, provenance, extended benchmark +portfolio, or comparison result is absent. + +For MITC4: + +- fixed drilling stabilization closes `NR-O01` and removes `NR-O02` by scope; +- `NR-O03` and `NR-O04` are removed from the approved verification scope; +- the B33 mixed tolerance closes the blocking part of `NR-O05`; +- missing heavyweight reference metadata is `not-required-by-policy`; +- confirmed formulation defects remain blocking. + +## Agent and Skill Workflow + +Requirement, I/O, Reference Model, Reference Verification, Coordinator, Implementation +Planning, Physics Evaluation, and Release instructions must use the minimal reference +case contract. Numerical Review must remain independent of later artifact governance. + +Implementation Planning must use the project-local `harness` skill when the user asks +for an implementation plan. It must: + +1. prepare a multi-Step draft with one layer or module per Step; +2. make each Step self-contained for an independent Codex execution; +3. include prerequisite documents, TDD RED/GREEN/VERIFY work, exact MSVC/CMake/CTest + acceptance commands, and specific prohibitions; +4. request user approval for the Step draft; +5. only after approval create or update `phases/index.json`, + `phases//index.json`, and `phases//stepN.md`; +6. leave Harness execution to an explicit subsequent user request. + +## Scope Preservation + +- Do not modify files under `reference/`. +- Do not run Abaqus or another reference solver. +- Do not add C++ MITC4 implementation in this policy-alignment change. +- Keep the future geometric-nonlinear residual/tangent derivation documentation, but do + not authorize nonlinear product execution. +- Historical research evidence may retain its original cautions. Current requirements, + formulation decisions, I/O contracts, and the latest numerical review must clearly + identify the approved policy that supersedes earlier recommendations.