docs: design independent reference validation policy
This commit is contained in:
KOKO\Mimi
@@ -0,0 +1,129 @@
|
||||
# Independent Reference Validation Policy Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Align FESA project policy and the MITC4 upstream contracts with independent black-box Abaqus reference validation and prepare the implementation-planning agent to create approved Harness Steps.
|
||||
|
||||
**Architecture:** Keep FESA HDF5 as authoritative solver output and retain deterministic source-node/component comparison, while reducing the reference phase to a lightweight inventory of existing `.inp` and required CSV paths. Remove metadata, naming, portfolio, drilling-calibration, and geometry/director-calibration gates from MITC4 without changing reference artifacts or implementing C++.
|
||||
|
||||
**Tech Stack:** Markdown, TOML agent prompts, project-local Codex `SKILL.md`, PowerShell static validation, Git.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Do not modify files under `reference/`.
|
||||
- Do not run Abaqus or another reference solver.
|
||||
- Do not modify C++ source, tests, or CMake.
|
||||
- Preserve HDF5 as authoritative FESA output.
|
||||
- Use the B33 mixed tolerance exactly for MITC4 U; U blocks and UR warns only.
|
||||
- Retain fixed numerical drilling stabilization but exclude calibration and drilling evidence outputs.
|
||||
- Remove `NR-O03` and `NR-O04` from MITC4 implementation readiness.
|
||||
- Implementation Planning may create Harness phase files only after user approval of the Step draft.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Align project-wide policy documents
|
||||
|
||||
**Files:**
|
||||
- Modify: `AGENTS.md`
|
||||
- Modify: `docs/PRD.md`
|
||||
- Modify: `docs/ADR.md`
|
||||
- Modify: `docs/ARCHITECTURE.md`
|
||||
- Modify: `docs/SOLVER_AGENT_DESIGN.md`
|
||||
- Modify: `docs/SOLVER_SKILL_DESIGN.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: approved design in `docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md`
|
||||
- Produces: project-wide independent-solver and minimal-reference policy used by every downstream agent
|
||||
|
||||
- [ ] Replace canonical/legacy/provenance bundle governance with the minimal reference-case contract.
|
||||
- [ ] Separate Abaqus input syntax and external comparison from FESA formulation behavior.
|
||||
- [ ] Record the new decision as a superseding ADR while preserving historical decisions.
|
||||
- [ ] State the MITC4 B33 tolerance reuse and fixed drilling scope without claiming implementation exists.
|
||||
- [ ] Verify changed policy terms with targeted `rg` checks and `git diff --check`.
|
||||
- [ ] Commit with `docs: simplify FESA reference validation policy`.
|
||||
|
||||
### Task 2: Align agent and skill workflow contracts
|
||||
|
||||
**Files:**
|
||||
- Modify: affected `.codex/agents/*.toml`
|
||||
- Modify: affected `.codex/skills/*/SKILL.md`
|
||||
- Modify: affected workflow README templates under `docs/`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 1 project policy
|
||||
- Produces: specialist prompts that no longer recreate removed artifact gates
|
||||
|
||||
- [ ] Update Requirement, Numerical Review, I/O, Reference Model, Reference Verification, Coordinator, Implementation Planning, Physics, and Release boundaries.
|
||||
- [ ] Require Implementation Planning Agent to invoke `$harness`, produce a self-contained multi-Step draft, request user approval, and only then write `phases/` files.
|
||||
- [ ] Keep actual Harness execution outside the planning agent unless explicitly requested.
|
||||
- [ ] Validate TOML syntax, skill YAML frontmatter, required sections, and prohibited stale terms.
|
||||
- [ ] Forward-test the revised skills with an independent agent prompt.
|
||||
- [ ] Commit with `docs: align FESA agents with minimal reference cases`.
|
||||
|
||||
### Task 3: Revise MITC4 requirements and formulation
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/requirements/linear-static-mitc4-shell.md`
|
||||
- Modify: `docs/research/linear-static-mitc4-shell-research.md`
|
||||
- Modify: `docs/formulations/mitc4-shell-formulation.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Tasks 1-2 policy and approved fixed drilling rule
|
||||
- Produces: implementation-ready feature scope without removed calibration gates
|
||||
|
||||
- [ ] Correct the observed S4/S4R reference inventory and adopt the B33 U/UR mixed tolerance.
|
||||
- [ ] Replace drilling candidate/sweep language with the fixed positive rotational-diagonal rule.
|
||||
- [ ] Remove drilling calibration, drilling-energy warning, `NR-O03`, `NR-O04`, and expanded portfolio requirements.
|
||||
- [ ] Preserve the future nonlinear derivation as documentation-only.
|
||||
- [ ] Add a research decision note without rewriting source findings as if they changed.
|
||||
- [ ] Verify requirement IDs and formulation traceability contain no contradictory blocker.
|
||||
- [ ] Commit with `docs: simplify MITC4 verification and drilling scope`.
|
||||
|
||||
### Task 4: Revise MITC4 I/O and reference-case inventory
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/io-definitions/linear-static-mitc4-shell-io.md`
|
||||
- Create: `docs/reference-models/linear-static-mitc4-shell-reference-models.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 3 requirements/formulation
|
||||
- Produces: exact lightweight S4/S4R input-to-displacement-CSV pairing and HDF5 comparison mapping
|
||||
|
||||
- [ ] Remove canonical/legacy/metadata/provenance/schema-version readiness requirements.
|
||||
- [ ] Remove mandatory drilling stiffness, ratio, and energy output evidence.
|
||||
- [ ] Keep minimal source-node plus U/UR component matching and finite/unique-row checks.
|
||||
- [ ] Inventory the exact existing `reference/shell` and `reference/shellR` paths read-only.
|
||||
- [ ] Verify no file under `reference/` changed.
|
||||
- [ ] Commit with `docs: define lightweight MITC4 reference cases`.
|
||||
|
||||
### Task 5: Rerun MITC4 numerical review under the approved policy
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/numerical-reviews/linear-static-mitc4-shell-review.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Tasks 1-4 current source-of-truth documents
|
||||
- Produces: a fresh formulation-only implementation-planning verdict
|
||||
|
||||
- [ ] Reclassify removed artifact, calibration, `NR-O03`, and `NR-O04` blockers explicitly.
|
||||
- [ ] Review the fixed rotational-diagonal drilling rule for dimensions, symmetry, positivity, and separation from physical recovery.
|
||||
- [ ] Confirm U/UR tolerance and current S4/S4R reference pairing are downstream comparison inputs, not formulation blockers.
|
||||
- [ ] Issue the verdict supported by the remaining confirmed formulation findings only.
|
||||
- [ ] Run Markdown structure, targeted policy consistency, `git diff --check`, and no-reference-mutation checks.
|
||||
- [ ] Commit with `docs: rerun MITC4 numerical review under minimal reference policy`.
|
||||
|
||||
### Task 6: Final repository policy verification
|
||||
|
||||
**Files:**
|
||||
- Verify: all files changed by Tasks 1-5
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: all preceding tasks
|
||||
- Produces: clean, internally consistent documentation and agent/skill configuration evidence
|
||||
|
||||
- [ ] Run `uv run --with pytest python -m pytest -v -rs` because project-local agent/skill workflow documents changed.
|
||||
- [ ] Parse every changed TOML file and validate every changed skill frontmatter.
|
||||
- [ ] Run repository-wide targeted searches for stale mandatory provenance, canonical, README, drilling sweep, `NR-O03`, and `NR-O04` language in active contracts.
|
||||
- [ ] Run `git diff --check` and inspect `git status --short`.
|
||||
- [ ] Confirm `git diff -- reference` is empty.
|
||||
- [ ] Report any historical documents intentionally left unchanged.
|
||||
@@ -0,0 +1,178 @@
|
||||
# Independent Reference Validation Policy Design
|
||||
|
||||
## Status
|
||||
|
||||
- date: `2026-08-12`
|
||||
- status: `approved`
|
||||
- scope: FESA project policy, agents, skills, and `linear-static-mitc4-shell` upstream contracts
|
||||
- implementation_code: out of scope
|
||||
- reference_artifact_mutation: prohibited
|
||||
|
||||
## Decision
|
||||
|
||||
FESA is an independent finite element solver. It accepts an approved subset of the
|
||||
Abaqus `.inp` syntax, but it does not reproduce or claim equivalence with Abaqus
|
||||
element formulations, integration rules, stabilization, internal state, or result
|
||||
generation procedures. Abaqus CSV values are external numerical reference values.
|
||||
Exact equality is allowed but is neither required nor evidence of identical internal
|
||||
behavior.
|
||||
|
||||
A feature passes reference verification when FESA solves the declared input and its
|
||||
required comparison quantities satisfy the feature-approved tolerance. Reference
|
||||
artifact governance that does not participate in that comparison must not block
|
||||
implementation planning or verification.
|
||||
|
||||
## Minimal Reference Case Contract
|
||||
|
||||
A reference case requires only:
|
||||
|
||||
- the declared `.inp` file used by both solvers;
|
||||
- the declared Abaqus CSV file for each blocking or warning-only quantity;
|
||||
- the FESA `results.h5` produced from that `.inp`;
|
||||
- a deterministic mapping of the compared source identity and components;
|
||||
- the approved tolerance.
|
||||
|
||||
The following are not readiness requirements:
|
||||
|
||||
- canonical directory or filename conventions;
|
||||
- legacy-alias approval;
|
||||
- bundle `README.md`;
|
||||
- `metadata.json`;
|
||||
- Abaqus version or generation provenance;
|
||||
- duplicated unit, coordinate, model, step, final-frame, material, thickness, or
|
||||
element-type metadata;
|
||||
- a reference CSV schema version.
|
||||
|
||||
Reference paths and filenames are consumed as declared without rename or repair.
|
||||
Reference artifacts remain read-only. A comparator must still reject ambiguous or
|
||||
unsafe comparisons: missing files, missing required source IDs/components, duplicate
|
||||
IDs, unmatched required rows, and nonfinite values. This is an executable matching
|
||||
contract, not a metadata or bundle-governance contract.
|
||||
|
||||
For a feature with one supported static step and one final frame, the input/CSV pair
|
||||
implicitly identifies that result state. Model properties and source element types are
|
||||
read from the `.inp` rather than copied into another metadata file.
|
||||
|
||||
## Reference Model Phase
|
||||
|
||||
The Reference Model phase remains in the workflow as a lightweight reference-case
|
||||
inventory. It records:
|
||||
|
||||
- the purpose of each existing case;
|
||||
- exact input and required CSV paths;
|
||||
- blocking and warning-only quantities;
|
||||
- FESA HDF5 projection and source-ID/component matching;
|
||||
- the approved tolerance;
|
||||
- whether the required files are present and readable.
|
||||
|
||||
It does not design a broad benchmark portfolio unless the feature requirements
|
||||
explicitly request one. It does not gate on naming, README, metadata, provenance, or
|
||||
duplicated descriptive fields.
|
||||
|
||||
## MITC4 Reference Acceptance
|
||||
|
||||
Both source element types map to the same internal `FESA-MITC4` formulation:
|
||||
|
||||
- `reference/shell/shell.inp` and `reference/shell/shell displacements.csv` cover
|
||||
source `S4`;
|
||||
- `reference/shellR/shellR.inp` and
|
||||
`reference/shellR/shellR displacements.csv` cover source `S4R`.
|
||||
|
||||
Only matched global `U1/U2/U3` rows are blocking. `UR1/UR2/UR3` rows are compared and
|
||||
reported as warning-only evidence. Reactions and stresses may be inspected but are not
|
||||
reference pass/fail quantities.
|
||||
|
||||
For each case and component `c`:
|
||||
|
||||
```text
|
||||
reference_scale_c = max_i(abs(abaqus_value_i,c))
|
||||
tolerance_c = 1e-9 + 1e-6 * reference_scale_c
|
||||
```
|
||||
|
||||
The `1e-9` absolute floor is expressed in the model's user-consistent length unit for
|
||||
translation and as a dimensionless rotation value for warning-only UR. Values are not
|
||||
zero-clamped. Missing, duplicate, unmatched, or nonfinite required rows fail before
|
||||
numeric tolerance evaluation. U tolerance failure fails reference verification; UR
|
||||
tolerance exceedance emits a deterministic warning only.
|
||||
|
||||
No additional locking, convergence, distorted, curved, director-angle, or invalid
|
||||
geometry reference portfolio is required for this MITC4 implementation acceptance.
|
||||
The previously open `NR-O03` and `NR-O04` calibration/inspection requirements are
|
||||
removed from the feature gate.
|
||||
|
||||
## MITC4 Drilling Boundary
|
||||
|
||||
Drilling calibration is excluded, but a fixed numerical stabilization remains required
|
||||
to make the external six-DOF shell system solvable. The feature does not implement a
|
||||
physical drilling strain or accept director-parallel nodal moments.
|
||||
|
||||
Use one element-level scalar computed from the positive diagonal entries of the
|
||||
physical rotational stiffness block:
|
||||
|
||||
```text
|
||||
k_d = 1e-3 * min(K_phys[i,i] | i is a physical rotational DOF and K_phys[i,i] > 0)
|
||||
K_drill_local = k_d * I4
|
||||
K_drill_global = T_d^T * K_drill_local * T_d
|
||||
```
|
||||
|
||||
Restricting the minimum to rotational entries preserves the `force*length` dimension.
|
||||
The literal minimum over mixed translational and rotational diagonals is not approved.
|
||||
The stabilization must be finite, symmetric, positive, deterministic, and excluded
|
||||
from physical generalized strain, resultant, and stress recovery.
|
||||
|
||||
The following are out of scope:
|
||||
|
||||
- reference-family selection and `rho_d` normalization;
|
||||
- coefficient sweeps, stable plateaus, rank/conditioning calibration, and adjacent
|
||||
decade sensitivity;
|
||||
- drilling-to-physical energy thresholds or warning policies;
|
||||
- mandatory drilling stiffness, ratio, or energy result datasets.
|
||||
|
||||
Basic unit tests may verify construction, symmetry, positivity, deterministic behavior,
|
||||
and removal of the four nonphysical drilling null modes. Current reference cases contain
|
||||
no drilling-direction load; their blocking safeguard against unacceptable physical
|
||||
response contamination is the approved U comparison.
|
||||
|
||||
## Numerical Review Boundary
|
||||
|
||||
Numerical Review decides whether the documented formulation is internally complete and
|
||||
consistent enough to plan implementation. It must not fail because a later Reference
|
||||
Model document, canonical bundle, README, metadata, provenance, extended benchmark
|
||||
portfolio, or comparison result is absent.
|
||||
|
||||
For MITC4:
|
||||
|
||||
- fixed drilling stabilization closes `NR-O01` and removes `NR-O02` by scope;
|
||||
- `NR-O03` and `NR-O04` are removed from the approved verification scope;
|
||||
- the B33 mixed tolerance closes the blocking part of `NR-O05`;
|
||||
- missing heavyweight reference metadata is `not-required-by-policy`;
|
||||
- confirmed formulation defects remain blocking.
|
||||
|
||||
## Agent and Skill Workflow
|
||||
|
||||
Requirement, I/O, Reference Model, Reference Verification, Coordinator, Implementation
|
||||
Planning, Physics Evaluation, and Release instructions must use the minimal reference
|
||||
case contract. Numerical Review must remain independent of later artifact governance.
|
||||
|
||||
Implementation Planning must use the project-local `harness` skill when the user asks
|
||||
for an implementation plan. It must:
|
||||
|
||||
1. prepare a multi-Step draft with one layer or module per Step;
|
||||
2. make each Step self-contained for an independent Codex execution;
|
||||
3. include prerequisite documents, TDD RED/GREEN/VERIFY work, exact MSVC/CMake/CTest
|
||||
acceptance commands, and specific prohibitions;
|
||||
4. request user approval for the Step draft;
|
||||
5. only after approval create or update `phases/index.json`,
|
||||
`phases/<task-name>/index.json`, and `phases/<task-name>/stepN.md`;
|
||||
6. leave Harness execution to an explicit subsequent user request.
|
||||
|
||||
## Scope Preservation
|
||||
|
||||
- Do not modify files under `reference/`.
|
||||
- Do not run Abaqus or another reference solver.
|
||||
- Do not add C++ MITC4 implementation in this policy-alignment change.
|
||||
- Keep the future geometric-nonlinear residual/tangent derivation documentation, but do
|
||||
not authorize nonlinear product execution.
|
||||
- Historical research evidence may retain its original cautions. Current requirements,
|
||||
formulation decisions, I/O contracts, and the latest numerical review must clearly
|
||||
identify the approved policy that supersedes earlier recommendations.
|
||||
Reference in New Issue
Block a user