fix: restore implementation comparison safeguards
This commit is contained in:
@@ -26,6 +26,7 @@ Hard boundaries:
|
|||||||
- Do not change formulations directly to make implementation easier.
|
- Do not change formulations directly to make implementation easier.
|
||||||
- Do not change I/O contracts or reference artifacts to make tests pass.
|
- Do not change I/O contracts or reference artifacts to make tests pass.
|
||||||
- Do not change reference artifacts.
|
- Do not change reference artifacts.
|
||||||
|
- Do not modify `docs/<feature-id>/reference-model.md` or its reference-model contracts/evidence to make comparisons pass, including declared comparison quantities, source identity/component rules, artifact contracts, or tolerance policy.
|
||||||
- Do not run Abaqus, Nastran, or any reference solver.
|
- Do not run Abaqus, Nastran, or any reference solver.
|
||||||
- Do not generate or modify Abaqus reference CSV files.
|
- Do not generate or modify Abaqus reference CSV files.
|
||||||
- Do not approve release readiness.
|
- Do not approve release readiness.
|
||||||
@@ -116,11 +117,15 @@ Required reference-comparison.md sections:
|
|||||||
1. Metadata: feature_id, source docs and reports, status, owner_agent: implementation-agent, date.
|
1. Metadata: feature_id, source docs and reports, status, owner_agent: implementation-agent, date.
|
||||||
2. Artifact Inventory: exact declared input/CSV paths, generated `results.h5`, and optional FESA debug CSV view.
|
2. Artifact Inventory: exact declared input/CSV paths, generated `results.h5`, and optional FESA debug CSV view.
|
||||||
3. Comparison Contract: HDF5 projection from `io.md`, source-ID/component matching, row prechecks, tolerance source, and blocking/warning behavior.
|
3. Comparison Contract: HDF5 projection from `io.md`, source-ID/component matching, row prechecks, tolerance source, and blocking/warning behavior.
|
||||||
4. Quantity Results: row counts, errors, worst id/component, and pass/fail or warning for every declared quantity.
|
4. Quantity Results: per-row decisions, row counts, max absolute error, max relative or component-normalized error, RMS error, norm error, worst id/component, and pass/fail or warning for every declared quantity; include each error metric only where the approved feature contract makes it applicable.
|
||||||
5. Failure Classification and Handoff Recommendation through the Coordinator Agent.
|
5. Failure Classification and Handoff Recommendation through the Coordinator Agent.
|
||||||
6. No-Change Assertion: source, test, CMake, declared inputs, reference artifacts, and tolerance policies were not modified.
|
6. No-Change Assertion: source, test, CMake, declared inputs, reference artifacts, and tolerance policies were not modified.
|
||||||
7. Open Issues.
|
7. Open Issues.
|
||||||
|
|
||||||
|
Reference comparison quality gate:
|
||||||
|
- Record per-row decisions and the applicable max absolute error, max relative or component-normalized error, RMS error, and norm error for each declared quantity.
|
||||||
|
- Do not calculate, invent, or treat a metric as required when the approved feature contract does not make it applicable.
|
||||||
|
|
||||||
Validation commands:
|
Validation commands:
|
||||||
- cmake -S . -B .harness/build -A x64
|
- cmake -S . -B .harness/build -A x64
|
||||||
- cmake --build .harness/build --config Debug
|
- cmake --build .harness/build --config Debug
|
||||||
|
|||||||
@@ -70,7 +70,9 @@ test inventory, handoff recommendation, no-change assertion, and open issues.
|
|||||||
|
|
||||||
`reference-comparison.md` records the exact input/CSV artifact inventory, `results.h5`, HDF5
|
`reference-comparison.md` records the exact input/CSV artifact inventory, `results.h5`, HDF5
|
||||||
projection, source-ID/component matching, row prechecks, approved tolerance, per-quantity
|
projection, source-ID/component matching, row prechecks, approved tolerance, per-quantity
|
||||||
results, classification, handoff, no-change assertion, and open issues.
|
per-row decisions, max absolute error, max relative or component-normalized error, RMS error,
|
||||||
|
norm error where the approved feature contract makes each metric applicable, classification,
|
||||||
|
handoff, no-change assertion, and open issues.
|
||||||
|
|
||||||
Required validation commands:
|
Required validation commands:
|
||||||
|
|
||||||
@@ -99,6 +101,9 @@ uv run --with pytest python -m pytest -v -rs
|
|||||||
- Do not change reference artifacts.
|
- Do not change reference artifacts.
|
||||||
- Do not change tolerance policies.
|
- Do not change tolerance policies.
|
||||||
- Do not change declared reference inputs.
|
- Do not change declared reference inputs.
|
||||||
|
- Do not modify `docs/<feature-id>/reference-model.md` or its reference-model contracts/evidence,
|
||||||
|
including declared comparison quantities, source identity/component rules, artifact contracts,
|
||||||
|
or approved tolerance, to make comparisons pass.
|
||||||
- Do not run Abaqus, Nastran, or any reference solver.
|
- Do not run Abaqus, Nastran, or any reference solver.
|
||||||
- Do not generate or modify Abaqus reference CSV files.
|
- Do not generate or modify Abaqus reference CSV files.
|
||||||
- Do not approve release readiness.
|
- Do not approve release readiness.
|
||||||
@@ -114,6 +119,9 @@ uv run --with pytest python -m pytest -v -rs
|
|||||||
- Build/test reports record command, exit code, duration, stdout/stderr tail, and failure classification.
|
- Build/test reports record command, exit code, duration, stdout/stderr tail, and failure classification.
|
||||||
- Reference comparison rejects missing, extra, duplicate, and nonfinite required rows before tolerance.
|
- Reference comparison rejects missing, extra, duplicate, and nonfinite required rows before tolerance.
|
||||||
- Warning-only quantities never change the blocking pass/fail result.
|
- Warning-only quantities never change the blocking pass/fail result.
|
||||||
|
- Reference comparison records per-row decisions and, only where the approved feature contract
|
||||||
|
makes each metric applicable, max absolute error, max relative or component-normalized error,
|
||||||
|
RMS error, and norm error.
|
||||||
- Correction attempts stop when repeated failure indicates upstream contract ambiguity.
|
- Correction attempts stop when repeated failure indicates upstream contract ambiguity.
|
||||||
|
|
||||||
## Handoff
|
## Handoff
|
||||||
|
|||||||
Reference in New Issue
Block a user