test: add quad02 stored reference regression
This commit is contained in:
NINI
@@ -13,7 +13,7 @@ Every new agent session must read this file together with `PROGRESS.md` before p
|
||||
- If an item becomes obsolete, move it to `PROGRESS.md` with a short reason instead of silently deleting it.
|
||||
|
||||
## Current Objective
|
||||
Continue the new Phase 1 rebaseline plan in `phases/1-linear-static-mitc4-rebaseline`, starting with P1R-14 stored-reference displacement regression using accepted Phase 1-compatible S4 cases. The old `phases/1-linear-static-mitc4` path is historical and superseded by the paper-based MITC4 formulation reset.
|
||||
Continue the new Phase 1 rebaseline plan in `phases/1-linear-static-mitc4-rebaseline`, starting with P1R-15 independent evaluator closeout. The old `phases/1-linear-static-mitc4` path is historical and superseded by the paper-based MITC4 formulation reset.
|
||||
|
||||
## Required Reading For New Agents
|
||||
1. `AGENTS.md`
|
||||
@@ -36,7 +36,7 @@ Continue the new Phase 1 rebaseline plan in `phases/1-linear-static-mitc4-rebase
|
||||
## Active Phase Files
|
||||
- Active phase directory: `phases/1-linear-static-mitc4-rebaseline`
|
||||
- Execute with: `python scripts/execute.py 1-linear-static-mitc4-rebaseline`
|
||||
- Step numbering is zero-based. `step0.md` is complete and recorded in `phases/1-linear-static-mitc4-rebaseline/step0-audit.md`; `step1.md` is complete and created the `quad_02_phase1.inp` normalized reference path; `step2.md` is complete and revalidated core harness guardrails; `step3.md` is complete and revalidated the Phase 1 parser/domain subset; `step4.md` is complete and strengthened validation/singular diagnostics; `step5.md` is complete and revalidated the DofManager/reaction foundation; `step6.md` is complete and revalidated the minimum result model plus displacement CSV comparator; `step7.md` is complete and revalidated MITC4 natural coordinates, tying points, center directors, and integration bases; `step8.md` is complete and revalidated degenerated-continuum displacement, direct covariant strain rows, and MITC shear tying rows; `step9.md` is complete and revalidated plane-stress material, convected-to-local transform, and `2 x 2 x 2` material integration scaffolding; `step10.md` is complete and revalidated MITC4 stiffness, internal force, six-DOF transform, and drilling stabilization; `step11.md` is complete and added MITC4 membrane, bending, shear, twist, drilling-sensitivity, and thin-cantilever locking-sensitivity tests; `step12.md` is complete and revalidated full-space assembly, reduced projection, deterministic sparse-pattern scaffold, solver adapter injection, and full-vector internal/reaction force state; `step13.md` is complete and revalidated active AnalysisModel construction plus input-to-AnalysisState-to-U/RF result workflow; `step15.md` is the independent evaluator closeout.
|
||||
- Step numbering is zero-based. `step0.md` is complete and recorded in `phases/1-linear-static-mitc4-rebaseline/step0-audit.md`; `step1.md` is complete and created the `quad_02_phase1.inp` normalized reference path; `step2.md` is complete and revalidated core harness guardrails; `step3.md` is complete and revalidated the Phase 1 parser/domain subset; `step4.md` is complete and strengthened validation/singular diagnostics; `step5.md` is complete and revalidated the DofManager/reaction foundation; `step6.md` is complete and revalidated the minimum result model plus displacement CSV comparator; `step7.md` is complete and revalidated MITC4 natural coordinates, tying points, center directors, and integration bases; `step8.md` is complete and revalidated degenerated-continuum displacement, direct covariant strain rows, and MITC shear tying rows; `step9.md` is complete and revalidated plane-stress material, convected-to-local transform, and `2 x 2 x 2` material integration scaffolding; `step10.md` is complete and revalidated MITC4 stiffness, internal force, six-DOF transform, and drilling stabilization; `step11.md` is complete and added MITC4 membrane, bending, shear, twist, drilling-sensitivity, and thin-cantilever locking-sensitivity tests; `step12.md` is complete and revalidated full-space assembly, reduced projection, deterministic sparse-pattern scaffold, solver adapter injection, and full-vector internal/reaction force state; `step13.md` is complete and revalidated active AnalysisModel construction plus input-to-AnalysisState-to-U/RF result workflow; `step14.md` is complete and added the first stored Abaqus displacement regression for `quad_02_phase1`; `step15.md` is the independent evaluator closeout.
|
||||
- Every step file contains a sprint contract with objective, required reading, scope, allowed files, explicit non-goals, tests to write first, reference artifacts, acceptance command, evaluator checklist, and handoff requirements.
|
||||
- Historical phase directory: `phases/1-linear-static-mitc4`
|
||||
- Historical phase status: blocked/superseded. Do not resume the old P1-15/P1-16 path unless the user explicitly requests recovery of that exact phase.
|
||||
@@ -77,7 +77,7 @@ Each gate should be satisfied before moving to the next implementation band unle
|
||||
| G2 - Parser and domain | satisfied | Parser subset revalidated in step 3; validation and singular diagnostics revalidated in step 4. | Parser acceptance/rejection tests, validation negative tests, and validation output |
|
||||
| G3 - DOF/math/results infrastructure | satisfied | Core aliases, DOF mapping, validation harness, model diagnostic context, DofManager, sparse-connectivity inputs, full-vector reaction formula, result model metadata, displacement CSV comparator, full-space assembly, reduced projection, sparse-pattern scaffold, and solver adapter boundary were revalidated in steps 2, 5, 6, and 12. | P1R-02, P1R-05, P1R-06, and P1R-12 validation output |
|
||||
| G4 - MITC4 element readiness | satisfied | MITC4 formulation was rewritten from local papers; Steps 7 through 11 rebuilt geometry/director/local-basis scaffolding, displacement interpolation, direct covariant strain rows, MITC shear tying rows, plane-stress material, convected-to-local transform, `2 x 2 x 2` material integration scaffolding, stiffness/internal force, six-DOF transform, drilling stabilization, and patch/locking-sensitivity tests. | P1R-07 through P1R-11 validation output |
|
||||
| G5 - End-to-end solver | partial | Linear static input-to-result workflow is revalidated through step 13; stored `quad_02_phase1` displacement regression remains for step 14. | P1R-13 validation output; future reference regression output |
|
||||
| G5 - End-to-end solver | satisfied-with-gap | Linear static input-to-result workflow is revalidated through step 13, and `quad_02_phase1` stored displacement regression passes in step 14. The broader PRD target of three stored references remains open in R-013. | P1R-13 and P1R-14 validation output |
|
||||
|
||||
## Phase 1 Implementation Milestones
|
||||
All milestones are intended to become one or more self-contained sprint contracts or `phases/{phase}/stepN.md` files. Each sprint must follow `docs/HARNESS_ENGINEERING.md` and be evaluated independently.
|
||||
@@ -95,7 +95,7 @@ All milestones are intended to become one or more self-contained sprint contract
|
||||
| P1R-11 | completed | verification generator | Add MITC4 patch, locking-sensitivity, and benchmark tests. | P1R-10 | Membrane/bending/shear/twist/locking tests |
|
||||
| P1R-12 | completed | assembly generator | Rebuild assembly, solver adapter boundary, constrained solve, and full-vector RF recovery. | P1R-05, P1R-10 | Assembly and full-vector reaction tests |
|
||||
| P1R-13 | completed | analysis generator | Rebuild linear static workflow from input to U/RF result fields. | P1R-03, P1R-04, P1R-06, P1R-12 | End-to-end linear static tests |
|
||||
| P1R-14 | pending | reference generator | Run stored reference displacement regression using accepted Phase 1-compatible S4 cases. | P1R-13 | At least one automated CSV displacement regression |
|
||||
| P1R-14 | completed | reference generator | Run stored reference displacement regression using accepted Phase 1-compatible S4 cases. | P1R-13 | At least one automated CSV displacement regression |
|
||||
| P1R-15 | pending | evaluator | Independent Phase 1 evaluator closeout. | P1R-14 | Pass/fail report, synchronized PLAN/PROGRESS |
|
||||
|
||||
## Phase 1 Sprint Contract Rules
|
||||
|
||||
+28
-1
@@ -13,10 +13,37 @@ Every new agent session must read this file together with `PLAN.md` before plann
|
||||
- Do not remove history unless the user explicitly asks for archival cleanup.
|
||||
|
||||
## Current Status
|
||||
Phase 1 has a new rebaseline phase definition in `phases/1-linear-static-mitc4-rebaseline`. Steps 0 through 13 are complete. `quad_02_phase1.inp` is now the normalized Phase 1-compatible input path for the stored `quad_02` S4 reference pair, while the original `quad_02.inp` remains preserved unsupported provenance. Core numeric aliases, DOF mapping, validation harness, model diagnostic context, the Phase 1 parser/domain subset, validation/singular diagnostics, DofManager/reaction foundation, minimum result model metadata, displacement CSV comparator foundation, MITC4 geometry/director scaffolding, MITC4 displacement/strain/tying row scaffolding, MITC4 material/transform/integration scaffolding, MITC4 stiffness/drilling/internal-force scaffolding, MITC4 patch/locking-sensitivity tests, full-space assembly, reduced projection, sparse-pattern scaffold, solver adapter injection, full-vector internal/reaction force state, active AnalysisModel construction, and input-to-AnalysisState-to-U/RF result workflow have been revalidated. The old `phases/1-linear-static-mitc4` path is historical and superseded after the MITC4 formulation reset.
|
||||
Phase 1 has a new rebaseline phase definition in `phases/1-linear-static-mitc4-rebaseline`. Steps 0 through 14 are complete. `quad_02_phase1.inp` is now the normalized Phase 1-compatible input path for the stored `quad_02` S4 reference pair, while the original `quad_02.inp` remains preserved unsupported provenance. Core numeric aliases, DOF mapping, validation harness, model diagnostic context, the Phase 1 parser/domain subset, validation/singular diagnostics, DofManager/reaction foundation, minimum result model metadata, displacement CSV comparator foundation, MITC4 geometry/director scaffolding, MITC4 displacement/strain/tying row scaffolding, MITC4 material/transform/integration scaffolding, MITC4 stiffness/drilling/internal-force scaffolding, MITC4 patch/locking-sensitivity tests, full-space assembly, reduced projection, sparse-pattern scaffold, solver adapter injection, full-vector internal/reaction force state, active AnalysisModel construction, input-to-AnalysisState-to-U/RF result workflow, and the first stored Abaqus displacement regression have been revalidated. The old `phases/1-linear-static-mitc4` path is historical and superseded after the MITC4 formulation reset.
|
||||
|
||||
## Completed Work
|
||||
|
||||
### 2026-05-04 - P1R-14 stored reference regression completed
|
||||
Author: Codex
|
||||
|
||||
Changed files:
|
||||
- `tests/test_main.cpp`
|
||||
- `docs/VERIFICATION_PLAN.md`
|
||||
- `references/quad_02_notes.md`
|
||||
- `phases/1-linear-static-mitc4-rebaseline/index.json`
|
||||
- `PLAN.md`
|
||||
- `PROGRESS.md`
|
||||
|
||||
Summary:
|
||||
- Added a fixture discovery test that keeps `references/quad_02.inp` as unsupported Abaqus/CAE provenance while verifying `references/quad_02_phase1.inp` is the accepted executable Phase 1 derivative.
|
||||
- Added the first automated stored Abaqus displacement regression: run `quad_02_phase1.inp`, extract FESA `/results/steps/Step-1/frames/0/fieldOutputs/U`, and compare against `references/quad_02_displacements.csv` by node id.
|
||||
- Used the documented tolerance for this reference pair: `abs_tol = 1.0e-12`, `rel_tol = 1.0e-5`, `reference_scale = 1.0`.
|
||||
- Updated verification/reference notes so `quad_02_phase1` is no longer a future regression target; it is now active test coverage.
|
||||
|
||||
Verification:
|
||||
- `python scripts/validate_workspace.py` configured CMake, built `fesa_core` and `fesa_tests`, and ran CTest successfully.
|
||||
- CTest result: 1 test executable passed.
|
||||
- Stored reference result: `quad_02_phase1.inp` displacement regression passed against `quad_02_displacements.csv` using the tolerance above.
|
||||
|
||||
Follow-up:
|
||||
- Continue with P1R-15 independent evaluator closeout.
|
||||
- R-013 remains open: Phase 1 still needs enough additional stored Abaqus S4 reference cases for the PRD target of three stored references: one single-element case, one simple multi-element plate/shell case, and one curved shell benchmark.
|
||||
- R-010 remains open: no `quad_02_reactions.csv` or other Abaqus reaction CSV exists yet, so `RF` remains verified by internal full-vector equilibrium.
|
||||
|
||||
### 2026-05-04 - P1R-13 linear static workflow completed
|
||||
Author: Codex
|
||||
|
||||
|
||||
@@ -190,6 +190,21 @@ Initial guidance:
|
||||
|
||||
Final benchmark tolerances must be stored with each reference case.
|
||||
|
||||
### Step 14 Stored Reference Regression Status
|
||||
The first automated stored-reference displacement regression is active for:
|
||||
|
||||
```text
|
||||
input: references/quad_02_phase1.inp
|
||||
expected U: references/quad_02_displacements.csv
|
||||
```
|
||||
|
||||
Comparison rules:
|
||||
- The original `references/quad_02.inp` remains unsupported provenance and must still be rejected by parser compatibility tests because it contains Abaqus/CAE scaffolding.
|
||||
- The normalized `quad_02_phase1.inp` is the executable Phase 1 input for this reference pair.
|
||||
- The FESA `U` field is compared node-id-by-node-id against Abaqus CSV columns `U-U1`, `U-U2`, `U-U3`, `UR-UR1`, `UR-UR2`, and `UR-UR3`.
|
||||
- The active tolerance is `abs_tol = 1.0e-12`, `rel_tol = 1.0e-5`, `reference_scale = 1.0`.
|
||||
- Abaqus reaction CSV is still unavailable, so `RF` remains verified by full-vector equilibrium tests until a `*_reactions.csv` artifact is provided.
|
||||
|
||||
## Phase 1 Benchmark Matrix
|
||||
| Case | Purpose | Required Output |
|
||||
|---|---|---|
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
{ "step": 11, "name": "mitc4-patch-benchmark-tests", "status": "completed" },
|
||||
{ "step": 12, "name": "assembly-sparse-solver-path", "status": "completed" },
|
||||
{ "step": 13, "name": "linear-static-workflow", "status": "completed" },
|
||||
{ "step": 14, "name": "stored-reference-regression", "status": "pending" },
|
||||
{ "step": 14, "name": "stored-reference-regression", "status": "completed" },
|
||||
{ "step": 15, "name": "phase1-evaluator-closeout", "status": "pending" }
|
||||
]
|
||||
}
|
||||
|
||||
@@ -70,7 +70,7 @@ UX, UY, UZ, RX, RY, RZ
|
||||
```
|
||||
|
||||
## Initial Tolerance
|
||||
Use the project default reference displacement starting point until case-specific calibration is justified:
|
||||
The active automated displacement regression uses:
|
||||
|
||||
```text
|
||||
abs_tol = 1.0e-12
|
||||
@@ -80,6 +80,11 @@ reference_scale = 1.0
|
||||
|
||||
Do not tune tolerances or drilling stiffness to make this single case pass.
|
||||
|
||||
## Automated Regression Status
|
||||
`quad_02_phase1.inp` and `quad_02_displacements.csv` are wired into the Phase 1 test suite as the first stored Abaqus displacement regression.
|
||||
|
||||
The regression compares FESA `U` against the stored Abaqus CSV by node id and uses the tolerance above.
|
||||
|
||||
## Current Limitations
|
||||
- `RF` has no paired Abaqus reaction CSV yet; verify `RF` by full-vector equilibrium until a `quad_02_reactions.csv` artifact is provided.
|
||||
- Solver displacement comparison against this case must wait until the MITC4 rebuild and end-to-end linear static workflow are complete.
|
||||
- This is currently the only passing stored Abaqus reference regression. The PRD target still requires at least three stored reference models: one single-element case, one simple multi-element plate/shell case, and one curved shell benchmark.
|
||||
|
||||
@@ -190,6 +190,23 @@ std::vector<fesa::Real> toVector(const fesa::MITC4ElementDofVector& values) {
|
||||
return {values.begin(), values.end()};
|
||||
}
|
||||
|
||||
std::string readTextFile(const std::string& path) {
|
||||
std::ifstream input(path);
|
||||
std::ostringstream buffer;
|
||||
buffer << input.rdbuf();
|
||||
FESA_CHECK(input.good() || !buffer.str().empty());
|
||||
return buffer.str();
|
||||
}
|
||||
|
||||
void checkComparisonPass(const fesa::ComparisonResult& comparison) {
|
||||
if (!comparison.pass) {
|
||||
throw std::runtime_error("reference comparison failed: max_abs_error=" +
|
||||
std::to_string(comparison.max_abs_error) +
|
||||
", max_rel_error=" + std::to_string(comparison.max_rel_error) +
|
||||
", diagnostics=" + std::to_string(comparison.diagnostics.size()));
|
||||
}
|
||||
}
|
||||
|
||||
fesa::MITC4StrainVector localStrainAt(const fesa::MITC4Geometry& geometry,
|
||||
const fesa::MITC4ElementDofVector& values,
|
||||
fesa::Real xi,
|
||||
@@ -1057,6 +1074,45 @@ FESA_TEST(displacement_comparator_reports_duplicate_actual_nodes) {
|
||||
FESA_CHECK(fesa::containsDiagnostic(compared.diagnostics, "FESA-COMPARE-DUPLICATE-ACTUAL"));
|
||||
}
|
||||
|
||||
FESA_TEST(quad02_reference_fixture_discovery_is_consistent) {
|
||||
fesa::AbaqusInputParser parser;
|
||||
const auto original = parser.parseFile(sourceRoot() + "/references/quad_02.inp");
|
||||
FESA_CHECK(!original.ok());
|
||||
FESA_CHECK(fesa::containsDiagnostic(original.diagnostics, "FESA-PARSE-UNSUPPORTED-KEYWORD"));
|
||||
|
||||
const auto normalized = parser.parseFile(sourceRoot() + "/references/quad_02_phase1.inp");
|
||||
FESA_CHECK(normalized.ok());
|
||||
FESA_CHECK(normalized.domain.nodes.size() == 121);
|
||||
FESA_CHECK(normalized.domain.elements.size() == 100);
|
||||
FESA_CHECK(normalized.domain.steps.size() == 1);
|
||||
FESA_CHECK(normalized.domain.steps.front().name == "Step-1");
|
||||
|
||||
const auto reference = fesa::loadDisplacementCsv(sourceRoot() + "/references/quad_02_displacements.csv");
|
||||
FESA_CHECK(!fesa::hasError(reference.diagnostics));
|
||||
FESA_CHECK(reference.rows.size() == normalized.domain.nodes.size());
|
||||
for (const auto& [node_id, node] : normalized.domain.nodes) {
|
||||
(void)node;
|
||||
FESA_CHECK(reference.rows.count(node_id) == 1);
|
||||
}
|
||||
}
|
||||
|
||||
FESA_TEST(quad02_phase1_stored_displacement_reference_regression) {
|
||||
const auto input_text = readTextFile(sourceRoot() + "/references/quad_02_phase1.inp");
|
||||
const auto analysis = fesa::runLinearStaticInputString(input_text, "quad_02_phase1.inp");
|
||||
FESA_CHECK(analysis.ok());
|
||||
FESA_CHECK(analysis.state.converged);
|
||||
FESA_CHECK(analysis.result_file.steps.size() == 1);
|
||||
const auto& frame = analysis.result_file.steps[0].frames[0];
|
||||
FESA_CHECK(frame.field_outputs.count("U") == 1);
|
||||
|
||||
const auto expected = fesa::loadDisplacementCsv(sourceRoot() + "/references/quad_02_displacements.csv");
|
||||
FESA_CHECK(!fesa::hasError(expected.diagnostics));
|
||||
const auto comparison = fesa::compareDisplacements(frame.field_outputs.at("U"), expected, {1.0e-12, 1.0e-5, 1.0});
|
||||
checkComparisonPass(comparison);
|
||||
FESA_CHECK(comparison.max_abs_error <= 1.0e-5);
|
||||
FESA_CHECK(comparison.max_rel_error <= 1.0e-5);
|
||||
}
|
||||
|
||||
FESA_TEST(mitc4_shape_functions_node_order_and_tying_points) {
|
||||
auto center = fesa::shapeFunctions(0.0, 0.0);
|
||||
const fesa::Real sum = center.n[0] + center.n[1] + center.n[2] + center.n[3];
|
||||
|
||||
Reference in New Issue
Block a user