diff --git a/AGENTS.md b/AGENTS.md index 6e80e7c..2b1e0ce 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,6 +15,8 @@ - 프로젝트의 배경, 목적, 사용자, 범위는 `docs/PRD.md`에서 확인한다. - 전체 아키텍처와 모듈별 책임은 `docs/ARCHITECTURE.md`에서 확인한다. - 주요 아키텍처 결정과 그 이유 및 트레이드오프는 `docs/ADR.md`에서 확인한다. +- 외부 reference tolerance 값, 판정 과정, 보고서와 변경 관리는 + `docs/TOLERANCE.md`를 따른다. - AI 에이전트는 설계나 구현 결정을 내리기 전에 PRD에서 제품 범위를, ARCHITECTURE에서 소유권과 데이터 흐름을, ADR에서 이미 결정된 트레이드오프를 먼저 확인한다. 기능별 요구조건, 연구, 정식화, 수치/reference 검토, I/O, 구현, 검증, 물리 및 release 의미는 `docs//`의 승인 문서를 source of truth로 삼는다. - Harness는 솔버 자체가 아니라 요구조건, TDD, phase 실행, 검증을 통제하는 개발 운영 인프라이다. 전체 실행 흐름은 `docs/HARNESS_WORKFLOW.md`, 설치와 설정은 `docs/HARNESS.md`를 따른다. - FESA는 Abaqus와 독립적인 솔버다. 문서와 구현은 full Abaqus compatibility뿐 아니라 @@ -32,6 +34,18 @@ - 비교기는 필요한 source row identity와 component를 결정적으로 대응시키고 누락, 추가, 중복, nonfinite 값을 tolerance 전에 거부한다. Reference artifact는 rename, rewrite 또는 보정하지 않는다. +- 모든 외부 reference comparison은 `docs/TOLERANCE.md`의 공통 정책을 사용한다. + 설계 근거는 `docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md`와 + ADR-022에 보존한다. 동일 logical quantity, unit dimension, coordinate system과 blocking behavior의 + component family에서 Abaqus-only scale `S=max(abs(reference))`를 구한다. `abs(reference)`가 + `0.01*S` 이하인 행은 `abs(fesa-reference) <= 0.01*S`, 그 외 행은 상대오차 `<=0.05`로 + 판정하고, family scale-relative RMS `RMS(error)/S <= 0.01`도 함께 통과해야 한다. 별도 + absolute-error gate는 사용하지 않으며 zero-scale family는 FESA도 exact zero일 때만 + 통과한다. +- B33 section-resultant reference는 `(instance, element label, endpoint node label, + component)`로 HDF5 element endpoint와 직접 대응한다. Node-station collapse 또는 평균을 + 사용하지 않는다. MITC4 `U1/U2/U3`는 blocking, `UR1/UR2/UR3`는 warning-only 분류를 + 유지하면서 같은 공통 수치 정책을 사용한다. - CSV는 FESA 공식 output이 아니며, FESA HDF5에서 추출한 deterministic CSV view는 비교 디버깅/검토용 보조 artifact로만 둔다. ## FESA 개발의 핵심 원칙 diff --git a/docs/ADR.md b/docs/ADR.md index d07b85c..82238ca 100644 --- a/docs/ADR.md +++ b/docs/ADR.md @@ -129,6 +129,9 @@ linear-static hook 사이에 조건문으로 새 physics를 삽입하지 않는 ### ADR-014: B33 cantilever baseline은 component-scale 혼합 tolerance로 비교한다 +**상태**: Artifact inventory와 component mapping의 역사적 결정은 유지하며 tolerance와 +node-station projection 결정은 ADR-022로 대체됨. + **결정**: `reference/cantilever beam/`의 B33 input, displacement CSV, reaction CSV, elemental-force CSV를 V0 reference baseline으로 사용한다. Displacement와 reaction은 node identity로, section resultant는 CSV node station과 정규화된 FESA element endpoint로 비교한다. 같은 model, step/frame, quantity, component에 대해 `reference_scale = max(abs(reference rows))`를 계산하고 모든 row에 `absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation absolute floor는 `1e-9`, force/moment floor는 `1e-3`이다. Beam stress는 출력하되 Abaqus stress reference comparison은 N/A다. **이유**: 자유단 moment처럼 이론적으로 0인 값에는 행별 상대오차가 정의되지 않으며 Abaqus 결과에 작은 수치 잔차가 남을 수 있다. Component scale을 사용하면 전체 물리량 크기에 비해 작은 잔차를 허용하면서 각 row를 결정적으로 판정할 수 있다. @@ -232,6 +235,9 @@ false match와 결과 보정을 방지한다. 더 강한 provenance가 필요한 ### ADR-020: MITC4 displacement reference는 고정 절대오차로 판정한다 +**상태**: S4 artifact 선택과 U blocking/UR warning-only 분류는 유지하며 고정 절대오차 +결정은 ADR-022로 대체됨. + **결정**: Full-integration FESA-MITC4의 sole S4 reference comparison은 matched global `U1/U2/U3` row에 고정 절대오차 `1.0e-5`를 적용해 pass/fail을 판정한다. `UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하지만 초과는 deterministic warning만 @@ -306,3 +312,37 @@ formulation, 연산·reduction 순서, sign, units, coordinates와 result identi 리팩터링보다 우선하며 HDF5 schema, reference artifact와 ADR-014/ADR-020 tolerance는 변경하지 않는다. MITC3, solid, dynamic과 plastic behavior는 별도 feature gate 전까지 구현된 것으로 간주하지 않는다. + +### ADR-022: 외부 reference comparison은 공통 family-scale 정책을 사용한다 + +운영 상수, 검증 순서, report schema와 변경 관리는 `docs/TOLERANCE.md`를 따른다. + +**결정**: B33, MITC4와 이후 기능의 외부 reference comparison은 동일한 무차원 수치 +정책을 사용한다. 같은 model/case, step/frame, logical quantity, unit dimension, coordinate +system과 blocking behavior의 component family에 대해 reference-only scale +`S=max(abs(reference))`를 계산한다. `abs(reference)<=0.01*S`인 행은 +`abs(fesa-reference)<=0.01*S`, 나머지 행은 `abs(fesa-reference)/abs(reference)<=0.05`를 +만족해야 한다. 모든 행과 함께 family scale-relative RMS `RMS(error)/S<=0.01`도 +통과해야 한다. 독립 absolute-error gate는 두지 않으며 `S=0`이면 모든 FESA 값도 exact +zero일 때만 통과한다. + +B33 family는 translation, rotation, reaction force, reaction moment, section force와 +section moment로 구분한다. 2026-08-18에 다시 생성된 elemental-force CSV의 +`Element Label`과 `Node Label`을 사용해 `(instance, element, endpoint node, component)`를 +HDF5 endpoint에 직접 대응하고 node-station collapse나 평균을 사용하지 않는다. MITC4는 +translation `U1/U2/U3` family를 blocking으로, rotation `UR1/UR2/UR3` family를 +warning-only로 유지한다. + +**이유**: Feature별 absolute floor와 fixed absolute tolerance는 모델 단위와 크기에 따라 +서로 다른 엄격도를 만들었다. 개별 component scale은 물리적으로 zero-like인 MITC4 +`U1/U2` residue를 자체 scale로 만들어 잘못 실패시킨다. 동일 차원의 family scale, +near-zero 대체 분기와 relative RMS를 결합하면 zero row의 불안정한 raw relative error를 +피하면서 모든 행과 전체 오차 수준을 함께 검사할 수 있다. B33의 element-endpoint identity는 +reference와 HDF5가 제공하는 실제 source identity를 보존한다. + +**트레이드오프**: Family 최대값이 작은 component의 허용폭을 결정하므로 행별 relative +gate와 family RMS gate를 모두 유지해야 한다. Reference family 전체가 zero이면 별도 +절대 scale이 없으므로 exact-zero만 허용하는 엄격한 fail-closed 정책이 된다. 기존 B33과 +MITC4 verification report 및 comparator schema는 새 정책에 맞춰 다시 생성해야 하지만 +reference artifact, HDF5 schema, element formulation과 blocking/warning 분류는 변경하지 +않는다. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8cae88a..d2de1c8 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -419,14 +419,19 @@ Schema requirements: `[N,T,My,Mz]`, generalized strain/resultant는 두 Gauss point에 기록한다. - General beam section stress는 section point의 axial `S11`만 복구한다. Section point가 없으면 centroid `(0,0)`을 `source=fesa-default`로 기록한다. -- 승인된 `reference/cantilever beam/cantilever beam elemental forces.csv`는 node station - 기준 `SF1/SM1/SM2/SM3`을 제공한다. FESA endpoint를 동일한 section-cut 부호로 - 정규화하고 interior node의 두 endpoint가 tolerance 안에서 일치하는지 먼저 확인한 뒤 - `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`로 비교한다. -- Reference tolerance는 같은 model, step/frame, quantity, component의 Abaqus rows에서 - `reference_scale = max(abs(reference_value))`를 구하고 각 row에 - `absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation floor는 - `1e-9`, force/moment floor는 `1e-3`이다. +- 승인된 `reference/cantilever beam/cantilever beam elemental forces.csv`는 각 B33 + element의 두 endpoint에 `Element Label`, `Node Label`과 `SF1/SM1/SM2/SM3`을 제공한다. + `(instance, element label, endpoint node label, component)` identity로 HDF5 + `[element,endpoint,N/T/My/Mz]`에 직접 대응하고 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, + `SM3 -> T`로 비교한다. External reference comparison은 node-station collapse나 평균을 + 사용하지 않는다. +- Reference tolerance는 같은 model/case, step/frame, logical quantity, unit dimension, + coordinate system과 blocking behavior의 component family에서 Abaqus-only scale + `S=max(abs(reference))`를 구한다. `abs(reference)<=0.01*S`인 행은 absolute error + `<=0.01*S`, 그 외 행은 relative error `<=0.05`로 판정하며 family scale-relative RMS + `RMS(error)/S<=0.01`도 통과해야 한다. 독립 absolute-error gate는 사용하지 않고 + zero-scale family는 FESA도 exact zero일 때만 통과한다. 전체 판정 과정과 report/change + management contract는 `docs/TOLERANCE.md`를 따른다. - Beam stress는 HDF5 schema와 unit/analytical test로 검증하지만 Abaqus reference comparison은 N/A다. @@ -469,9 +474,8 @@ stabilization만 둔다. Drilling calibration, artificial-energy policy와 별 dataset은 이 기능 범위가 아니다. Full-integration FESA-MITC4의 reference comparison은 `reference/shell/` S4의 기존 input 및 -displacement CSV만 사용한다. Global `U1/U2/U3`만 blocking이고 모든 matched row에 고정 -절대오차 `1.0e-5`를 적용한다. `UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하되 -warning-only evidence다. MITC4 판정에는 component scale을 사용하지 않으며 B33의 기존 -혼합 tolerance는 변경하지 않는다. S4R은 +displacement CSV만 사용한다. Global `U1/U2/U3` translation family만 blocking이고 +`UR1/UR2/UR3` rotation family는 warning-only evidence다. 두 family 모두 ADR-022의 공통 +family-scale row/RMS 정책을 사용한다. S4R은 같은 kernel을 선택하는 source mapping과 metadata를 unit/integration tests로 검증하며 `reference/shellR/` artifact는 acceptance comparison에 포함하지 않는다. diff --git a/docs/PRD.md b/docs/PRD.md index 371131e..4eae466 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -113,7 +113,7 @@ Parser keyword, element kernel, solver backend 또는 output dataset 중 하나 | FESA-PRD-014 | 선형 정적 pipeline은 stiffness factorization과 load substitution을 분리해야 한다. | `Kff` factorization이 load vector assembly보다 먼저 수행되고, `rhs=Ff-Kfc*dc` substitution으로 full displacement를 복구한다. | orchestration test, solver-adapter test | | FESA-PRD-015 | FESA는 V0 결과와 diagnostic을 안정된 외부 계약으로 출력해야 한다. | CLI가 `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5` exit code를 사용하고 diagnostic field가 계약과 일치하며 HDF5가 displacement, reaction, end force, section resultant, generalized strain/resultant, axial `S11`을 포함하고 실패 시 불완전한 최종 파일을 남기지 않는다. | CLI integration test, HDF5 schema/atomicity test | | FESA-PRD-016 | Dense와 sparse math storage 및 backend 경계를 분리해야 한다. | `Vector`는 contiguous, `Matrix`는 row-major contiguous storage와 MKL CBLAS를 사용하고 `SparseMatrix`는 별도 0-based CSR 타입이며 MKL 타입이 public core API에 노출되지 않는다. | math unit test, dependency review | -| FESA-PRD-017 | B33 reference comparison은 component-scale 혼합 tolerance를 사용해야 한다. | 모든 matched row가 `abs_error <= absolute_floor + 1e-6 * reference_scale`을 만족하고 missing/extra/nonfinite row는 comparison 전에 실패한다. | reference comparison unit/integration test, verification report | +| FESA-PRD-017 | 모든 외부 reference comparison은 승인된 공통 family-scale tolerance를 사용해야 한다. | 각 matched row가 near-zero 대체 조건 또는 상대오차 `0.05`를 통과하고 각 family의 scale-relative RMS가 `0.01` 이하이며 missing/extra/duplicate/nonfinite row는 comparison 전에 실패한다. | reference comparison unit/integration test, verification report | | FESA-PRD-018 | 승인된 B33 reference artifact는 현재 경로의 read-only baseline으로 유지해야 한다. | `reference/cantilever beam/` 파일을 rename, rewrite 또는 보정하지 않고 기능이 선언한 exact path에서 읽는다. | artifact inventory, Git diff review | | FESA-PRD-019 | B33 beam section과 local axis를 Abaqus 의미에 맞게 매핑해야 한다. | `n1 -> local y`, `t x n1 -> local z`, `Iy=I11`, `Iz=I22`, `I12=0`을 적용하고 nonpositive property, zero-length element, tangent-parallel guide vector를 구조화된 model diagnostic으로 거부한다. | section-mapping unit test, element geometry test | @@ -121,12 +121,12 @@ Parser keyword, element kernel, solver backend 또는 output dataset 중 하나 - MSVC x64 Debug 환경에서 configure, build, CTest를 검증한다. - reference test 결과는 deterministic해야 한다. - HDF5 schema는 versioned contract로 관리한다. -- B33 reference tolerance는 model, step/frame, quantity, component별 Abaqus scale만 사용한다. -- 승인된 SI B33 bundle의 absolute floor는 displacement/rotation `1e-9`, force/moment - `1e-3`이며 relative coefficient는 `1e-6`이다. -- MITC4 U/UR comparison은 고정 절대오차 `1.0e-5`를 사용한다. `U1/U2/U3`만 - blocking이고 `UR1/UR2/UR3` 초과는 warning-only다. B33의 component-scale 혼합 - tolerance는 별도 기존 계약으로 유지한다. +- 모든 외부 reference comparison은 동일 logical quantity와 unit dimension의 component + family별 Abaqus-only maximum scale을 사용한다. Near-zero ratio는 `0.01`, 일반 행 + 상대오차는 `0.05`, family scale-relative RMS는 `0.01`이며 독립 absolute-error gate는 + 사용하지 않는다. +- B33 section resultant는 source element label과 endpoint node label로 직접 대응한다. + MITC4 `U1/U2/U3`만 blocking이고 `UR1/UR2/UR3` 초과는 warning-only다. - parser, solver, HDF5 writer는 실패 원인을 구조화된 diagnostic으로 보고한다. - oneMKL, oneTBB, HDF5는 CMake에서 명시 탐지하고 실패 원인을 분류한다. - 대규모 모델 성능 최적화보다 V0의 명확성, 테스트 가능성, 검증 traceability를 우선한다. diff --git a/docs/SOLVER_AGENT_DESIGN.md b/docs/SOLVER_AGENT_DESIGN.md index e72ce98..e1a24a6 100644 --- a/docs/SOLVER_AGENT_DESIGN.md +++ b/docs/SOLVER_AGENT_DESIGN.md @@ -148,7 +148,8 @@ gap은 Coordinator가 해당 owner 단계로 되돌린다. Reference comparison은 source identity와 component로 행을 결정적으로 대응시키며 missing, extra, duplicate 또는 nonfinite required row를 tolerance 전에 거부한다. CSV는 외부 reference이고 -FESA의 authoritative output은 `results.h5`다. +FESA의 authoritative output은 `results.h5`다. 공통 tolerance 값, family 구성, row/RMS +판정, report evidence와 변경 관리는 `docs/TOLERANCE.md`를 따른다. ## 요구사항 단위 산출물 구조 diff --git a/docs/TOLERANCE.md b/docs/TOLERANCE.md new file mode 100644 index 0000000..4fe2f1d --- /dev/null +++ b/docs/TOLERANCE.md @@ -0,0 +1,431 @@ +# FESA Reference Tolerance Policy + +## 문서 정보 + +- policy_id: `common-reference-tolerance` +- status: `approved-and-implemented` +- effective_date: `2026-08-18` +- scope: Abaqus CSV와 FESA HDF5 사이의 외부 reference comparison +- decision_record: `docs/ADR.md`의 ADR-022 +- design_record: `docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md` + +## 목적 + +이 문서는 FESA의 외부 reference tolerance를 관리하는 운영 기준이다. 공통 tolerance +상수뿐 아니라 row identity 사전검사, family 구성, 행별 판정, relative RMS, 최종 verdict, +보고서 evidence와 변경 절차를 하나의 source of truth로 제공한다. + +이 정책은 Abaqus와 FESA의 내부 알고리즘 동등성을 판단하지 않는다. 승인된 같은 입력 +모델에서 나온 외부 수치 결과가 합의된 오차 범위 안에 있는지만 검증한다. + +## 적용 범위 + +### 적용 대상 + +- Abaqus reference CSV와 FESA `results.h5`의 deterministic comparison +- B33 displacement, rotation, reaction, section resultant +- MITC4 displacement 및 rotation +- 이후 requirements에서 family와 row identity가 승인된 reference quantity + +### 적용하지 않는 대상 + +다음 검증은 이 문서의 `0.01/0.05/0.01` 값을 사용하지 않는다. 각 feature의 formulation, +requirements, numerical-review 또는 physics contract가 별도 tolerance를 정한다. + +- element stiffness symmetry, rank, rigid-body mode +- analytical solution, patch test와 mesh convergence +- linear-system residual과 global equilibrium +- energy, stress/strain sanity +- geometry, Jacobian과 director validation +- parser, HDF5 schema와 exact identity 검증 + +Schema와 identity 검증은 수치 tolerance보다 먼저 수행하며 tolerance로 완화할 수 없다. + +## 용어 + +| 용어 | 정의 | +| --- | --- | +| FESA 값 \(f_i\) | Authoritative `results.h5`에서 deterministic projection으로 얻은 값 | +| Reference 값 \(r_i\) | 승인된 read-only Abaqus CSV에서 얻은 값 | +| Matched row | Model/case, step/frame, source identity, quantity와 component가 정확히 대응된 한 행 | +| Family | 같은 logical quantity, unit dimension, coordinate system과 blocking behavior를 공유하는 component 집합 | +| Reference scale \(S\) | Family reference 값만으로 계산한 \(\max_i |r_i|\) | +| Near-zero band \(B\) | \(0.01S\) | +| Blocking family | 행 또는 RMS가 실패하면 전체 reference gate를 실패시키는 family | +| Warning-only family | 동일한 실패를 warning으로 기록하지만 전체 gate를 실패시키지 않는 family | + +## 공통 상수 + +모든 외부 reference comparison은 다음 세 상수를 사용한다. + +```text +near_zero_ratio = 0.01 +relative_tolerance = 0.05 +relative_rms_tolerance = 0.01 +``` + +- `near_zero_ratio`: family 최대 reference 크기의 1% 이하를 near-zero row로 분류 +- `relative_tolerance`: 일반 row의 최대 상대오차 5% +- `relative_rms_tolerance`: family error RMS가 reference scale의 1% 이하 + +독립적인 absolute-error gate는 사용하지 않는다. Maximum absolute error는 진단 정보이며 +그 자체로 pass/fail을 결정하지 않는다. + +## Comparison family + +Family는 다음 identity를 모두 포함한다. + +```text +model_or_case +step_and_frame +logical_quantity +unit_dimension +coordinate_system +blocking_behavior +``` + +서로 다른 물리 차원, 좌표계 또는 blocking behavior는 같은 scale을 공유하지 않는다. +Component가 작다는 이유만으로 별도 family로 분리하지 않는다. + +### 현재 승인된 family + +| Feature | Family | Components | Dimension | Coordinates | Behavior | +| --- | --- | --- | --- | --- | --- | +| B33 | displacement-translation | `UX, UY, UZ` | length | global Cartesian | blocking | +| B33 | displacement-rotation | `URX, URY, URZ` | rotation | global Cartesian | blocking | +| B33 | reaction-force | `RF1, RF2, RF3` | force | global Cartesian | blocking | +| B33 | reaction-moment | `RM1, RM2, RM3` | force × length | global Cartesian | blocking | +| B33 | section-force | `N` | force | beam local | blocking | +| B33 | section-moment | `T, My, Mz` | force × length | beam local | blocking | +| MITC4 | displacement-translation | `U1, U2, U3` | length | global Cartesian | blocking | +| MITC4 | displacement-rotation | `UR1, UR2, UR3` | rotation | global Cartesian | warning-only | + +## 전체 검증 과정 + +```mermaid +flowchart TD + A[승인 input과 read-only reference CSV 확인] --> B[FESA로 같은 input 해석] + B --> C[authoritative results.h5 생성] + C --> D[CSV와 HDF5를 canonical row로 projection] + D --> E{row identity, schema, finite precheck} + E -- fail --> X[수치 비교 전 hard failure] + E -- pass --> F[comparison family 구성] + F --> G[reference-only scale S와 near-zero band 계산] + G --> H[모든 row에 near-zero 또는 relative gate 적용] + H --> I[family scale-relative RMS 계산] + I --> J[blocking 또는 warning-only behavior 적용] + J --> K[deterministic JSON과 report 기록] +``` + +### 1. Reference artifact 확인 + +기능별 `reference-model.md`와 `io.md`가 지정한 정확한 input/CSV 경로만 사용한다. +Comparator와 agent는 reference artifact를 rename, rewrite, 보정 또는 zero-clamp하지 않는다. +Metadata, README 또는 canonical filename은 feature requirement가 수치 계약으로 명시하지 +않는 한 readiness 조건이 아니다. + +### 2. FESA 결과 생성 + +Reference와 같은 `.inp`를 FESA로 해석하고 authoritative `results.h5`를 생성한다. +CSV로 다시 내보낸 FESA 값은 디버깅용 view일 뿐 authoritative comparison source가 아니다. + +### 3. Canonical row projection + +CSV와 HDF5 값을 기능 계약의 stable source identity와 component로 canonical row에 투영한다. +Internal index 또는 equation index를 source identity 대신 사용하지 않는다. + +- B33 displacement/reaction: + `(model, Step-1, frame 0, instance, source node label, quantity, component)` +- B33 section resultant: + `(model, Step-1, frame 0, instance, source element label, endpoint source node label, + quantity, component)` +- MITC4 displacement/rotation: + `(case, Step-1, frame 0, instance, source node label, quantity, component)` + +B33 section mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`다. +Element endpoint를 node station으로 collapse하거나 대표 endpoint를 선택하거나 평균하지 +않는다. + +### 4. 수치 비교 전 hard precheck + +다음 중 하나라도 발견되면 tolerance 계산을 시작하지 않고 comparison을 실패시킨다. + +- required file 또는 dataset 누락 +- CSV/HDF5 header, shape, component inventory 불일치 +- missing 또는 extra canonical row +- duplicate row identity +- source instance/node/element/endpoint identity 불일치 +- element endpoint와 input connectivity 불일치 +- nonfinite FESA/reference 값 또는 계산 중 nonfinite error + +이 오류는 warning-only family에 속하더라도 hard failure다. Tolerance는 schema나 identity +문제를 숨기는 수단이 아니다. + +### 5. Reference scale과 near-zero band + +각 family \(g\)에 대해 read-only reference 값만 사용한다. + +\[ +S_g = \max_i |r_i| +\] + +\[ +B_g = 0.01S_g +\] + +FESA 값을 scale 계산에 사용하지 않는다. `max(1, ...)`, 임의 floor, unit별 absolute +tolerance 또는 reference zero clamp를 추가하지 않는다. + +### 6. 행별 판정 + +먼저 absolute error를 계산한다. + +\[ +e_i = |f_i-r_i| +\] + +#### Near-zero row + +\[ +|r_i| \le B_g +\] + +이면 다음을 만족할 때 통과한다. + +\[ +e_i \le B_g +\] + +#### 일반 relative row + +\[ +|r_i| > B_g +\] + +이면 다음을 만족할 때 통과한다. + +\[ +\frac{e_i}{|r_i|} \le 0.05 +\] + +Near-zero 분류와 두 threshold는 경계값을 포함한다. 모든 matched row를 판정하며 작은 +reference 값이나 zero row를 삭제하지 않는다. + +### 7. Zero-scale family + +\(S_g=0\)이면 family의 모든 reference 값이 정확히 0이다. 독립 absolute floor가 없으므로 +다음 fail-closed 규칙을 적용한다. + +- 모든 FESA 값도 정확히 0이면 모든 row와 family RMS가 통과한다. +- 하나라도 0이 아닌 FESA 값이 있으면 해당 row와 family가 실패한다. +- Report에는 NaN/Inf 대신 `zero-reference-scale-nonzero-error`를 기록한다. + +### 8. Family scale-relative RMS + +Family의 모든 absolute error로 error RMS를 계산하고 같은 reference scale로 정규화한다. + +\[ +\operatorname{relative\_rms}_g = +\frac{\sqrt{\frac{1}{n_g}\sum_i e_i^2}}{S_g} +\] + +\[ +\operatorname{relative\_rms}_g \le 0.01 +\] + +이 값은 row-relative error들의 RMS가 아니다. Family의 모든 row gate와 RMS gate가 모두 +통과해야 family가 통과한다. + +### 9. 최종 verdict + +| 상태 | 전체 reference verdict | +| --- | --- | +| Precheck failure | 항상 실패 | +| Blocking family row failure | 실패 | +| Blocking family RMS failure | 실패 | +| Warning-only family row failure | 전체 verdict 유지, row warning 생성 | +| Warning-only family RMS failure | 전체 verdict 유지, RMS warning 생성 | +| Warning-only row와 RMS 동시 실패 | 두 종류의 deterministic warning을 모두 생성 | +| 모든 blocking family 통과 | Reference gate 통과 | + +Warning-only family도 동일한 수치 규칙과 report schema를 사용한다. 차이는 전체 verdict에 +미치는 영향뿐이다. + +### 10. Evidence report + +Report는 사람이 원본 값에서 판정을 재현할 수 있어야 한다. + +#### Row별 필수 항목 + +- model/case와 feature contract가 정한 step/frame +- instance와 source node/element/endpoint identity +- quantity와 component +- FESA 값, reference 값, absolute error +- reference scale과 near-zero band +- 적용 branch: `near-zero`, `relative` 또는 `zero-scale-exact` +- 적용 threshold와, 해당되는 경우 row relative error +- blocking 여부와 row pass/fail + +Units와 coordinate system은 family와 feature reference/I/O contract에 반드시 정의한다. +Row JSON이 이를 직접 반복하지 않더라도 report가 해당 contract와 family identity를 +결정적으로 가리켜야 한다. + +#### Family별 필수 항목 + +- family identity와 component 목록 +- total `row_count`와 `near_zero_count` +- reference scale과 near-zero band +- maximum absolute error와 deterministic worst row +- scale-relative RMS와 RMS pass/fail +- family pass/fail과 diagnostic code + +#### 전체 필수 항목 + +- missing/extra/duplicate/nonfinite precheck 결과 +- blocking failure와 warning-only warning 목록 +- overall `passed` verdict +- 사용한 input/reference/HDF5 identity + +Report에는 nonfinite metric을 기록하지 않는다. Historical report의 수치나 판정을 새 +정책에 맞춰 소급 수정하지 않고 새 comparison evidence를 별도 생성한다. + +## 계산 예시 + +### 일반 relative row + +Family scale이 \(S=100\)이면 near-zero band는 \(B=1\)이다. + +- \(r=50\), \(f=52.4\): 상대오차 \(2.4/50=0.048\), 통과 +- \(r=50\), \(f=52.6\): 상대오차 \(2.6/50=0.052\), 실패 + +### Near-zero row + +같은 family에서 \(S=100\), \(B=1\)일 때: + +- \(r=0.2\), \(e=0.9\): \(0.9 \le 1\), 통과 +- \(r=0.2\), \(e=1.1\): \(1.1 > 1\), 실패 + +### RMS gate + +네 reference 값이 모두 \(100\)이고 \(S=100\)일 때, 각 error가 \(0.8\)이면 relative +RMS는 \(0.008\)로 통과한다. 각 error가 \(1.1\)이면 각 row의 상대오차는 \(0.011\)로 +통과하지만 relative RMS도 \(0.011\)이므로 family는 실패한다. + +### Zero-scale family + +- Reference `[0,0,0]`, FESA `[0,0,0]` → 통과 +- Reference `[0,0,0]`, FESA `[0,1e-12,0]` → exact-zero 규칙 실패, + `zero-reference-scale-nonzero-error` + +## 구현 위치와 검증 진입점 + +| 책임 | 위치 | +| --- | --- | +| 공통 evaluator | `tests/reference/reference_tolerance_policy.h/.cpp` | +| 공통 boundary tests | `tests/reference/reference_tolerance_policy_test.cpp` | +| B33 comparator와 JSON | `tests/reference/reference_comparison.h/.cpp` | +| B33 tests | `tests/reference/reference_comparison_test.cpp`, `tests/reference/b33_reference_comparison_test.cpp` | +| MITC4 comparator와 JSON | `tests/reference/mitc4_reference_comparison.h/.cpp` | +| MITC4 tests | `tests/reference/mitc4_reference_comparison_test.cpp`, `tests/reference/mitc4_reference_cases_test.cpp` | + +Focused verification: + +```powershell +cmake --build .harness/build --config Debug --target fesa_reference_tests +ctest --test-dir .harness/build -C Debug -R 'ReferenceTolerancePolicy|ReferenceComparisonContract|B33ReferenceComparison|Mitc4ReferenceComparison|Mitc4S4Reference' --output-on-failure +``` + +전체 verification: + +```powershell +cmake --build .harness/build --config Debug +ctest --test-dir .harness/build -C Debug --show-only=json-v1 +ctest --test-dir .harness/build -C Debug --output-on-failure +``` + +Generated JSON ledger는 `.harness/build/reference/` 아래에 두며 Git에 commit하지 않는다. +Feature별 current evidence는 해당 `docs//reference-comparison.md`에 기록한다. + +## 규범 요구조건 + +- **FESA-TOL-001** — Comparator shall use only the exact feature-approved read-only reference + artifacts and authoritative FESA HDF5 values. +- **FESA-TOL-002** — Comparator shall reject missing, extra, duplicate, nonfinite, schema- or + identity-mismatched rows before numerical tolerance evaluation. +- **FESA-TOL-003** — Comparator shall group matched rows only by the approved family identity, + unit dimension, coordinate system and blocking behavior. +- **FESA-TOL-004** — Every nonzero-scale family shall use near-zero ratio `0.01`, row relative + tolerance `0.05` and scale-relative RMS tolerance `0.01`. +- **FESA-TOL-005** — A zero-scale family shall pass only when every FESA value is exactly zero and + shall otherwise report `zero-reference-scale-nonzero-error` without NaN/Inf. +- **FESA-TOL-006** — Every matched row and family RMS shall be evaluated; no row shall be clamped, + omitted, averaged or hidden by an independent absolute-error gate. +- **FESA-TOL-007** — Blocking-family failure shall fail the reference gate; warning-only failure + shall emit deterministic row/RMS warnings without changing the blocking verdict. +- **FESA-TOL-008** — The report shall contain every row decision, family row count, scale, + near-zero inventory, worst error, relative RMS, diagnostic and overall verdict. +- **FESA-TOL-009** — Repeated comparison of the same artifacts and HDF5 shall preserve row, + family, diagnostic and JSON ordering. +- **FESA-TOL-010** — Tolerance constants, family membership, row identity or blocking behavior + shall not change without explicit approval and fresh comparison evidence. + +## Requirement Verification Matrix + +| ID | Verification method | Acceptance criteria | Evidence | +| --- | --- | --- | --- | +| FESA-TOL-001 | Reference integration tests and run-local before/after content audit | Exact declared paths used; comparison run이 reference를 변경하지 않음 | Feature reference tests; run-local hash/diff evidence | +| FESA-TOL-002 | Missing/extra/duplicate/nonfinite/identity negative tests | Deterministic failure before row metrics | Comparator contract tests | +| FESA-TOL-003 | Family membership and component tests | Exact approved components, units, coordinates and behavior | JSON family metrics | +| FESA-TOL-004 | Inclusive boundary, near-zero and RMS tests | `0.01/0.05/0.01` gates match this document | Common evaluator tests | +| FESA-TOL-005 | All-zero and one-nonzero zero-scale tests | Finite metrics and exact diagnostic | Common evaluator and JSON tests | +| FESA-TOL-006 | Row inventory and no-clamp/no-average tests | Every canonical row has one decision | Comparator row-count/identity tests | +| FESA-TOL-007 | Blocking, row-warning, RMS-warning and combined-warning tests | Correct verdict and deterministic warnings | MITC4 comparator tests | +| FESA-TOL-008 | JSON schema assertions | Required row/family/overall fields present and finite | B33/MITC4 deterministic JSON tests | +| FESA-TOL-009 | Repeated writer/comparison tests | Byte-stable ordering for the same inputs | Deterministic report tests | +| FESA-TOL-010 | ADR/doc/diff/revalidation review | All affected contracts and evidence updated together | ADR-022, feature reports, review record | + +## 새 reference quantity 추가 절차 + +새 기능이나 quantity는 comparator code부터 추가하지 않는다. + +1. Feature requirements에 exact input/CSV paths와 blocking/warning-only quantity를 정의한다. +2. Units, coordinate system, source row identity, component mapping과 result location을 정의한다. +3. Existing family에 포함할지 새 family를 만들지 dimension/behavior 기준으로 결정한다. +4. Numerical review와 reference-model에서 row precheck와 readiness를 승인한다. +5. I/O 문서에서 CSV column을 authoritative HDF5 dataset/component에 연결한다. +6. RED test로 identity, family, boundary, zero-scale, RMS와 report contract를 고정한다. +7. Comparator를 구현하고 focused/full MSVC CTest를 통과한다. +8. Fresh reference-comparison report를 생성하고 이후 physics/release gate를 진행한다. + +## Tolerance 변경 관리 + +Tolerance 상수, 수식, family membership, row identity 또는 blocking behavior를 변경하려면 +다음 항목을 함께 처리한다. + +1. 사용자 명시 승인 +2. 이 문서와 ADR의 decision, rationale과 tradeoff 갱신 +3. Common design 및 영향받는 feature requirements/reference-model/I/O 문서 갱신 +4. Boundary와 failure semantics를 먼저 고정한 RED test +5. 공통 evaluator 및 영향받는 comparator의 최소 수정 +6. Focused reference tests와 전체 MSVC/CTest +7. 모든 영향 feature의 fresh reference comparison +8. 기존 physics/release evidence의 유효성 재검토 + +과거 report의 원본 hash, command, row count와 metric table은 소급 수정하지 않는다. +Historical status를 보존하고 새 정책으로 생성한 evidence를 별도 섹션 또는 새 report에 +추가한다. Reference CSV는 tolerance 변경을 맞추기 위해 수정하지 않는다. + +## 검토 체크리스트 + +- [ ] Exact reference input/CSV와 authoritative HDF5를 사용했는가? +- [ ] Canonical row identity와 component mapping이 기능 계약과 일치하는가? +- [ ] Missing/extra/duplicate/nonfinite row를 tolerance 전에 거부했는가? +- [ ] Family가 units, coordinates와 blocking behavior를 섞지 않는가? +- [ ] Scale을 reference 값만으로 계산했는가? +- [ ] 모든 row에 near-zero 또는 relative branch가 기록됐는가? +- [ ] Zero-scale에서 exact-zero와 finite diagnostic을 보장하는가? +- [ ] 모든 family의 scale-relative RMS를 판정했는가? +- [ ] Warning-only row/RMS failure를 deterministic warning으로 기록했는가? +- [ ] Report에 row count, scale, worst row, RMS와 overall verdict가 있는가? +- [ ] Reference artifact와 historical evidence를 변경하지 않았는가? +- [ ] Focused 및 full verification evidence가 있는가? diff --git a/docs/linear-static-3d-euler-beam/formulation.md b/docs/linear-static-3d-euler-beam/formulation.md index 4ad7c8b..dbe4b7b 100644 --- a/docs/linear-static-3d-euler-beam/formulation.md +++ b/docs/linear-static-3d-euler-beam/formulation.md @@ -1311,9 +1311,10 @@ $$ 가 일치해야 한다. 따라서 이 V0 조건의 equilibrium end action은 요소 끝에 작용하는 outward action이고, endpoint section resultant는 동일 section을 양의 local $x$ 면으로 -자른 값이다. Abaqus node-station force row에는 equilibrium end action이 아니라 section -resultant를 동일한 section-cut 부호로 정규화해 비교하며, interior node의 두 endpoint가 -승인 tolerance 안에서 먼저 일치해야 한다. 승인된 component mapping은 +자른 값이다. Abaqus element-endpoint force row에는 equilibrium end action이 아니라 +section resultant를 동일한 section-cut 부호로 정규화해 비교한다. Reference의 element +label과 endpoint node label을 HDF5 row에 직접 대응하며 node station collapse, 대표 endpoint +선택 또는 평균은 하지 않는다. 승인된 component mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`이고, CSV에 없는 transverse `SF2`, `SF3`는 reference comparison 대상이 아니다. @@ -1529,19 +1530,25 @@ $$ \sum\mathbf M_{\mathrm{reaction}}\right\|\le\epsilon_M. $$ -승인된 B33 reference row는 같은 model, step/frame, quantity, component별로 Abaqus -reference 값만 사용해 +승인된 B33 reference row는 같은 model, step/frame, quantity, component로 family를 만들고 +Abaqus reference 값만 사용해 $$ -\operatorname{reference\_scale}=\max_i|r_i|,\qquad -\operatorname{row\_tolerance}=\operatorname{absolute\_floor} -+10^{-6}\operatorname{reference\_scale} +S=\max_i|r_i| $$ -를 적용한다. 승인된 SI bundle의 displacement/rotation floor는 $10^{-9}$, -force/moment floor는 $10^{-3}$이다. Reference 값을 zero-clamp하거나 row를 제거하지 -않으며 missing, extra, duplicate, nonfinite, schema/identity mismatch는 수치 판정 전에 -실패한다. 이 reference policy는 위 formulation/analytical tolerance를 대체하지 않는다. +를 계산한다. $|r_i|\le0.01S$인 near-zero row는 $|f_i-r_i|\le0.01S$로, +나머지 row는 $|f_i-r_i|/|r_i|\le0.05$로 판정한다. 또한 + +$$ +\frac{\sqrt{n^{-1}\sum_i(f_i-r_i)^2}}{S}\le0.01 +$$ + +을 family 전체에서 만족해야 한다. $S=0$이면 모든 FESA 값이 정확히 0일 때만 통과하고, +그렇지 않으면 zero-reference-scale-nonzero-error로 실패한다. 독립적인 절대오차 gate는 +사용하지 않는다. Reference 값을 zero-clamp하거나 row를 제거하지 않으며 missing, extra, +duplicate, nonfinite, schema/identity mismatch는 수치 판정 전에 실패한다. 이 reference +policy는 위 formulation/analytical tolerance를 대체하지 않는다. ## 19. Numerical Risks diff --git a/docs/linear-static-3d-euler-beam/io.md b/docs/linear-static-3d-euler-beam/io.md index 71a0bbc..de05461 100644 --- a/docs/linear-static-3d-euler-beam/io.md +++ b/docs/linear-static-3d-euler-beam/io.md @@ -12,7 +12,7 @@ - owner_agent: `io-definition-agent` - date: `2026-08-09` - authoritative_output: `results.h5` -- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` +- reference_baseline: exact read-only artifacts under `reference/cantilever beam/` 이 문서는 승인된 V0의 semantic I/O contract만 정의한다. Parser, HDF5 writer, comparison tooling의 C++ API나 구현 구조는 정의하지 않으며 Abaqus full compatibility를 @@ -336,7 +336,7 @@ CSV는 `SF1/SM1/SM2/SM3`만 제공하고 section-cut result와 비교하므로 ` 0으로 만들거나 reference row로 합성하지 않는다. HDF5의 transverse end action은 unit/analytical test와 physics sanity 대상이다. -### Frame, instance, and node-station normalization +### Frame, instance, and element-endpoint normalization - Legacy `Frame` value `Increment 1: Step Time = 1.000`은 canonical `(Step-1, frame 0)`으로 변환한다. 승인 bundle의 다른 increment/time string은 `schema-mismatch`다. @@ -344,29 +344,27 @@ unit/analytical test와 physics sanity 대상이다. `instance_name`과 case-insensitive lookup 후 raw identity 일치 여부를 확인한다. - `Node Label`은 instance 안의 preserved source node label로 resolve한다. Displacement와 reaction key는 `(model_id,Step-1,0,instance_name,source_node_label,quantity,component)`다. -- Elemental-force CSV는 element label이 없으므로 HDF5 endpoint section resultants를 - source node station으로 project한다. Boundary station은 유일한 incident endpoint를 쓴다. -- Interior station collapse는 정확히 두 incident B33 endpoints, 동일 section/local-axis - orientation, 일관된 chain connectivity, 해당 node의 concentrated force/moment가 없는 - 경우에만 허용한다. 두 positive-face section-cut 값이 아래 승인 component tolerance - 안에서 먼저 일치해야 한다. -- Interior 값이 일치하면 stable internal element ID가 작은 endpoint를 deterministic - representative로 선택한다. 두 값을 평균하지 않는다. 불일치는 `tolerance-failure`다. -- Reversed connectivity, local-axis discontinuity, section jump, branch 또는 loaded interior - station은 element label 없는 legacy schema로 collapse할 수 없다. 이 approved bundle - 밖에서는 element-aware reference row가 필요하며 legacy projection은 `schema-mismatch`로 - 중단한다. +- Elemental-force CSV의 `Element Label`은 instance 안의 preserved source element label로, + `Node Label`은 해당 B33 connectivity의 정확한 endpoint source node로 resolve한다. +- Section-resultant key는 + `(model_id,Step-1,0,instance_name,source_element_label,source_node_label,quantity,component)`다. + 한 element에는 connectivity와 일치하는 두 endpoint 행이 정확히 존재해야 한다. +- Missing, extra, duplicate 또는 element-connectivity-mismatched endpoint row는 + `schema-mismatch`로 tolerance 전에 실패한다. Source node station collapse, + deterministic representative 선택 또는 endpoint 평균은 사용하지 않는다. Projected canonical comparison row는 다음 fields를 가진다. ```text -model_id, step_name, frame_index, instance_name, source_node_label, -quantity, component, value, unit_dimension, coordinate_system, hdf5_dataset_path +model_id, step_name, frame_index, instance_name, +[source_element_label], source_node_label, quantity, component, value, +unit_dimension, coordinate_system, hdf5_dataset_path ``` -Stable ordering은 quantity inventory order, instance declaration order, stable source node -order, component order다. Approved model ID는 `cantilever-beam-b33`; nodal quantities는 -global Cartesian, section resultants는 beam local이다. +`source_element_label`은 section-resultant row에 필수이고 nodal row에는 적용하지 않는다. +Stable ordering은 quantity inventory order, instance declaration order, stable source +node/element-endpoint order, component order다. Approved model ID는 `cantilever-beam-b33`; +nodal quantities는 global Cartesian, section resultants는 beam local이다. ### Row-set precheck and tolerance @@ -376,27 +374,32 @@ identity-mismatched row가 하나라도 있으면 tolerance 계산 전에 실패 mandatory인 `SF2/SF3` 대응 end action, generalized results와 `S11`은 계약상 비교 대상이 아니므로 extra reference row가 아니다. -Matched rows는 같은 `model_id`, step/frame, quantity, component로 group한다. +Matched rows는 같은 model/case, step/frame, logical quantity, unit dimension, coordinate +system과 blocking behavior의 component family로 group한다. B33 family는 translation +`UX/UY/UZ`, rotation `URX/URY/URZ`, reaction force `RF1/RF2/RF3`, reaction moment +`RM1/RM2/RM3`, section force `N`, section moment `T/My/Mz`다. ```text -reference_scale = max(abs(reference_value_i)) -row_tolerance = absolute_floor + 1e-6 * reference_scale -row_pass = abs(fesa_value_i - reference_value_i) <= row_tolerance +S = max(abs(reference_value_i)) +error_i = abs(fesa_value_i-reference_value_i) +near_zero_band = 0.01*S + +if abs(reference_value_i) <= near_zero_band: + row_pass = error_i <= near_zero_band +else: + row_pass = error_i/abs(reference_value_i) <= 0.05 + +relative_rms = sqrt(mean(error_i^2))/S +family_pass = all(row_pass) and relative_rms <= 0.01 ``` -즉 exact policy는 `absolute_floor + 1e-6 * reference_scale`이다. `reference_scale`은 -read-only Abaqus values만 사용하고 FESA 값으로 조정하지 않는다. Scale이 zero면 relative -term은 zero다. Reference value나 작은 residue를 zero-clamp하지 않고 모든 row를 판정한다. - -| approved SI component class | absolute floor | -| --- | ---: | -| displacement and rotation | `1e-9` | -| force and moment | `1e-3` | - -Interior endpoint consistency도 해당 CSV quantity/component의 same Abaqus-only scale과 -floor를 사용한다. Verification report는 모든 row pass/fail과 quantity별 max absolute -error, component-scale normalized error, RMS error, norm error, worst row/component를 -기록한다. 이 reference tolerance는 analytical/formulation tolerance를 대체하지 않는다. +`S`는 read-only Abaqus values만 사용하고 FESA 값으로 조정하지 않는다. 독립적인 +absolute-error gate와 zero clamp는 사용하지 않는다. `S=0`이면 모든 FESA 값도 exact +zero일 때 relative RMS를 0으로 기록하고 통과하며, 하나라도 nonzero이면 NaN/Inf 대신 +`zero-reference-scale-nonzero-error`로 실패한다. Verification report는 모든 row의 적용 +branch와 pass/fail, family identity/components, scale, near-zero band/count, max absolute +error, scale-relative RMS와 worst row/component를 기록한다. 이 reference tolerance는 +analytical/formulation tolerance를 대체하지 않는다. ## CLI and Diagnostics Contract @@ -445,17 +448,15 @@ release approval을 이 status가 의미하지 않는다. ### Resolved numerical-review handoff -`NR-O03-STATION-NORMALIZATION`은 approved legacy bundle에 대해 unloaded, consistently -oriented two-endpoint interior station만 collapse하고, tolerance check 후 smaller stable -element ID를 선택하는 규칙으로 구체화했다. Reversed/branched/loaded/jumped station은 -element-aware reference가 없는 한 비교하지 않는다. +`NR-O03-STATION-NORMALIZATION`은 2026-08-18 regenerated elemental-force CSV의 +`Element Label`로 해소됐다. Comparator는 `(instance, element label, endpoint node label, +component)`를 직접 대응하며 station collapse, 대표 endpoint 선택 또는 평균을 하지 않는다. ### Reference Model Agent -- Exact legacy inventory, generator `Abaqus/CAE Learning Edition 2024`, source commit - `2b34d0b`, external SI provenance와 stress comparison N/A를 계약에 고정한다. -- 추가 reference model은 canonical filenames와 metadata를 사용하며 이 legacy file을 - 변경하지 않는다. +- Exact legacy inventory, current row schema와 stress comparison N/A를 계약에 고정한다. +- 추가 reference case의 exact paths와 수치 비교 계약은 해당 feature requirement가 정하며, + canonical naming이나 optional metadata를 readiness 조건으로 추가하지 않는다. ### Implementation Planning Agent @@ -467,7 +468,7 @@ element-aware reference가 없는 한 비교하지 않는다. ### Reference Verification Agent -- Artifact precheck 뒤 HDF5-to-legacy projection, node-station eligibility, row-set equality, - component-scale comparison 순서를 유지한다. +- Artifact precheck 뒤 HDF5-to-CSV source identity projection, direct element-endpoint row-set + equality, common family-scale comparison 순서를 유지한다. - Missing/extra/nonfinite row를 무시하거나 `SF2/SF3`/stress reference row를 합성하지 않는다. diff --git a/docs/linear-static-3d-euler-beam/numerical-review.md b/docs/linear-static-3d-euler-beam/numerical-review.md index ad673e3..88942c7 100644 --- a/docs/linear-static-3d-euler-beam/numerical-review.md +++ b/docs/linear-static-3d-euler-beam/numerical-review.md @@ -191,8 +191,13 @@ Confirmed defects, risks, and open issues are separated. 2. `NR-O02-DETERMINISTIC-REDUCTION`: stable COO sort and duplicate-summation rules are project policy and must be made explicit before NR-T11. 3. `NR-O03-STATION-NORMALIZATION`: reversed connectivity/local-axis orientation and legitimate jumps at loaded interior nodes need an explicit downstream row-normalization/eligibility rule. The legacy baseline may use its documented stable orientation and unloaded interior stations, but mismatch must never be averaged. NR-T07 covers element signs. +2026-08-18 amendment: `NR-O03-STATION-NORMALIZATION` is resolved and superseded for the +approved case. The regenerated elemental-force CSV supplies `Element Label`, so comparison uses +direct `(instance, element label, endpoint node label, component)` identity and never collapses, +selects or averages adjacent endpoints. + No open issue requires formulation revision. NR-O01/NR-O02 are implementation-planning -handoffs; NR-O03 belongs to I/O and reference-model contracts. +handoffs; the resolved NR-O03 identity is fixed by the I/O and reference-model contracts. ## Required Revisions @@ -206,7 +211,7 @@ handoffs; NR-O03 belongs to I/O and reference-model contracts. ### Reference Model Agent -- Make NR-O03 orientation and unloaded-interior assumptions explicit without modifying the approved legacy artifacts. +- Enforce the resolved NR-O03 direct element-endpoint identity without modifying reference artifacts. ## Downstream Handoff diff --git a/docs/linear-static-3d-euler-beam/reference-comparison.md b/docs/linear-static-3d-euler-beam/reference-comparison.md index 61b359f..2af8948 100644 --- a/docs/linear-static-3d-euler-beam/reference-comparison.md +++ b/docs/linear-static-3d-euler-beam/reference-comparison.md @@ -10,11 +10,45 @@ - source_io_definition: `docs/linear-static-3d-euler-beam/io.md` - source_implementation_plan: `docs/linear-static-3d-euler-beam/implementation-plan.md` - source_implementation_report: `docs/linear-static-3d-euler-beam/implementation-report.md` -- status: `pass-for-physics-evaluation` +- historical_status: `pass-for-physics-evaluation` +- current_status: `pass-for-physics-evaluation` +- superseded_on: `2026-08-18` +- revalidated_on: `2026-08-18` - owner_agent: `reference-verification-agent` - date: `2026-08-09` - review_fix_date: `2026-08-10` +The 2026-08-09 evidence below is preserved as a historical record of the former 11-station, +component-scale comparison. ADR-022 and the regenerated 20-row element-endpoint CSV supersede +that identity and tolerance. The following revalidation is the current reference-gate evidence. + +## 2026-08-18 Common-policy Revalidation + +`cmake --build .harness/build --config Debug` passed, followed by +`ctest --test-dir .harness/build -C Debug --output-on-failure`: 214/214 tests passed. +The B33 reference test generated +`.harness/build/reference/cantilever-beam-b33/comparison.json` from the exact read-only +reference paths. + +- exact canonical rows: 212 = 66 displacement + 66 reaction + 80 direct element-endpoint + section-resultant rows +- exact families: 6; all row gates and family RMS gates passed +- direct section identity: 20 endpoint rows, 10 B33 elements x 2 connectivity-matched endpoints +- warnings, identity/schema/nonfinite failures: 0 +- overall verdict: `passed=true` + +| family | rows | scale | near-zero rows | max absolute error | scale-relative RMS | +| --- | ---: | ---: | ---: | ---: | ---: | +| translation `UX/UY/UZ` | 33 | `1.90476272e-2` | 23 | `5.333229170789711e-10` | `8.459623217000381e-9` | +| rotation `URX/URY/URZ` | 33 | `2.85714399e-3` | 23 | `1.000013943180944e-10` | `1.1723557271997219e-8` | +| reaction force `RF1/RF2/RF3` | 33 | `1.0e6` | 32 | `8.195638656616211e-7` | `2.0922817757699256e-13` | +| reaction moment `RM1/RM2/RM3` | 33 | `1.0e7` | 32 | `5.0514936447143555e-6` | `8.811160558471221e-14` | +| section force `N` | 20 | `0` | 20 | `0` | `0` | +| section moment `T/My/Mz` | 60 | `1.0e7` | 41 | `1.2499958951957524e-1` | `2.2910664449287597e-9` | + +All values above come from the generated JSON ledger. The historical report body starts below +and must not be read as the current identity/tolerance evidence. + The prerequisite build/test report has status `pass-for-reference-verification`. This report applies only the approved Abaqus B33 reference tolerance. It does not approve physics sanity or release readiness. diff --git a/docs/linear-static-3d-euler-beam/reference-model.md b/docs/linear-static-3d-euler-beam/reference-model.md index 277fc05..f1277d5 100644 --- a/docs/linear-static-3d-euler-beam/reference-model.md +++ b/docs/linear-static-3d-euler-beam/reference-model.md @@ -12,9 +12,10 @@ - status: `ready-for-implementation-planning` - owner_agent: `reference-model-agent` - date: `2026-08-09` +- amended_on: `2026-08-18` - approved_reference_model: `cantilever-beam-b33` -- approved_reference_schema: `abaqus-cae-report-csv-v0` -- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` +- approved_reference_identity: direct source-node and element-endpoint rows +- reference_baseline: exact current files under `reference/cantilever beam/` 이 문서는 구현 전에 필요한 code verification, analytical solution verification 및 approved B33 reference comparison의 모델·artifact 계약을 정의한다. 이 status는 모델과 @@ -45,7 +46,7 @@ test fixtures이며 reference artifact bundle로 가장하지 않는다. 논리 모델 `cantilever-beam-b33`의 exact read-only legacy bundle만 사용하여 FESA `results.h5`의 displacement, reaction 및 endpoint section resultant를 Abaqus/CAE report CSV row와 비교한다. Artifact precheck와 exact row-set matching이 먼저 통과해야 하며, -수치 비교는 component-scale mixed tolerance를 사용한다. Axial `S11` output은 필수지만 +수치 비교는 ADR-022의 common family-scale tolerance를 사용한다. Axial `S11` output은 필수지만 Abaqus beam stress comparison은 명시적 N/A다. ### Excluded validation scope @@ -136,8 +137,8 @@ single-step deck and changes only the named condition. - boundary_conditions: source node 1, DOFs 1 through 6 fixed - load: source node 11, global DOF 3, magnitude `-1e6` N - model_id: `cantilever-beam-b33` -- logical_schema: `abaqus-cae-report-csv-v0` -- source_commit: `2b34d0b` +- historical_schema_record: `abaqus-cae-report-csv-v0` for the 2026-08-09 inventory only +- historical_source_commit: `2b34d0b` for the 2026-08-09 inventory only - generator: `Abaqus/CAE Learning Edition 2024` - units: SI - nodal_coordinate_system: global Cartesian @@ -145,7 +146,7 @@ single-step deck and changes only the named condition. - step_name: `Step-1` - increment: `1` - step_time: `1.0` -- artifact_status: all four exact paths present; structural precheck observed; FESA comparison not run +- artifact_status: all four exact paths present; 20-row element-endpoint comparison passed on 2026-08-18 - stress: N/A for Abaqus reference comparison; mandatory FESA `S11` remains covered by unit/analytical and HDF5 schema tests The input and CSV numeric reference values are not recalculated, repaired, rounded, clamped or @@ -193,93 +194,44 @@ For this approved legacy bundle: - `README.md`: N/A - stress CSV: N/A because beam stress reference comparison is outside the approved V0 scope -The approved design and this contract record model ID, provenance, generator, source commit, -units, coordinate systems, step/frame identity, logical CSV schema, exact inventory, tolerance -policy and the stress N/A reason. The optional metadata file's absence and the approved legacy -README/stress exclusions therefore do not change the record to `needs-reference-artifacts`. +This contract records the exact required paths, row identities, comparison quantities, tolerance +policy and the stress N/A reason. Historical schema/provenance fields are retained as dated +inventory only and are not readiness gates. The optional metadata file's absence and the approved +legacy README/stress exclusions do not change the record to `needs-reference-artifacts`. -### Future reference bundles +### Later reference cases and optional metadata -Every later reference model shall use this structure unless its approved requirement explicitly -marks a quantity N/A: - -```text -reference/ - / - model.inp - metadata.json # optional - _displacements.csv - _reactions.csv - _internalforces.csv - _stresses.csv - README.md -``` - -CSV names are canonical `_*.csv` names. `README.md` is mandatory for later bundles; -`metadata.json` is optional. A quantity CSV may be omitted only when the upstream acceptance -contract explicitly records N/A and gives its verification replacement. Missing required files -or required Reference Model Contract provenance keep that model at `needs-reference-artifacts`. - -## Reference Metadata Contract - -This document is the required source of truth for the following metadata. A later bundle may -optionally duplicate it in `metadata.json` using at least this schema: - -```json -{ - "feature_id": "linear-static-3d-euler-beam", - "model_id": "", - "artifact_status": "needs-reference-artifacts | ready-for-verification", - "input_file": "model.inp", - "abaqus_version": "", - "generation_owner": "", - "generation_date": "", - "source_commit": "", - "units": "", - "coordinate_system": "", - "analysis_type": "single linear static", - "element_types": ["B33"], - "step_name": "Step-1", - "increment": 1, - "step_time": 1.0, - "output_requests": ["U", "RF", "SF"], - "reference_csv_schema_version": "", - "reference_csv_files": [""], - "tolerance_policy": "", - "limitations": [""] -} -``` - -No agent may invent unknown provenance fields or mark a bundle ready merely because filenames -exist. An absent `metadata.json` is allowed. If the file exists, inventory it read-only and report -any disagreement with this contract or stored artifacts as an upstream contract/provenance issue. +Each later feature requirement names its exact input and comparison CSV paths, required row +identity/components and N/A quantities. Canonical filenames, `README.md`, `metadata.json`, +generator/version, provenance, duplicated unit/schema fields and a portfolio-wide directory layout +are not readiness gates unless that feature explicitly makes one part of its numerical comparison +contract. No agent may invent unknown provenance. If optional metadata exists, inspect it read-only +and report disagreement without rewriting the reference artifacts. ## Abaqus Reference CSV Requirements Header comparison trims whitespace around each comma-separated field but does not rename fields. For every file, `Frame` must normalize exactly from `Increment 1: Step Time = 1.000` to `(Step-1, frame 0)`, `Part Instance Name` must resolve to the -preserved instance identity, `Node Label` must be a unique source-node station, and all projected +preserved instance identity, every declared source identity must be unique, and all projected numeric values must be finite. | exact legacy path | expected trimmed header | unique row key | observed inventory | | --- | --- | --- | --- | | `reference/cantilever beam/cantilever beam displacements.csv` | `Frame, Part Instance Name, Node Label, U-U1, U-U2, U-U3, UR-UR1, UR-UR2, UR-UR3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed | | `reference/cantilever beam/cantilever beam reactions.csv` | `Frame, Part Instance Name, Node Label, RF-RF1, RF-RF2, RF-RF3, RM-RM1, RM-RM2, RM-RM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed | -| `reference/cantilever beam/cantilever beam elemental forces.csv` | `Frame, Part Instance Name, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed | +| `reference/cantilever beam/cantilever beam elemental forces.csv` | `Frame, Part Instance Name, Element Label, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3` | `(Frame, Part Instance Name, Element Label, Node Label)` | 20 rows; 10 B33 elements x 2 connectivity-matched endpoints; header/key/finite/arity checks observed | The wide-row key becomes unique canonical component rows after adding `quantity` and `component`. Missing, extra, duplicate, nonfinite, header/schema or identity mismatch stops comparison as `needs-reference-artifacts` or `schema-mismatch`. No bad or near-zero row may be silently dropped. -For the elemental-force CSV, source node station is not an element-end identity. A boundary -station uses its only incident endpoint. An interior station may collapse exactly two endpoints -only when chain connectivity, section and local axes are consistent and the station has no -concentrated force/moment. The two positive-local-x section-cut values must first agree within the -approved component tolerance. If they agree, choose the endpoint with smaller stable internal -element ID; never average. Reversed orientation, branch, section jump, local-axis discontinuity or -loaded interior station requires an element-aware future schema and is a `schema-mismatch` under -this legacy schema. +For the elemental-force CSV, `(Part Instance Name, Element Label, Node Label)` is the source +element-endpoint identity. Each B33 element must have exactly two rows whose node labels match its +ordered input connectivity. Every row maps directly to the corresponding HDF5 +`[element,endpoint,N/T/My/Mz]` row. Missing, extra, duplicate or connectivity-mismatched endpoint +identity fails as `schema-mismatch` before tolerance. The comparator does not collapse endpoints to +a node station, choose a representative or average values. ## Coverage Matrix @@ -288,20 +240,20 @@ the canonical V0 step identity. | verification quantity | requirement ids | model_id | FESA HDF5 dataset | legacy CSV and components | row identity/location | tolerance | verification method | status | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| nodal displacement/rotation | 003, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/displacement` | `reference/cantilever beam/cantilever beam displacements.csv`: `U-U1/U-U2/U-U3 -> UX/UY/UZ`, `UR-UR1/UR-UR2/UR-UR3 -> URX/URY/URZ` | preserved instance + source node; global nodal | `1e-9 + 1e-6*reference_scale` per displacement/rotation component | HDF5-to-read-only CSV after artifact/row-set precheck | ready for implementation planning; comparison not run | -| nodal reaction force | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RF-RF1/RF-RF2/RF-RF3 -> RF1/RF2/RF3` | preserved instance + source node; global nodal | `1e-3 + 1e-6*reference_scale` per force component | HDF5-to-read-only CSV plus global equilibrium | ready for implementation planning; comparison not run | -| nodal reaction moment | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RM-RM1/RM-RM2/RM-RM3 -> RM1/RM2/RM3` | preserved instance + source node; global nodal | `1e-3 + 1e-6*reference_scale` per moment component | HDF5-to-read-only CSV plus moment equilibrium | ready for implementation planning; comparison not run | -| section axial force | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SF-SF1 -> N` | positive-local-x endpoint projected to eligible source node station | `1e-3 + 1e-6*reference_scale` for `N` | endpoint consistency, deterministic station selection, HDF5-to-CSV | ready for implementation planning; comparison not run | -| section moments/torsion | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SM-SM1 -> My`, `SM-SM2 -> Mz`, `SM-SM3 -> T` | positive-local-x endpoint projected to eligible source node station; beam local | `1e-3 + 1e-6*reference_scale` separately for `My`, `Mz`, `T` | endpoint consistency, deterministic station selection, HDF5-to-CSV | ready for implementation planning; comparison not run | +| nodal displacement/rotation | 003, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/displacement` | `reference/cantilever beam/cantilever beam displacements.csv`: `U-U1/U-U2/U-U3 -> UX/UY/UZ`, `UR-UR1/UR-UR2/UR-UR3 -> URX/URY/URZ` | preserved instance + source node; global nodal | common translation/rotation family policy | HDF5-to-read-only CSV after artifact/row-set precheck | passed 2026-08-18 | +| nodal reaction force | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RF-RF1/RF-RF2/RF-RF3 -> RF1/RF2/RF3` | preserved instance + source node; global nodal | common reaction-force family policy | HDF5-to-read-only CSV plus global equilibrium | passed 2026-08-18 | +| nodal reaction moment | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RM-RM1/RM-RM2/RM-RM3 -> RM1/RM2/RM3` | preserved instance + source node; global nodal | common reaction-moment family policy | HDF5-to-read-only CSV plus moment equilibrium | passed 2026-08-18 | +| section axial force | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SF-SF1 -> N` | direct instance + source element + endpoint node; beam local | common section-force family policy | direct HDF5-to-endpoint CSV comparison | passed 2026-08-18 | +| section moments/torsion | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SM-SM1 -> My`, `SM-SM2 -> Mz`, `SM-SM3 -> T` | direct instance + source element + endpoint node; beam local | common section-moment family policy | direct HDF5-to-endpoint CSV comparison | passed 2026-08-18 | | equilibrium end action | 031, 035, 043 | analytical models and physics portfolio | `/steps/Step-1/frames/0/element/end_force_local` | Abaqus CSV N/A for direct outward-action comparison | element endpoint `xi=-1,+1`; local outward action `[FX,FY,FZ,MX,MY,MZ]` | analytical normalized `1e-12`; residual `1e-10` | unit/analytical end-sign tests and later physics sanity | planned | | generalized strain/resultant | 029, 031, 035 | code and analytical models | `/steps/Step-1/frames/0/element/generalized_strain` and `/steps/Step-1/frames/0/element/generalized_resultant` | Abaqus CSV N/A | two Gauss points; beam local | matrix/formulation normalized `1e-12`, analytical relative `1e-9` | formulation/unit/HDF5 schema tests | planned | | axial stress | 029, 032, 035 | axial/local-z analytical models | `/steps/Step-1/frames/0/element/stress_s11` | stress CSV N/A; Abaqus beam stress reference comparison N/A | element, Gauss point, input section point or `fesa-default` centroid | analytical relative `1e-9`; exact unit/row schema | unit/analytical recovery and HDF5 schema tests | planned; reference N/A | -For every matched reference group, -`reference_scale=max(abs(Abaqus reference rows))` for the same model, step/frame, quantity and -component, and `row_tolerance=absolute_floor+1e-6*reference_scale`. Abaqus values alone set the -scale. A zero scale uses only the floor. Every row decision and max absolute, component-scale -normalized, RMS, norm and worst-row/component metrics must be reported. +For every matched family, `S=max(abs(Abaqus reference rows))` uses only read-only reference values. +Rows with `abs(reference)<=0.01*S` use `error<=0.01*S`; other rows use relative error `<=0.05`; +the family also requires `RMS(error)/S<=0.01`. A zero-scale family requires exact-zero FESA values. +Every row branch/decision and family scale, near-zero band/count, max absolute error, +scale-relative RMS, worst row/component and zero-scale diagnostic must be reported. ### Complete must-requirement coverage @@ -346,19 +298,20 @@ it does not waive the requirement. | `FESA-REQ-LS3DEB-033` | `smoke-b33-cli`, diagnostic negative cases | CLI exit-code/field/order integration tests; CSV N/A | | `FESA-REQ-LS3DEB-034` | every implementation model/test | per-step RED/GREEN/VERIFY and full MSVC/CTest evidence; model CSV N/A | | `FESA-REQ-LS3DEB-035` | `NR-T01` through `NR-T11` and analytical inventory | exact numerical criteria in model records | -| `FESA-REQ-LS3DEB-036` | `cantilever-beam-b33` and comparison-policy unit fixtures | Abaqus-only component scale and exact formula tests | -| `FESA-REQ-LS3DEB-037` | same comparison fixtures, including zero-scale groups | exact SI floors and zero-scale tests | +| `FESA-REQ-LS3DEB-036` | `cantilever-beam-b33` and comparison-policy unit fixtures | exact family membership and Abaqus-only scale tests | +| `FESA-REQ-LS3DEB-037` | same comparison fixtures, including boundary and zero-scale families | near-zero, relative-row, scale-relative RMS and zero-scale tests | | `FESA-REQ-LS3DEB-038` | malformed reference cases in `neg-b33-input-contract` | pre-tolerance fail-fast and no-clamp/no-drop tests | -| `FESA-REQ-LS3DEB-039` | verification-report schema fixture | every row decision and aggregate/worst metrics test | +| `FESA-REQ-LS3DEB-039` | verification-report schema fixture | every row branch/decision and family aggregate/worst metrics test | | `FESA-REQ-LS3DEB-040` | exact legacy Artifact Bundle Contract | inventory plus `git diff --exit-code -- reference/` process check | | `FESA-REQ-LS3DEB-041` | approved artifact precheck | four files, B33, exact headers, unique keys, finite values | -| `FESA-REQ-LS3DEB-042` | three comparison quantities and station normalization | exact component mapping, endpoint consistency and no-average tests | +| `FESA-REQ-LS3DEB-042` | three comparison quantities and direct endpoint projection | exact component mapping plus missing/extra/duplicate/connectivity-mismatch endpoint tests | | `FESA-REQ-LS3DEB-043` | gate audit; later physics portfolio | reference execution N/A at this step; enforce build/test -> comparison -> physics -> release order | | `FESA-REQ-LS3DEB-044` | process/Git diff audit and limitations review | no reference execution/mutation and no out-of-scope support claim | ## Artifact Acceptance Checklist -Read-only inventory inspection on `2026-08-09` established the following pre-implementation facts: +The following is historical 2026-08-09 pre-implementation inventory evidence. Its 11-row +elemental-force observation is superseded by the 2026-08-18 regenerated 20-row endpoint file: - all four exact legacy paths exist; - the input declares `TYPE=B33`; @@ -370,6 +323,11 @@ Read-only inventory inspection on `2026-08-09` established the following pre-imp - absent `metadata.json` is allowed by project-wide policy; legacy `README.md` and stress CSV are accepted N/A exceptions; - no reference value was recalculated and no comparison was performed. +Read-only inspection on `2026-08-18` confirms that the elemental-force CSV has 20 rows, +exactly two connectivity-matched endpoint rows for each of the 10 B33 elements. Displacement and +reaction CSVs remain 11-row source-node tables. The fresh comparison passed on 2026-08-18; +current metrics are recorded in `reference-comparison.md`. + Before an actual comparison, tooling must repeat all artifact checks, verify the exact Frame and instance identities, compare the complete projected row sets, and stop on any failure. The Step AC must also show no working-tree diff under `reference/`. Passing this checklist is not a reference @@ -383,8 +341,8 @@ comparison pass. model. - `NR-O01` (official oneMKL PARDISO contract) and `NR-O02` (deterministic duplicate-reduction algorithm) remain implementation-planning inputs, not reference artifact defects. -- Future reversed, branched, loaded-interior or section-jump reference models require an - element-aware canonical CSV schema; the legacy node-station schema must not be generalized. +- Future reversed, branched, loaded-interior or section-jump reference models use the same + element-endpoint identity and require their own approved reference coverage. ### Implementation Planning Agent @@ -403,8 +361,8 @@ artifacts for code/analytical fixtures. ### Reference Verification Agent -Run `ARTIFACT CHECK -> HDF5 ROW PROJECTION -> EXACT ROW-SET CHECK -> ENDPOINT CONSISTENCY -> -COMPONENT-SCALE COMPARE -> REPORT`. Use only the exact legacy files and the HDF5 paths/component +Run `ARTIFACT CHECK -> HDF5 ROW PROJECTION -> EXACT ROW-SET CHECK -> DIRECT ENDPOINT MATCH -> +COMMON FAMILY-SCALE COMPARE -> REPORT`. Use only the exact legacy files and the HDF5 paths/component mappings in the Coverage Matrix. Do not synthesize `SF2/SF3` or stress rows, clamp values, omit rows, average interior endpoints, or change the approved tolerance. diff --git a/docs/linear-static-3d-euler-beam/release.md b/docs/linear-static-3d-euler-beam/release.md index 299dce2..6963d9a 100644 --- a/docs/linear-static-3d-euler-beam/release.md +++ b/docs/linear-static-3d-euler-beam/release.md @@ -19,11 +19,17 @@ - reference_model_id: `cantilever-beam-b33` - reference_schema: `abaqus-cae-report-csv-v0` - reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` -- status: `ready-for-release` +- historical_status: `ready-for-release` +- current_status: `superseded-pending-revalidation` +- superseded_on: `2026-08-18` - owner_agent: `release-agent` - date: `2026-08-10` - final_review_source_head: `b7a1258ce0f36a85b888e23470cf9d936a7595cd` +The readiness evidence below is historical. The 2026-08-18 build/reference revalidation passed, +but fresh physics and release audits are still required before a current `ready-for-release` +verdict may be issued. + This is an internal feature-readiness verdict. It authorizes no publish, deploy, package, tag, push, external release, or reference-artifact operation. diff --git a/docs/linear-static-3d-euler-beam/requirements.md b/docs/linear-static-3d-euler-beam/requirements.md index 32188e2..f43a407 100644 --- a/docs/linear-static-3d-euler-beam/requirements.md +++ b/docs/linear-static-3d-euler-beam/requirements.md @@ -10,7 +10,7 @@ - approved_design: `docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md` - approval_basis: design `status: approved`, user approval on `2026-08-08`, and amendment on `2026-08-09` - source_formulation: `docs/linear-static-3d-euler-beam/formulation.md` -- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` +- reference_baseline: exact read-only artifacts under `reference/cantilever beam/` ## Purpose @@ -113,7 +113,7 @@ compatibility, 새로운 수학 계약 또는 새로운 구현 정책을 추가 - nodal_displacement: required, global six components, HDF5-to-Abaqus CSV comparison - reaction: required, global six components plus global force/moment equilibrium - equilibrium_end_action: required, local endpoint six components; unit/analytical and physics tests -- section_resultant: required, endpoint `[N,T,My,Mz]`, node-station-normalized reference comparison +- section_resultant: required, endpoint `[N,T,My,Mz]`, element-endpoint reference comparison - generalized_strain_and_resultant: required at two Gauss points; formulation and schema tests - stress: axial `S11` required; Abaqus reference comparison N/A - residual: required, free-DOF and normalized global equilibrium checks @@ -122,21 +122,22 @@ compatibility, 새로운 수학 계약 또는 새로운 구현 정책을 추가 ## Tolerance Policy -- **FESA-REQ-LS3DEB-036** — The approved B33 reference comparison shall group rows by the same model, step/frame, quantity, and component, compute `reference_scale` only from read-only Abaqus values, and apply `absolute_floor + 1e-6 * reference_scale` to every matched row. -- **FESA-REQ-LS3DEB-037** — For the approved SI bundle, displacement and rotation shall use `absolute_floor=1e-9`, force and moment shall use `absolute_floor=1e-3`, and a zero component scale shall use the applicable absolute floor alone. +- **FESA-REQ-LS3DEB-036** — The approved B33 reference comparison shall group rows into translation, rotation, reaction-force, reaction-moment, section-force and section-moment families and compute each family scale `S=max(abs(reference))` only from read-only Abaqus values. +- **FESA-REQ-LS3DEB-037** — A matched row with `abs(reference)<=0.01*S` shall pass when `abs(fesa-reference)<=0.01*S`; every other row shall pass when its relative error is at most `0.05`; each family shall also satisfy `RMS(error)/S<=0.01`. No independent absolute-error gate is used, and a zero-scale family passes only when every FESA value is exactly zero. - **FESA-REQ-LS3DEB-038** — Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation. -- **FESA-REQ-LS3DEB-039** — The verification report shall record every row decision and maximum absolute error, component-scale normalized error, RMS error, norm error, and worst row/component for each compared quantity. +- **FESA-REQ-LS3DEB-039** — The verification report shall record every row decision and comparison branch, family identity/components, reference scale, near-zero band/count, maximum absolute error, scale-relative RMS, worst row/component and zero-scale diagnostic for each compared family. ## Reference Artifact Requirements -The approved logical model is `cantilever-beam-b33`, schema is -`abaqus-cae-report-csv-v0`, source commit is `2b34d0b`, generator is +The approved logical model is `cantilever-beam-b33`. The input header records `Abaqus/CAE Learning Edition 2024`, and the coordinate/output contract is global Cartesian nodal output plus beam-local section-force output at `Step-1`, increment 1, step time 1.0. +The historical `abaqus-cae-report-csv-v0` and source commit `2b34d0b` inventory is not a +readiness requirement for the regenerated elemental-force CSV. - **FESA-REQ-LS3DEB-040** — The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle. - **FESA-REQ-LS3DEB-041** — Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start. -- **FESA-REQ-LS3DEB-042** — Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and node-station-normalized section resultant by `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; adjacent interior endpoints shall first agree within approved tolerance and shall not be averaged to hide a mismatch. +- **FESA-REQ-LS3DEB-042** — Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and section resultants by direct `(instance, element label, endpoint node label, component)` identity using `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; missing, extra, duplicate or connectivity-mismatched endpoint rows shall fail before tolerance and shall not be collapsed or averaged. - **FESA-REQ-LS3DEB-043** — Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations. - **FESA-REQ-LS3DEB-044** — FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior. @@ -180,13 +181,13 @@ nodal output plus beam-local section-force output at `Step-1`, increment 1, step | FESA-REQ-LS3DEB-033 | The CLI shall support `fesa.exe --output `, default output to the current directory's `results.h5`, use exit codes `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5`, and emit `severity`, `code`, `file`, `line`, `keyword`, `entity_identity`, and `message` diagnostics to stderr in deterministic order. | output | Stabilize automation and failure classification. | Approved design §10 | must | CLI integration and diagnostic ordering tests | Default/explicit output works and every failure class returns its exact code and complete ordered fields. | Exact codes, fields, and order | io-definition-agent; implementation-planning-agent | approved | | FESA-REQ-LS3DEB-034 | Every production C++ behavior shall be developed in one step as GoogleTest `RED -> GREEN -> VERIFY`, with a related C++ test file, focused CTest evidence, full MSVC x64 Debug build/CTest evidence, at least one discovered test, and no new warning under the FESA target's `/W4 /WX` policy. | verification | Enforce project TDD and warning policy. | Approved design §§11.1, 11.4; ADR-012 | must | Implementation report; build/CTest logs | The related test fails first, then focused/full tests pass, discovery finds tests, and FESA emits no warning. | Zero test failures and new warnings | implementation-planning-agent; implementation-agent; build-test-executor-agent | approved | | FESA-REQ-LS3DEB-035 | Numerical tests shall satisfy normalized `1e-12` for stiffness symmetry and two-point-Gauss/closed-form agreement, normalized `1e-10` for rigid-mode and linear-system residual, and relative `1e-9` for analytical solutions, while checking six rigid modes, rank 6, positive deformation energy, transformation orthogonality/energy invariance, prescribed-displacement recovery, and axial/torsion/two-plane bending benchmarks. | verification | Detect sign, integration, rank, and transform defects. | Approved design §§11.2, 11.3; formulation §18 | must | Unit, analytical, and orchestration tests | Every listed invariant and analytical case passes at its stated threshold. | `1e-12` matrix; `1e-10` residual; `1e-9` analytical | formulation-agent; numerical-review-agent; implementation-planning-agent | approved | -| FESA-REQ-LS3DEB-036 | The approved B33 reference comparison shall group rows by the same model, step/frame, quantity, and component, compute `reference_scale` only from read-only Abaqus values, and apply `absolute_floor + 1e-6 * reference_scale` to every matched row. | tolerance | Give zero and nonzero rows one deterministic rule. | Approved design §11.3; ADR-014 | must | Comparison unit/integration test; report review | Every group uses the Abaqus-only maximum absolute scale and every matched row uses the exact formula. | Relative coefficient `1e-6` | reference-model-agent; reference-verification-agent | approved | -| FESA-REQ-LS3DEB-037 | For the approved SI bundle, displacement and rotation shall use `absolute_floor=1e-9`, force and moment shall use `absolute_floor=1e-3`, and a zero component scale shall use the applicable absolute floor alone. | tolerance | Preserve dimensional meaning near zero. | Approved design §11.3; ADR-014 | must | Comparison tests with zero/near-zero rows | Each quantity uses its exact SI floor and zero-scale groups use no relative contribution. | SI `1e-9` displacement/rotation; `1e-3` force/moment | reference-model-agent; reference-verification-agent | approved | +| FESA-REQ-LS3DEB-036 | The approved B33 reference comparison shall group rows into translation, rotation, reaction-force, reaction-moment, section-force and section-moment families and compute `S=max(abs(reference))` from read-only Abaqus values. | tolerance | Give zero-like components a dimensionally compatible reference scale. | Common tolerance design; ADR-022 | must | Comparison unit/integration test; report review | Every family uses the exact declared components and Abaqus-only maximum scale. | Family scale | reference-model-agent; reference-verification-agent | approved | +| FESA-REQ-LS3DEB-037 | Near-zero rows shall use the `0.01*S` fallback, other rows relative error `0.05`, and every family scale-relative RMS `0.01`; zero-scale families require exact-zero FESA values and no independent absolute gate is used. | tolerance | Stabilize zero-like rows while retaining row and aggregate checks. | Common tolerance design; ADR-022 | must | Boundary, zero-scale and RMS comparison tests | Every row and family uses the exact common constants and branch rules without NaN/Inf. | `0.01`, `0.05`, `0.01` | reference-model-agent; reference-verification-agent | approved | | FESA-REQ-LS3DEB-038 | Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation. | tolerance | Prevent false passes through omission or clamping. | Approved design §§11.3, 12; ADR-014 | must | Negative comparison tests | Every listed invalid case fails before numeric comparison and zero values remain unchanged. | No ignored invalid rows | reference-verification-agent | approved | -| FESA-REQ-LS3DEB-039 | The verification report shall record every row decision and maximum absolute error, component-scale normalized error, RMS error, norm error, and worst row/component for each compared quantity. | tolerance | Make the pass/fail decision auditable. | Approved design §11.3; ADR-014 | must | Verification report schema/review | Per-row decisions and all required aggregate/worst metrics are present for every quantity. | Report completeness | reference-verification-agent; release-agent | approved | +| FESA-REQ-LS3DEB-039 | The verification report shall record every row branch/decision, family identity/components, scale, near-zero band/count, maximum absolute error, scale-relative RMS, worst row/component and zero-scale diagnostic. | tolerance | Make the pass/fail decision auditable. | Common tolerance design; ADR-022 | must | Verification report schema/review | Per-row decisions and all required family metrics are present without nonfinite report values. | Report completeness | reference-verification-agent; release-agent | approved | | FESA-REQ-LS3DEB-040 | The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle. | reference | Protect the approved correctness baseline. | Approved design §12; ADR-010, ADR-014 | must | Artifact inventory; Git diff review | Exact filenames exist and no reference file is added, removed, renamed, or content-modified. | Exact path/content identity | reference-model-agent; reference-verification-agent; release-agent | approved | | FESA-REQ-LS3DEB-041 | Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start. | reference | Detect stale B31 or malformed evidence. | Approved design §§6.3, 12 | must | Artifact-check integration test | All checks pass before comparison and every failure uses an approved classification. | Exact inventory and schema | reference-model-agent; reference-verification-agent | approved | -| FESA-REQ-LS3DEB-042 | Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and node-station-normalized section resultant by `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; adjacent interior endpoints shall first agree within approved tolerance and shall not be averaged to hide a mismatch. | reference | Compare equivalent quantities despite legacy station rows. | Approved design §§8.2, 12 | must | Reference comparison test/report | All components match by source identity and interior endpoints pass before deterministic representative selection. | Requirements 036 and 037 policy | io-definition-agent; reference-model-agent; reference-verification-agent | approved | +| FESA-REQ-LS3DEB-042 | Reference verification shall compare displacement and reaction by source-node identity and section resultants by direct `(instance, element label, endpoint node label, component)` identity using `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; endpoint rows shall not be collapsed or averaged. | reference | Preserve the source identity supplied by the element-endpoint CSV and HDF5. | Common tolerance design; ADR-022 | must | Reference comparison test/report | All 20 B33 endpoint rows map one-to-one and malformed endpoint identity fails before tolerance. | Requirements 036 and 037 policy | io-definition-agent; reference-model-agent; reference-verification-agent | approved | | FESA-REQ-LS3DEB-043 | Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations. | governance | Keep numerical similarity distinct from physical/release approval. | Approved design §§11, 12, 13 | must | Gate evidence audit | Each downstream report cites the preceding pass and physics evidence covers all six checks before release review. | Relevant upstream tolerances | coordinator-agent; physics-evaluation-agent; release-agent | approved | | FESA-REQ-LS3DEB-044 | FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior. | governance | Prevent baseline contamination and scope inflation. | Approved design §§2.2, 12; ADR-010 | must | Process audit; Git diff; release documentation review | No reference execution or artifact mutation occurs and every exclusion is recorded without a support claim. | N/A | coordinator-agent; reference-model-agent; release-agent | approved | diff --git a/docs/linear-static-mitc4-shell/formulation.md b/docs/linear-static-mitc4-shell/formulation.md index 2f65312..7635dda 100644 --- a/docs/linear-static-mitc4-shell/formulation.md +++ b/docs/linear-static-mitc4-shell/formulation.md @@ -1540,11 +1540,11 @@ or drilling-energy warning is part of this check. ### 17.5 Reference-comparison boundary -Abaqus comparison uses only the declared full-integration S4 case and blocks only -on matched global `U1/U2/U3` rows under the fixed absolute criterion -`abs(fesa-reference) <= 1.0e-5`. `UR1/UR2/UR3` uses the same fixed absolute value but -an exceedance emits only a deterministic nonblocking warning. A reported reference -scale is diagnostic only and does not enter the MITC4 decision or normalization. +Abaqus comparison uses only the declared full-integration S4 case. Matched global +`U1/U2/U3` families are blocking and `UR1/UR2/UR3` families are warning-only. Both use +the common family-scale row rule (`S=max(abs(reference))`, near-zero ratio `0.01`, relative +tolerance `0.05`) and scale-relative RMS tolerance `0.01`. `S=0` requires exact-zero FESA +values and there is no independent absolute-error gate. FESA `S4` and `S4R` inputs must produce the same internal numerical rows for identical supported models while preserving distinct source metadata. This common-path property is verified without consuming an S4R Abaqus artifact; Abaqus S4R is not an acceptance @@ -1631,7 +1631,7 @@ research brief remain the project source of truth. | `024-029` | deterministic element buffers, partitioned linear lifecycle, full-residual reaction | planning | | `039-048` | nodal/global and shell/local recovery inventory, units, identities, physical shell energy | I/O schema | | `049-057` | normalized invariants, patches, fixed drilling, declared S4 reference and equilibrium | Numerical Review/reference/physics | -| `058-064` | fixed absolute `1.0e-5`; U blocking and UR warning-only | reference verification | +| `058-064` | common family-scale row/RMS policy; U blocking and UR warning-only | reference verification | | `065-072` | exact existing S4 paths, S4R reference non-consumption, immutability and displacement-only boundary | reference model | ### 20.1 Numerical Review revision traceability @@ -1685,7 +1685,8 @@ items or an expanded reference portfolio as missing evidence. without claiming element equivalence. - Do not consume `reference/shellR/` in acceptance comparison; preserve S4R support through source-mapping/common-kernel/metadata tests. -- Use the fixed absolute MITC4 tolerance `1.0e-5`; do not alter the separate B33 tolerance or add administrative metadata or portfolio gates. +- Use the project-wide common family-scale row/RMS tolerance; do not add an independent + absolute-error gate, administrative metadata or portfolio gates. - Do not create, repair, rename, or run reference artifacts during this formulation gate. diff --git a/docs/linear-static-mitc4-shell/io.md b/docs/linear-static-mitc4-shell/io.md index fd7604e..a77977b 100644 --- a/docs/linear-static-mitc4-shell/io.md +++ b/docs/linear-static-mitc4-shell/io.md @@ -36,8 +36,8 @@ The user approved the following I/O-specific decisions on `2026-08-12`: retaining the existing common metadata, nodal-result, diagnostic, CLI, and failure-atomicity conventions. -The approved requirements fix the drilling rule and the MITC4 fixed absolute -displacement tolerance `1.0e-5`. +The approved requirements fix the drilling rule and the project-wide common +family-scale reference tolerance. Drilling calibration/output, `NR-O03`, `NR-O04`, bundle-administration metadata, and an expanded reference portfolio are outside this contract. @@ -515,22 +515,31 @@ Before tolerance evaluation: Only `U1/U2/U3` affect pass/fail. `UR1/UR2/UR3` are always compared and reported but can emit only an approved deterministic nonblocking warning. -For every matched displacement row: +Matched U rows form one blocking translation family and matched UR rows form one warning-only +rotation family. For each family: ```text -row_tolerance = 1.0e-5 -row_pass = abs(fesa_value-reference_value) <= row_tolerance +S = max(abs(reference_value_i)) +error_i = abs(fesa_value_i-reference_value_i) +near_zero_band = 0.01*S + +if abs(reference_value_i) <= near_zero_band: + row_pass = error_i <= near_zero_band +else: + row_pass = error_i/abs(reference_value_i) <= 0.05 + +relative_rms = sqrt(mean(error_i^2))/S +family_pass = all(row_pass) and relative_rms <= 0.01 ``` -The `1.0e-5` value is in the user-consistent length unit for U and dimensionless for -UR. No reference or result value is zero-clamped and neither component scale nor a -row-specific denominator changes the fixed value. A reference scale may remain in the -report as diagnostic information only. U exceedance fails; UR exceedance emits a -deterministic warning only. The separate B33 mixed tolerance is unchanged. +Reference values alone define `S`; values are not zero-clamped or omitted and there is no +independent absolute-error gate. A zero-scale family passes only if every FESA value is exactly +zero, otherwise it fails without emitting NaN/Inf. U row or RMS exceedance fails; the same UR +exceedance emits a deterministic warning only. -The report records every U/UR row, blocking/nonblocking decision, absolute error, -fixed-tolerance-normalized error, RMS error, displacement/rotation vector-norm -error, worst source row/component, and every UR warning. +The report records every U/UR row, blocking/nonblocking decision, family scale, +near-zero branch, absolute and applicable row-relative error, scale-relative RMS, +worst source row/component, and every UR warning. ## 8. CLI and Diagnostics Contract @@ -615,7 +624,7 @@ physics review, or release status follows from approval of this document alone. | `031-038` | source-independent MITC4 identity, fixed drilling stabilization and full-integration policy; no drilling output | Implementation Planning tests | | `039-048` | additive HDF5 v0 paths, mandatory quantities, location identity, atomic output | Reference Model and Implementation Planning | | `049-057` | diagnostic/schema hooks and required verification-metric/physical-energy evidence | Numerical Review and planning | -| `058-064` | normalized U/UR rows, fixed absolute `1.0e-5`, blocking/warning behavior, report inventory | Reference Verification | +| `058-064` | common-policy U/UR families, blocking/warning behavior, report inventory | Reference Verification | | `065-072` | exact current S4 paths, S4R reference non-consumption, immutability and displacement-only gate | Reference Model | ## 11. Open Issues and Downstream Handoff @@ -623,8 +632,8 @@ physics review, or release status follows from approval of this document alone. ### 11.1 Numerical Review boundary No I/O-owned calibration value remains open. Numerical Review shall verify the exact -fixed drilling rule, basic geometry predicates, required HDF5 inventory, and fixed -absolute MITC4 tolerance mapping. Drilling calibration/energy output, `NR-O03`, `NR-O04`, bundle +fixed drilling rule, basic geometry predicates, required HDF5 inventory, and common +family-scale row/RMS tolerance mapping. Drilling calibration/energy output, `NR-O03`, `NR-O04`, bundle administration and reference-portfolio expansion are removed scope. ### 11.2 Reference Model Agent @@ -632,7 +641,7 @@ administration and reference-portfolio expansion are removed scope. - Write `docs/linear-static-mitc4-shell/reference-model.md` using this exact keyword/HDF5/reference-row contract. - Record only the two exact existing input/displacement pairs, comparison components, - HDF5 projection, source-row identity, fixed absolute MITC4 tolerance and immutability rule. + HDF5 projection, source-row identity, common family-scale tolerance and immutability rule. - Treat reaction/stress artifacts as nonblocking review evidence and do not create location-equivalence claims absent from this contract. diff --git a/docs/linear-static-mitc4-shell/numerical-review.md b/docs/linear-static-mitc4-shell/numerical-review.md index 73a7c1e..2da997b 100644 --- a/docs/linear-static-mitc4-shell/numerical-review.md +++ b/docs/linear-static-mitc4-shell/numerical-review.md @@ -75,7 +75,7 @@ confirmed mathematical defect는 없다. | `NR-O02` drilling-energy ratio | `removed from scope` | Drilling energy는 내부 quadratic identity일 뿐 physical energy나 mandatory output이 아니며 ratio/warning threshold도 요구하지 않는다. | | `NR-O03` smooth-director calibration | `removed from scope` | Pairwise positive incident-normal orientation, finite/nonzero averaging 및 duplicate-node fold modeling이 승인된 exact predicate다. 별도 angle calibration은 gate가 아니다. | | `NR-O04` distortion/warp calibration | `removed from scope` | Basic topology, finite/nonzero surface measure 및 required-point `J>0`가 승인된 predicate다. Quality sweep이나 cutoff는 gate가 아니다. | -| `NR-O05` U/UR tolerance | `resolved` | 모든 관련 문서가 sole S4 reference에 고정 절대오차 `1.0e-5`, U blocking, UR warning-only를 동일하게 정의한다. Reference scale은 판정에 사용하지 않으며 S4R은 reference gate가 아닌 common-path evidence다. | +| `NR-O05` U/UR tolerance | `resolved` | 모든 관련 문서가 sole S4 reference에 공통 family-scale row/RMS 규칙, U blocking, UR warning-only를 동일하게 정의한다. S4R은 reference gate가 아닌 common-path evidence다. | 이전의 `needs-reference-model` 판정에 포함됐던 canonical naming, README, `metadata.json`, provenance, expanded portfolio 및 아직 없는 comparison result는 현재 @@ -223,9 +223,10 @@ confirmed mathematical defect는 없다. not emitted, and `S13/S23` point stress is not synthesized. Different natural or section locations are never averaged. - Reference comparison first rejects missing, extra, duplicate, nonfinite or - identity-mismatched rows. Every U/UR row then uses fixed absolute tolerance - `1.0e-5`; no reference-scale decision term, zero clamp or row denominator is - introduced. U1/U2/U3 is blocking and UR1/UR2/UR3 is warning-only. + identity-mismatched rows. Every U/UR family then uses the common reference-only scale, + near-zero ratio `0.01`, row relative tolerance `0.05` and scale-relative RMS tolerance + `0.01`. `S=0` requires exact-zero FESA values; there is no zero clamp or independent + absolute-error gate. U1/U2/U3 is blocking and UR1/UR2/UR3 is warning-only. - Source S4 and S4R select the same FESA MITC4 kernel/quadrature/recovery path while preserving source type. This is an input mapping, not an Abaqus formulation, integration, stabilization or recovery equivalence claim. diff --git a/docs/linear-static-mitc4-shell/reference-comparison.md b/docs/linear-static-mitc4-shell/reference-comparison.md index 918c4e1..5d5b974 100644 --- a/docs/linear-static-mitc4-shell/reference-comparison.md +++ b/docs/linear-static-mitc4-shell/reference-comparison.md @@ -11,10 +11,39 @@ `docs/linear-static-mitc4-shell/reference-model.md` - source_io_definition: `docs/linear-static-mitc4-shell/io.md` - source_requirements: `docs/linear-static-mitc4-shell/requirements.md` -- status: `pass-for-physics-evaluation` +- historical_status: `pass-for-physics-evaluation` +- current_status: `pass-for-physics-evaluation` +- superseded_on: `2026-08-18` +- revalidated_on: `2026-08-18` - owner_agent: `reference-verification-agent` - date: `2026-08-13` +The fixed-absolute-tolerance evidence below is preserved as a historical record. ADR-022 +supersedes that decision with the common family-scale row/RMS policy. The following revalidation +is the current reference-gate evidence. + +## 2026-08-18 Common-policy Revalidation + +`cmake --build .harness/build --config Debug` passed, followed by +`ctest --test-dir .harness/build -C Debug --output-on-failure`: 214/214 tests passed. +The S4 reference test generated +`.harness/build/reference/mitc4-shell-s4-comparison/comparison.json` from the declared +read-only input/CSV pair. + +- exact canonical rows: 294 +- blocking translation family: 147 rows, all row/RMS gates passed +- warning-only rotation family: 147 rows, all row/RMS gates passed; warnings 0 +- identity/schema/nonfinite failures: 0 +- overall verdict: `passed=true` + +| family | rows | scale | near-zero rows | max absolute error | scale-relative RMS | +| --- | ---: | ---: | ---: | ---: | ---: | +| translation `U1/U2/U3` | 147 | `2.37408203e-5` | 122 | `1.903785349151439e-7` | `1.0332756650397166e-3` | +| rotation `UR1/UR2/UR3` | 147 | `7.60725743e-6` | 107 | `6.882854962740428e-8` | `2.9771372973060382e-3` | + +All values above come from the generated JSON ledger. The historical report body starts below +and must not be read as the current identity/tolerance evidence. + The prerequisite build/test report has status `pass-for-reference-verification`. This report verifies only the approved full-integration S4 displacement case. It does not compare S4R artifacts, claim Abaqus formulation equivalence, approve diff --git a/docs/linear-static-mitc4-shell/reference-model.md b/docs/linear-static-mitc4-shell/reference-model.md index 3d5ddc8..2008929 100644 --- a/docs/linear-static-mitc4-shell/reference-model.md +++ b/docs/linear-static-mitc4-shell/reference-model.md @@ -8,6 +8,7 @@ - status: `approved-for-implementation-planning` - owner_agent: `reference-model-agent` - date: `2026-08-13` +- comparison_revalidated_on: `2026-08-18` - artifact_policy: `read-only-existing-files` - authoritative_fesa_output: `results.h5` @@ -80,26 +81,31 @@ zero-clamped and mismatched rows are not omitted, averaged, or synthesized. ## 5. Tolerance and decision rule -For every matched row: +Matched `U1/U2/U3` rows form one blocking translation family and matched +`UR1/UR2/UR3` rows form one warning-only rotation family. For each family: ```text -tolerance = 1.0e-5 -absolute_error_i = abs(fesa_value_i - abaqus_value_i) +S = max(abs(abaqus_value_i)) +absolute_error_i = abs(fesa_value_i-abaqus_value_i) +near_zero_band = 0.01*S + +near-zero row: absolute_error_i <= near_zero_band +other row: absolute_error_i/abs(abaqus_value_i) <= 0.05 +relative_rms = sqrt(mean(absolute_error_i^2))/S <= 0.01 ``` -The fixed `1.0e-5` value is expressed in the model's user-consistent length unit for -`U1/U2/U3` and is dimensionless for `UR1/UR2/UR3`. Neither a component reference scale -nor a row-specific denominator changes the value. Reference scale may be reported as -non-decision diagnostic information. The separate B33 mixed tolerance is unchanged. +The scale uses read-only Abaqus values only. No independent absolute-error gate, zero clamp or +row omission is permitted. A zero-scale family passes only when all FESA values are exactly zero; +otherwise it reports `zero-reference-scale-nonzero-error` without NaN/Inf. The U family controls +the verdict and the UR family uses the same numeric rule for deterministic warning-only evidence. -- Every matched `U1/U2/U3` row must satisfy `absolute_error_i <= tolerance`. - Any U exceedance fails that case and the feature reference comparison. -- `UR1/UR2/UR3` uses the same fixed value. Every exceedance produces a deterministic - warning containing the case, source row, component, error, and tolerance, but does - not change pass/fail. +- Every matched `U1/U2/U3` row and the translation family RMS must satisfy the common + row/RMS rule. Any U row or family-RMS exceedance fails the feature reference comparison. +- `UR1/UR2/UR3` uses the same family-scale row/RMS rule. Every row or family-RMS + exceedance produces deterministic warning evidence but does not change pass/fail. -The comparison report records every U/UR row decision, maximum absolute error, -fixed-tolerance-normalized error, RMS error, vector-norm error, worst source +The comparison report records every U/UR row decision, family scale, near-zero branch, +maximum absolute error, row relative error, scale-relative RMS, worst source row/component, and every UR warning. ## 6. Coverage and handoff diff --git a/docs/linear-static-mitc4-shell/release.md b/docs/linear-static-mitc4-shell/release.md index 4b604b7..24edf9d 100644 --- a/docs/linear-static-mitc4-shell/release.md +++ b/docs/linear-static-mitc4-shell/release.md @@ -18,11 +18,17 @@ - source_physics_evaluation_report: `docs/linear-static-mitc4-shell/physics-evaluation.md` - audited_source_head: `820ba30c717b3d0e113775608e20dfd5fbc05d53` - audited_branch: `feat-linear-static-mitc4-shell` -- status: `ready-for-release` +- historical_status: `ready-for-release` +- current_status: `superseded-pending-revalidation` +- superseded_on: `2026-08-18` - owner_agent: `release-agent` - date: `2026-08-13` - release_boundary: internal FESA feature release readiness only; no publish, deploy, package, tag, commit, or external release was performed +The readiness evidence below is historical. The 2026-08-18 common-policy reference revalidation +passed, but fresh physics and release audits are still required before a current +`ready-for-release` verdict may be issued. + ## Release Scope | item | included | excluded | notes | diff --git a/docs/linear-static-mitc4-shell/requirements.md b/docs/linear-static-mitc4-shell/requirements.md index 728538b..58502ff 100644 --- a/docs/linear-static-mitc4-shell/requirements.md +++ b/docs/linear-static-mitc4-shell/requirements.md @@ -184,13 +184,13 @@ Formulation, Numerical Review, I/O, Reference Model, Implementation Planning 및 ## Tolerance Policy -- **FESA-REQ-LSMITC4-058** — Abaqus reference pass/fail shall apply only to matched global `U1/U2/U3` rows using the fixed absolute tolerance `abs(fesa-reference) <= 1.0e-5` for every row. -- **FESA-REQ-LSMITC4-059** — MITC4 row tolerance and tolerance-normalized error shall not depend on `reference_scale`, a row-specific denominator, zero clamp or component magnitude; a reference scale may be reported only as non-decision diagnostic information. -- **FESA-REQ-LSMITC4-060** — The fixed `1.0e-5` U tolerance is expressed in the model's user-consistent length unit and is independent of the approved B33 component-scale mixed tolerance. -- **FESA-REQ-LSMITC4-061** — Global `UR1/UR2/UR3` rows shall use the same fixed absolute value `1.0e-5` and shall be fully reported; an exceedance emits a deterministic nonblocking warning and never changes pass/fail. -- **FESA-REQ-LSMITC4-062** — The fixed `1.0e-5` UR tolerance is dimensionless. No separate UR large-error or drilling-energy threshold is required. +- **FESA-REQ-LSMITC4-058** — Abaqus reference pass/fail shall apply only to matched global `U1/U2/U3` rows grouped as one blocking translation family with Abaqus-only scale `S=max(abs(reference))`. +- **FESA-REQ-LSMITC4-059** — A U row with `abs(reference)<=0.01*S` shall pass when `abs(fesa-reference)<=0.01*S`; every other U row shall pass when its relative error is at most `0.05`; the U family shall also satisfy `RMS(error)/S<=0.01`. +- **FESA-REQ-LSMITC4-060** — MITC4 shall use the project-wide family-scale policy without an independent absolute-error gate, zero clamp or row omission; a zero-scale family passes only when every FESA value is exactly zero. +- **FESA-REQ-LSMITC4-061** — Global `UR1/UR2/UR3` rows shall form one warning-only rotation family using the same near-zero, relative-row and scale-relative RMS constants; an exceedance emits a deterministic nonblocking warning and never changes pass/fail. +- **FESA-REQ-LSMITC4-062** — No separate UR large-error, fixed absolute or drilling-energy threshold is required; every row decision and family metric shall remain finite and auditable. - **FESA-REQ-LSMITC4-063** — Missing, extra, duplicate, nonfinite, schema-mismatched or source-identity-mismatched rows shall fail artifact/schema validation before numeric tolerance evaluation for both U and UR inventories. -- **FESA-REQ-LSMITC4-064** — The comparison report shall record each U/UR row decision, maximum absolute error, fixed-tolerance-normalized error, RMS error, vector-norm error and worst source row/component; nonblocking UR warnings shall not be omitted from an otherwise passing report. +- **FESA-REQ-LSMITC4-064** — The comparison report shall record each U/UR row branch/decision, family identity/components, scale, near-zero band/count, maximum absolute error, scale-relative RMS, vector-norm diagnostic, worst source row/component and zero-scale diagnostic; nonblocking UR warnings shall not be omitted from an otherwise passing report. ## Reference Artifact Requirements @@ -226,8 +226,8 @@ without gaps or overlap. | `031-038` | 5-DOF physics embedded in 6-DOF with fixed drilling stabilization | numerical boundary | User approval; MITC literature and thesis 6-DOF discussion | must | Formulation review, invariant and rank tests | Exact `10^-3` positive rotational-diagonal rule; physical outputs exclude drilling | Fixed by Requirements 033-036 | Formulation; Numerical Review; Implementation Planning | approved | | `039-048` | Mandatory HDF5 output and failure atomicity | output | User approval; ADR-005/016/018 | must | Recovery, schema, identity, nonfinite and atomicity tests | Every quantity/location/unit/identity exists; failure commits no partial success | Exact component/location inventory; I/O Definition owns schema | Formulation; I/O Definition; Implementation Planning | approved | | `049-057` | TDD, invariants, patch, declared reference and physics | verification | User approval; shell formulation evidence; project process | must | CTest evidence, analytical/patch tests, the S4 reference case and physics review | Required tests pass; removed calibration/portfolio checks are not reintroduced | `1e-12` symmetry/frame; `1e-10` rigid/residual | Numerical Review; Implementation Planning | approved | -| `058-060` | Translational displacement pass/fail tolerance | tolerance | User approval; ADR-020 | must | Comparator unit/integration tests and report review | Every matched U row uses fixed absolute `1.0e-5` without scale, clamp or omission | Fixed by Requirements 058-060 | Reference Verification | approved | -| `061-062` | Rotational warning-only comparison | tolerance/warning | User approval; ADR-020 | must | Comparator/diagnostic tests and report review | UR never changes pass/fail; fixed absolute `1.0e-5` exceedance emits a deterministic warning | Fixed by Requirements 061-062 | Reference Verification | approved | +| `058-060` | Translational displacement pass/fail tolerance | tolerance | User approval; ADR-022 | must | Comparator unit/integration tests and report review | Every matched U row and U-family RMS use the common family-scale policy without clamp or omission | `0.01`, `0.05`, `0.01` | Reference Verification | approved | +| `061-062` | Rotational warning-only comparison | tolerance/warning | User approval; ADR-022 | must | Comparator/diagnostic tests and report review | UR never changes pass/fail; common-policy exceedance emits a deterministic warning | `0.01`, `0.05`, `0.01` | Reference Verification | approved | | `063-064` | Row/schema failure and report completeness | reference verification | User approval; ADR-005/014/018 | must | Negative comparator and report-schema tests | Invalid inventory fails before numeric comparison; all U/UR metrics remain visible | No ignored invalid rows | I/O Definition; Reference Verification | approved | | `065-068` | Exact S4 reference-case inventory, S4R exclusion and row validity | reference | User declaration; ADR-019 | must | Read-only S4 inventory, source-row/component precheck and S4R non-consumption test | Two declared S4 paths exist; required rows are unique, finite and deterministically mapped; S4R artifacts are not required or consumed | Requirements `058-063` | Reference Model; Reference Verification | approved | | `069-071` | S4 reference coverage and displacement-only comparison | reference | User approval | must | HDF5-to-CSV comparison | Declared S4 case only; U blocks and UR only warns; S4R mapping remains independently tested | Requirements `058-064` | Reference Verification; Physics Evaluation | approved | @@ -270,7 +270,7 @@ and tangent derivation may remain in the formulation document. - Record only the exact existing S4 input/displacement CSV paths from Requirement 065 as acceptance artifacts and keep every existing reference artifact read-only. - Do not consume the S4R bundle in reference verification; route S4R source support to parser/common-kernel/HDF5 tests from Requirement 066. -- Define only the HDF5-to-CSV source-node/component projection and the approved fixed absolute MITC4 tolerance `1.0e-5`; do not add bundle administration or portfolio gates. +- Define only the HDF5-to-CSV source-node/component projection and the approved common family-scale tolerance; do not add bundle administration or portfolio gates. ### Implementation Planning Agent diff --git a/docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md b/docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md index 14a7f5f..ac400f9 100644 --- a/docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md +++ b/docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md @@ -7,11 +7,15 @@ - status: `approved` - approved_by: user - approved_on: `2026-08-08` -- amended_on: `2026-08-09` +- amended_on: `2026-08-18` - source_formulation: `docs/linear-static-3d-euler-beam/formulation.md` -- reference_baseline: `reference/cantilever beam/` from source commit `2b34d0b` +- reference_baseline: exact read-only artifacts under `reference/cantilever beam/` - implementation_environment: C++17, MSVC, CMake, CTest, GoogleTest, Intel oneMKL, Intel oneTBB, HDF5 +2026-08-18 amendment: the former node-station projection and mixed absolute-floor tolerance are +superseded by the direct element-endpoint identity and common family-scale row/RMS policy in +`2026-08-17-common-reference-tolerance-design.md` and ADR-022. + ## 1. 목적 이 설계는 Abaqus `.inp` keyword subset을 읽어 2절점 3차원 @@ -288,8 +292,8 @@ node label 또는 node set이며 assembly-level set의 `INSTANCE` parameter를 Abaqus B31은 transverse shear deformation을 포함하는 Timoshenko beam이고 B33은 2절점 cubic Euler–Bernoulli beam이다. FESA V0는 `TYPE=B33`만 Euler 요소로 매핑한다. `TYPE=B31`은 `unsupported-element-formulation` 오류로 거부한다. -`reference/cantilever beam/cantilever beam.inp`는 source commit `2b34d0b`에서 -`TYPE=B33`으로 생성된 승인 reference input이다. Reference artifact check는 비교 전에 +`reference/cantilever beam/cantilever beam.inp`는 `TYPE=B33`으로 생성된 승인 reference +input이다. Reference artifact check는 비교 전에 이 element type을 다시 확인하며 B31로 되돌아간 입력이나 결과를 허용하지 않는다. 이 결정은 [[Abaqus Structural Element Families]], [[Beam and Frame Finite Elements]], @@ -370,19 +374,20 @@ Abaqus internal-force CSV는 equilibrium end action이 아니라 section resulta | `SM1` | `MY` | | `SM2` | `MZ` | -승인된 B33 reference의 `cantilever beam elemental forces.csv`는 element label 없이 -`Frame`, `Part Instance Name`, `Node Label`, `SF1`, `SM1`, `SM2`, `SM3`을 기록한다. +승인된 B33 reference의 `cantilever beam elemental forces.csv`는 `Frame`, +`Part Instance Name`, `Element Label`, `Node Label`, `SF1`, `SM1`, `SM2`, `SM3`을 +기록한다. 따라서 이 파일의 comparison은 다음 규칙을 사용한다. -- Reference row key는 `(Step-1, frame 0, instance name, source node label, component)`다. +- Reference row key는 `(Step-1, frame 0, instance name, source element label, + endpoint source node label, component)`다. 단일 step invariant에 따라 CSV의 `Increment 1: Step Time = 1.000`을 `(Step-1, frame 0)`으로 정규화한다. -- FESA endpoint section resultant를 동일한 section-cut 부호로 변환한 뒤 source node - station으로 정규화한다. Interior node의 두 인접 element endpoint 값은 먼저 승인된 - tolerance 안에서 서로 일치해야 한다. 불일치는 평균으로 숨기지 않고 - `tolerance-failure`로 보고한다. -- Interior endpoint가 일치하면 stable internal element ID가 작은 endpoint를 대표 row로 - 선택한다. Boundary node는 하나의 endpoint를 사용한다. +- 각 B33 element에는 input connectivity의 두 source node label과 일치하는 endpoint row가 + 정확히 두 개 있어야 하며, 각 row를 해당 HDF5 element endpoint에 직접 대응한다. + Missing, extra, duplicate 또는 connectivity mismatch는 tolerance 전에 실패한다. +- Comparator는 같은 source node의 인접 endpoint를 node station으로 축약하거나 대표값을 + 선택하거나 평균하지 않는다. - 이 bundle에서 비교하는 mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`다. CSV에 없는 `SF2`와 `SF3`는 reference comparison 대상이 아니며, FESA equilibrium end action의 transverse force는 unit/analytical test와 physics sanity로 @@ -496,8 +501,8 @@ GoogleTest target에는 FESA warning policy를 강제하지 않는다. - axial, torsion, y/z bending cantilever analytical cases - HDF5 schema, identity, component, metadata, atomic finalization - CLI `.inp -> results.h5` integration -- 승인된 B33 CSV header/Frame 정규화, node-station matching, interior endpoint 일치 검사 -- zero 및 near-zero reference row를 포함한 component-scale 혼합 허용오차 검사 +- 승인된 B33 CSV header/Frame 정규화와 element-endpoint direct identity 검사 +- zero 및 near-zero reference row를 포함한 공통 family-scale 상대오차/RMS 검사 ### 11.3 수치 tolerance @@ -506,37 +511,33 @@ GoogleTest target에는 FESA warning policy를 강제하지 않는다. | matrix symmetry and Gauss/closed-form comparison | normalized `1e-12` | | rigid-mode and linear-system residual | normalized `1e-10` | | analytical solution tests | relative `1e-9` | -| Abaqus B33 reference comparison | component-scale relative `1e-6` | -| SI displacement and rotation absolute floor | `1e-9` | -| SI force and moment absolute floor | `1e-3` | +| Abaqus reference row relative error | `0.05` | +| near-zero threshold ratio | reference family scale의 `0.01` | +| reference family scale-relative RMS | `0.01` | -Reference row 판정은 zero-reference에서도 의미가 있도록 component-scale 혼합 -허용오차를 사용한다. 같은 model, step/frame, quantity, component의 Abaqus reference -rows에 대해 다음 값을 계산한다. +Reference row 판정은 같은 model, step/frame, quantity, component로 구성한 reference +family별 공통 규칙을 사용한다. Reference 값으로만 family scale을 계산한다. ```text -reference_scale = max(abs(reference_value_i)) -row_tolerance = absolute_floor + 1e-6 * reference_scale -row_pass = abs(fesa_value_i - reference_value_i) <= row_tolerance +S = max(abs(reference_value_i)) +near_zero_i = abs(reference_value_i) <= 0.01 * S +row_pass_i = abs(fesa_value_i - reference_value_i) / abs(reference_value_i) <= 0.05 + if not near_zero_i +row_pass_i = abs(fesa_value_i - reference_value_i) <= 0.01 * S + if near_zero_i +relative_rms = sqrt(mean((fesa_value_i - reference_value_i)^2)) / S +family_pass = all(row_pass_i) and relative_rms <= 0.01 ``` -- Scale은 displacement, rotation, force, moment 및 각 component를 섞지 않는다. +- Family는 displacement, rotation, force, moment 및 각 component를 섞지 않는다. - Scale은 read-only Abaqus reference 값만 사용하며 FESA 결과로 조정하지 않는다. -- Scale이 0이면 absolute floor만 적용한다. +- `S == 0`이면 모든 FESA 값이 정확히 0일 때만 통과한다. 하나라도 0이 아니면 + `zero-reference-scale-nonzero-error`로 실패한다. - 모든 row를 개별 판정하며 reference 값을 zero-clamp하거나 row를 제거하지 않는다. - Missing/extra row, nonfinite value, schema 또는 identity mismatch는 tolerance 계산 전에 실패한다. -- Report는 max absolute error, component-scale normalized error, RMS error, norm error, - worst row/component를 기록한다. - -현재 B33 `SM1`의 reference scale은 `1.0e7`이고 moment row tolerance는 -`1.0e-3 + 1.0e-6 * 1.0e7 = 10.001`이다. 자유단 reference residue `-1.56e-2`와 -FESA의 이론적 zero 사이 오차 `1.56e-2`는 이 기준을 통과하지만, 예를 들어 `100`의 -자유단 moment 오차는 실패한다. - -승인된 cantilever bundle은 SI absolute floor를 사용한다. 이후 SI가 아닌 reference를 -추가하면 quantity별 absolute floor를 해당 단위로 변환하고 변환 근거를 metadata와 -verification report에 기록해야 한다. +- Report는 family scale, max absolute error, max row relative error, scale-relative RMS, + worst row/component와 판정 reason을 기록한다. 독립적인 absolute-error gate는 없다. ### 11.4 공통 build/test command @@ -561,9 +562,10 @@ CTest discovery 결과는 한 개 이상의 test를 포함해야 한다. ## 12. Reference 및 release gate `reference/cantilever beam/`을 V0의 승인된 Abaqus B33 reference baseline으로 사용한다. -논리 model ID는 `cantilever-beam-b33`, reference schema ID는 -`abaqus-cae-report-csv-v0`이며 source baseline은 commit `2b34d0b`이다. Input header가 -기록한 generator는 `Abaqus/CAE Learning Edition 2024`다. 모델은 SI 단위계, global +논리 model ID는 `cantilever-beam-b33`이다. 기존 `abaqus-cae-report-csv-v0`과 source +commit `2b34d0b` 기록은 2026-08-09 당시 inventory의 역사적 provenance이며, 2026-08-18 +regenerated elemental-force CSV의 현재 readiness gate가 아니다. Input header가 기록한 +generator는 `Abaqus/CAE Learning Edition 2024`다. 모델은 SI 단위계, global Cartesian nodal output, beam local section-force output, `Step-1`, increment 1, step time 1.0 계약으로 해석한다. @@ -588,8 +590,8 @@ Reference verification은 build/test gate 통과 후 승인 input으로 FESA `re - nodal displacement: CSV `U1/U2/U3/UR1/UR2/UR3`과 HDF5 displacement - nodal reaction: CSV `RF1/RF2/RF3/RM1/RM2/RM3`과 HDF5 reaction -- section resultant: CSV `SF1/SM1/SM2/SM3`과 node station으로 정규화한 HDF5 - section resultant +- section resultant: CSV `SF1/SM1/SM2/SM3`과 element label 및 endpoint node label로 + 직접 대응한 HDF5 section resultant - stress: 명시적 N/A Artifact check는 네 파일의 존재, B33 element type, expected header, 유일한 row key, diff --git a/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md b/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md index 8599803..bfde27e 100644 --- a/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md +++ b/docs/superpowers/specs/2026-08-12-independent-reference-validation-policy-design.md @@ -3,11 +3,16 @@ ## Status - date: `2026-08-12` +- amended_on: `2026-08-18` - status: `approved` - scope: FESA project policy, agents, skills, and `linear-static-mitc4-shell` upstream contracts - implementation_code: out of scope - reference_artifact_mutation: prohibited +2026-08-18 amendment: the former MITC4 fixed absolute tolerance is superseded by the common +family-scale row/RMS policy in `2026-08-17-common-reference-tolerance-design.md` and ADR-022. +The independent-solver, minimal-reference-case and drilling decisions remain unchanged. + ## Decision FESA is an independent finite element solver. It accepts an approved subset of the @@ -84,19 +89,23 @@ Only matched global `U1/U2/U3` rows are blocking. `UR1/UR2/UR3` rows are compare reported as warning-only evidence. Reactions and stresses may be inspected but are not reference pass/fail quantities. -For every matched U/UR row: +The project-wide common policy now evaluates each matched U/UR component family: ```text -tolerance = 1.0e-5 +S = max(abs(reference_value_i)) +near_zero_i = abs(reference_value_i) <= 0.01 * S +ordinary_row_pass_i = abs(error_i) / abs(reference_value_i) <= 0.05 +near_zero_row_pass_i = abs(error_i) <= 0.01 * S +relative_rms = sqrt(mean(error_i^2)) / S <= 0.01 ``` -The fixed value is expressed in the model's user-consistent length unit for translation -and as a dimensionless rotation value for warning-only UR. Reference scale may remain -diagnostic report data but does not enter the tolerance or normalized-error denominator. -Values are not zero-clamped. Missing, duplicate, unmatched, or nonfinite required rows -fail before numeric tolerance evaluation. U tolerance failure fails reference -verification; UR tolerance exceedance emits a deterministic warning only. The B33 mixed -tolerance remains unchanged. +The scale is computed from read-only reference values in the same model, step/frame, +quantity and component family. If `S == 0`, only exact-zero FESA values pass; otherwise +the reason is `zero-reference-scale-nonzero-error`. There is no independent absolute-error +gate. Values are not zero-clamped. Missing, duplicate, unmatched, or nonfinite required +rows fail before numeric tolerance evaluation. U tolerance failure fails reference +verification; UR tolerance exceedance emits a deterministic warning only. B33 uses the +same row and family-RMS thresholds. No additional locking, convergence, distorted, curved, director-angle, or invalid geometry reference portfolio is required for this MITC4 implementation acceptance. @@ -147,7 +156,7 @@ For MITC4: - fixed drilling stabilization closes `NR-O01` and removes `NR-O02` by scope; - `NR-O03` and `NR-O04` are removed from the approved verification scope; -- the fixed absolute MITC4 tolerance `1.0e-5` closes the blocking part of `NR-O05`; +- the common family-scale row/RMS policy closes the blocking part of `NR-O05`; - missing heavyweight reference metadata is `not-required-by-policy`; - confirmed formulation defects remain blocking. diff --git a/docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md b/docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md index a102962..2b26bda 100644 --- a/docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md +++ b/docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md @@ -3,11 +3,15 @@ ## Status - date: `2026-08-17` -- status: `approved-design-pending-implementation` +- status: `implemented-and-verified` +- verified_on: `2026-08-18` - scope: 모든 FESA 외부 reference comparison - production_solver_behavior: 변경 없음 - reference_artifact_mutation: 금지 +현재 운영 상수, 검증 순서, report schema와 변경 관리는 `docs/TOLERANCE.md`를 source of +truth로 사용한다. 이 문서는 승인 결정의 설계 근거와 대안 검토를 보존한다. + ## 목적 B33, MITC4 및 이후 추가되는 기능이 서로 다른 수치 tolerance 식을 만들지 않도록 공통 @@ -67,6 +71,26 @@ S_g = \max_{i \in g}|R_i| dimension, components 및 behavior를 명시해 family에 배치한다. 물리 차원이 같다는 이유만으로 서로 다른 logical quantity를 자동으로 합치지 않는다. +### B33 element-endpoint identity amendment + +2026-08-18에 사용자가 다시 생성한 +`reference/cantilever beam/cantilever beam elemental forces.csv`는 `Element Label`과 +`Node Label`을 모두 포함하고 B33 요소마다 두 endpoint 행을 제공한다. Section-resultant +comparison은 다음 key로 각 CSV 행을 HDF5 `[element, endpoint, component]` 행에 직접 +대응시킨다. + +```text +(model_id, step/frame, instance_name, source_element_label, + source_node_label, quantity, component) +``` + +Component mapping은 `SF-SF1 -> N`, `SM-SM1 -> My`, `SM-SM2 -> Mz`, +`SM-SM3 -> T`다. Comparator는 이 CSV를 source-node station으로 collapse하거나 인접 +endpoint를 평균하지 않는다. Missing, extra, duplicate 또는 connectivity-mismatched +element-endpoint identity는 tolerance 전에 실패한다. Production +`ResultRecovery::NormalizeSectionResultantsToNodeStations` 계약과 그 단위 테스트는 이 외부 +reference identity 변경의 범위가 아니다. + ## 행별 판정 각 대응 행에 대해 다음 absolute error를 계산한다. @@ -149,15 +173,16 @@ Comparator report는 최소한 다음 항목을 보존한다. ## 현재 B33/MITC4 Evidence -기존 solver를 재실행하거나 reference artifact를 수정하지 않고 다음 생성 ledger의 원본 -FESA/reference 값을 새 규칙으로 재계산했다. +MSVC Debug build와 214개 CTest를 통과한 현재 comparator가 다음 ledger를 생성했다. +Reference artifact는 수정하지 않았으며 B33은 regenerated 20-row element-endpoint identity를 +직접 비교한다. - `.harness/build/reference/cantilever-beam-b33/comparison.json` - `.harness/build/reference/mitc4-shell-s4-comparison/comparison.json` | case | row gate | maximum family relative RMS | RMS gate | verdict | | --- | --- | ---: | --- | --- | -| B33 | 모든 blocking 행 통과 | 약 `1.17e-8` | 통과 | 통과 | +| B33 direct endpoint | 모든 212 blocking 행 통과 | 약 `1.17e-8` | 통과 | 통과 | | MITC4 U | 모든 blocking 행 통과 | 약 `1.03e-3` | 통과 | 통과 | | MITC4 UR | 모든 warning-only 행 통과 | 약 `2.98e-3` | 통과 | warning 없음 | diff --git a/reference/cantilever beam/cantilever beam elemental forces.csv b/reference/cantilever beam/cantilever beam elemental forces.csv index 3fded76..329bb20 100644 --- a/reference/cantilever beam/cantilever beam elemental forces.csv +++ b/reference/cantilever beam/cantilever beam elemental forces.csv @@ -1,12 +1,21 @@ -Frame,Part Instance Name, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3 -Increment 1: Step Time = 1.000,PART-1_1-1,1,0.000000000E+00,1.000000000E+07,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,10,0.000000000E+00,1.000000000E+06,0.000000000E+00,0.000000000E+00 -Increment 1: Step Time = 1.000,PART-1_1-1,11,0.000000000E+00,-1.560000000E-02,0.000000000E+00,0.000000000E+00 +Frame,Part Instance Name, Element Label, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3 +Increment 1: Step Time = 1.000,PART-1_1-1,1,1,0.000000000E+00,1.000000000E+07,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,1,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,2,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,2,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,3,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,3,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,4,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,4,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,5,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,5,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,6,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,6,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,7,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,7,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,8,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,8,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,9,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,9,10,0.000000000E+00,9.999998750E+05,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,10,10,0.000000000E+00,9.999998750E+05,0.000000000E+00,0.000000000E+00 +Increment 1: Step Time = 1.000,PART-1_1-1,10,11,0.000000000E+00,-1.562500000E-02,0.000000000E+00,0.000000000E+00 diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 8c4f4cd..ef28eb2 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -76,6 +76,8 @@ target_link_libraries( add_executable( fesa_reference_tests + reference/reference_tolerance_policy.cpp + reference/reference_tolerance_policy_test.cpp reference/reference_comparison.cpp reference/reference_comparison_test.cpp reference/b33_reference_comparison_test.cpp diff --git a/tests/reference/b33_reference_comparison_test.cpp b/tests/reference/b33_reference_comparison_test.cpp index cf46735..8e2908a 100644 --- a/tests/reference/b33_reference_comparison_test.cpp +++ b/tests/reference/b33_reference_comparison_test.cpp @@ -26,8 +26,8 @@ namespace { constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11"; -constexpr std::size_t kExpectedRowCount = 176U; -constexpr std::size_t kExpectedMetricCount = 16U; +constexpr std::size_t kExpectedRowCount = 212U; +constexpr std::size_t kExpectedMetricCount = 6U; class Hdf5Handle { public: @@ -192,7 +192,10 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) { report.rows.begin(), report.rows.end(), [](const fesa::test::RowDecision& row) { return row.passed && std::isfinite(row.absolute_error) && - std::isfinite(row.tolerance) && row.tolerance > 0.0 && + std::isfinite(row.tolerance) && row.tolerance >= 0.0 && + std::isfinite(row.reference_scale) && + std::isfinite(row.near_zero_band) && + std::isfinite(row.relative_error) && row.fesa.model_id == "cantilever-beam-b33" && row.reference.model_id == "cantilever-beam-b33" && row.fesa.step_name == "Step-1" && @@ -206,11 +209,11 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) { std::all_of(report.metrics.begin(), report.metrics.end(), [](const fesa::test::ComponentMetrics& metric) { return std::isfinite(metric.reference_scale) && + std::isfinite(metric.near_zero_band) && std::isfinite(metric.maximum_absolute_error) && - std::isfinite(metric.maximum_normalized_error) && - std::isfinite(metric.rms_error) && - std::isfinite(metric.norm_error) && - metric.maximum_normalized_error <= 1.0; + std::isfinite(metric.relative_rms) && + metric.rms_passed && metric.passed && + metric.row_count > 0U && !metric.components.empty(); })); EXPECT_FALSE(report.stress_comparison_applicable); @@ -234,6 +237,9 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) { EXPECT_NE(json.find("\"stress_comparison_applicable\":false"), std::string::npos); EXPECT_NE(json.find("\"physics_evidence\""), std::string::npos); + EXPECT_NE(json.find("\"source_element_label\""), std::string::npos); + EXPECT_NE(json.find("\"family_identity\""), std::string::npos); + EXPECT_NE(json.find("\"row_count\""), std::string::npos); std::vector generated_names; for (const auto& entry : diff --git a/tests/reference/mitc4_reference_cases_test.cpp b/tests/reference/mitc4_reference_cases_test.cpp index b078cf6..118a420 100644 --- a/tests/reference/mitc4_reference_cases_test.cpp +++ b/tests/reference/mitc4_reference_cases_test.cpp @@ -122,7 +122,7 @@ void ExpectCommonMetadata(const fesa::test::Mitc4ComparisonReport& report, void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) { ASSERT_EQ(report.rows.size(), kNodeCount * kComponentCount); - ASSERT_EQ(report.metrics.size(), kComponentCount); + ASSERT_EQ(report.metrics.size(), 2U); ASSERT_EQ(report.vector_metrics.size(), kNodeCount); EXPECT_TRUE(report.passed); const std::size_t blocking_rows = static_cast(std::count_if( @@ -135,12 +135,35 @@ void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) { [](const fesa::test::Mitc4RowDecision& row) { return !row.blocking || row.within_tolerance; })); - EXPECT_EQ(report.warnings.size(), - static_cast( - std::count_if(report.rows.begin(), report.rows.end(), - [](const fesa::test::Mitc4RowDecision& row) { - return !row.blocking && !row.within_tolerance; - }))); + const auto rotation_metric = + std::find_if(report.metrics.begin(), report.metrics.end(), + [](const fesa::test::Mitc4ComponentMetrics& metric) { + return !metric.blocking; + }); + ASSERT_NE(rotation_metric, report.metrics.end()); + const std::size_t row_warning_count = static_cast( + std::count_if(report.rows.begin(), report.rows.end(), + [](const fesa::test::Mitc4RowDecision& row) { + return !row.blocking && !row.within_tolerance; + })); + ASSERT_EQ(report.warnings.size(), + row_warning_count + (rotation_metric->rms_passed ? 0U : 1U)); + std::size_t warning_index = 0U; + for (std::size_t row_index = 0U; row_index < report.rows.size(); + ++row_index) { + const auto& row = report.rows[row_index]; + if (!row.blocking && !row.within_tolerance) { + EXPECT_EQ(report.warnings[warning_index].code, + "rotation-reference-exceedance"); + EXPECT_EQ(report.warnings[warning_index].row, row_index); + ++warning_index; + } + } + if (!rotation_metric->rms_passed) { + EXPECT_EQ(report.warnings[warning_index].code, + "rotation-reference-rms-exceedance"); + EXPECT_EQ(report.warnings[warning_index].row, rotation_metric->worst_row); + } } // MITC4-E2E-S4-001 diff --git a/tests/reference/mitc4_reference_comparison.cpp b/tests/reference/mitc4_reference_comparison.cpp index 3aa44e1..bc8100c 100644 --- a/tests/reference/mitc4_reference_comparison.cpp +++ b/tests/reference/mitc4_reference_comparison.cpp @@ -24,6 +24,8 @@ #include #include +#include "reference_tolerance_policy.h" + namespace fesa::test { namespace { @@ -32,7 +34,6 @@ constexpr const char* kDisplacementPath = constexpr const char* kInternalFormulation = "FESA-MITC4"; constexpr const char* kIntegrationRule = "2x2x2-gauss; mitc4-edge-midpoint-shear"; -constexpr double kFixedAbsoluteTolerance = 1.0e-5; constexpr std::array kComponents{"U1", "U2", "U3", "UR1", "UR2", "UR3"}; const std::vector expected_header{"Part Instance Name", @@ -648,6 +649,18 @@ std::string FiniteText(const double value) { return stream.str(); } +const char* BranchName(const ReferenceToleranceBranch branch) { + switch (branch) { + case ReferenceToleranceBranch::kNearZero: + return "near-zero"; + case ReferenceToleranceBranch::kRelative: + return "relative"; + case ReferenceToleranceBranch::kZeroScaleExact: + return "zero-scale-exact"; + } + return "unknown"; +} + std::string JsonEscape(const std::string& value) { std::ostringstream stream; for (const unsigned char character : value) { @@ -687,6 +700,18 @@ std::string JsonEscape(const std::string& value) { return stream.str(); } +void WriteJsonStrings(std::ostream& stream, + const std::vector& values) { + stream << '['; + for (std::size_t index = 0U; index < values.size(); ++index) { + if (index != 0U) { + stream << ','; + } + stream << '"' << JsonEscape(values[index]) << '"'; + } + stream << ']'; +} + } // namespace Result Mitc4ReferenceComparison::Compare( @@ -713,14 +738,6 @@ Result Mitc4ReferenceComparison::Compare( } } - std::array scales{}; - for (const auto& row : reference_rows) { - for (std::size_t component = 0U; component < scales.size(); ++component) { - scales[component] = - (std::max)(scales[component], std::abs(row.values[component])); - } - } - Mitc4ComparisonReport report{}; report.case_id = reference_case.case_id; report.source_element_type = std::move(hdf5.source_element_type); @@ -729,80 +746,126 @@ Result Mitc4ReferenceComparison::Compare( report.passed = true; report.rows.reserve(hdf5.rows.size() * kComponents.size()); report.vector_metrics.reserve(hdf5.rows.size()); - std::array error_norms{}; - std::array maximum_errors{}; - std::array maximum_normalized{}; - std::array worst_rows{}; - double global_worst_normalized = -1.0; + std::array, 2> family_rows; for (const auto& hdf5_row : hdf5.rows) { const auto reference = reference_by_identity.find(hdf5_row.identity); if (reference == reference_by_identity.end()) { Fail("schema-mismatch", "A projected reference row is missing."); } - std::array errors{}; for (std::size_t component = 0U; component < kComponents.size(); ++component) { - const double tolerance = kFixedAbsoluteTolerance; - const double absolute_error = std::abs( - hdf5_row.values[component] - reference->second->values[component]); - const double normalized_error = absolute_error / tolerance; - if (!std::isfinite(tolerance) || !(tolerance > 0.0) || - !std::isfinite(absolute_error) || - !std::isfinite(normalized_error)) { - Fail("schema-mismatch", - "A finite row produced a nonfinite comparison metric."); - } const bool blocking = component < 3U; - const bool within_tolerance = absolute_error <= tolerance; const std::size_t row_index = report.rows.size(); - report.rows.push_back( - {reference_case.case_id, hdf5_row.identity.instance_name, - hdf5_row.identity.source_node_label, kComponents[component], - hdf5_row.values[component], reference->second->values[component], - absolute_error, tolerance, normalized_error, blocking, - within_tolerance}); - errors[component] = absolute_error; - error_norms[component] = - std::hypot(error_norms[component], absolute_error); - if (normalized_error > maximum_normalized[component]) { - maximum_normalized[component] = normalized_error; - maximum_errors[component] = absolute_error; - worst_rows[component] = row_index; - } - if (normalized_error > global_worst_normalized) { - global_worst_normalized = normalized_error; - report.worst_row = row_index; - } - if (blocking && !within_tolerance) { - report.passed = false; - } else if (!blocking && !within_tolerance) { - const std::string message = - "case=" + reference_case.case_id + - ";instance=" + hdf5_row.identity.instance_name + - ";node=" + std::to_string(hdf5_row.identity.source_node_label) + - ";component=" + kComponents[component] + - ";absolute_error=" + FiniteText(absolute_error) + - ";tolerance=" + FiniteText(tolerance); - report.warnings.push_back( - {"rotation-reference-exceedance", row_index, message}); - } + report.rows.push_back({reference_case.case_id, + hdf5_row.identity.instance_name, + hdf5_row.identity.source_node_label, + kComponents[component], + hdf5_row.values[component], + reference->second->values[component], + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + false, + {}, + blocking, + false}); + family_rows[blocking ? 0U : 1U].push_back(row_index); } - report.vector_metrics.push_back( - {hdf5_row.identity.instance_name, hdf5_row.identity.source_node_label, - std::hypot(errors[0U], errors[1U], errors[2U]), - std::hypot(errors[3U], errors[4U], errors[5U])}); } - report.metrics.reserve(kComponents.size()); - const double row_count = static_cast(hdf5.rows.size()); - for (std::size_t component = 0U; component < kComponents.size(); - ++component) { + std::vector families; + families.reserve(2U); + for (std::size_t family_index = 0U; family_index < family_rows.size(); + ++family_index) { + ReferenceToleranceFamily family{}; + family.identity = family_index == 0U ? "displacement-translation" + : "displacement-rotation"; + family.components = family_index == 0U + ? std::vector{"U1", "U2", "U3"} + : std::vector{"UR1", "UR2", "UR3"}; + family.blocking = family_index == 0U; + family.rows.reserve(family_rows[family_index].size()); + for (const std::size_t row_index : family_rows[family_index]) { + const auto& row = report.rows[row_index]; + family.rows.push_back({row.fesa_value, row.reference_value}); + } + families.push_back(std::move(family)); + } + const auto evaluation = ReferenceTolerancePolicy::Evaluate(families); + if (!evaluation.valid || evaluation.families.size() != families.size()) { + Fail("schema-mismatch", + "A finite row produced a nonfinite comparison metric."); + } + + report.metrics.reserve(evaluation.families.size()); + double maximum_absolute_error = -1.0; + for (std::size_t family_index = 0U; + family_index < evaluation.families.size(); ++family_index) { + const auto& policy = evaluation.families[family_index]; + for (std::size_t local = 0U; local < family_rows[family_index].size(); + ++local) { + const std::size_t row_index = family_rows[family_index][local]; + auto& row = report.rows[row_index]; + const auto& decision = policy.rows[local]; + row.absolute_error = decision.absolute_error; + row.tolerance = decision.threshold; + row.normalized_error = decision.relative_error; + row.reference_scale = policy.reference_scale; + row.near_zero_band = policy.near_zero_band; + row.relative_error_applicable = decision.relative_error_applicable; + row.tolerance_branch = BranchName(decision.branch); + row.within_tolerance = decision.passed; + if (row.absolute_error > maximum_absolute_error) { + maximum_absolute_error = row.absolute_error; + report.worst_row = row_index; + } + if (!row.within_tolerance) { + if (row.blocking) { + report.passed = false; + } else { + const std::string message = + "case=" + reference_case.case_id + + ";instance=" + row.instance_name + + ";node=" + std::to_string(row.source_node_label) + + ";component=" + row.component + + ";absolute_error=" + FiniteText(row.absolute_error) + + ";threshold=" + FiniteText(row.tolerance); + report.warnings.push_back( + {"rotation-reference-exceedance", row_index, message}); + } + } + } + const std::size_t worst_row = family_rows[family_index][policy.worst_row]; report.metrics.push_back( - {kComponents[component], scales[component], kFixedAbsoluteTolerance, - maximum_errors[component], maximum_normalized[component], - error_norms[component] / std::sqrt(row_count), - error_norms[component], worst_rows[component]}); + {policy.identity, policy.components, policy.blocking, + policy.reference_scale, policy.near_zero_band, + policy.near_zero_count, policy.maximum_absolute_error, + policy.relative_rms, worst_row, policy.rms_passed, policy.passed, + policy.diagnostic_code, policy.row_count}); + if (policy.blocking && !policy.passed) { + report.passed = false; + } + if (!policy.blocking && !policy.rms_passed) { + report.warnings.push_back( + {"rotation-reference-rms-exceedance", worst_row, + "case=" + reference_case.case_id + ";family=" + policy.identity + + ";relative_rms=" + FiniteText(policy.relative_rms)}); + } + } + for (std::size_t node = 0U; node < hdf5.rows.size(); ++node) { + const std::size_t row_offset = node * kComponents.size(); + report.vector_metrics.push_back( + {report.rows[row_offset].instance_name, + report.rows[row_offset].source_node_label, + std::hypot(report.rows[row_offset].absolute_error, + report.rows[row_offset + 1U].absolute_error, + report.rows[row_offset + 2U].absolute_error), + std::hypot(report.rows[row_offset + 3U].absolute_error, + report.rows[row_offset + 4U].absolute_error, + report.rows[row_offset + 5U].absolute_error)}); } return Result::Success(std::move(report)); } catch (const ComparisonFailure& exception) { @@ -845,8 +908,14 @@ Status Mitc4ReferenceComparison::WriteDeterministicJson( << "\",\"fesa_value\":" << row.fesa_value << ",\"reference_value\":" << row.reference_value << ",\"absolute_error\":" << row.absolute_error - << ",\"tolerance\":" << row.tolerance - << ",\"normalized_error\":" << row.normalized_error + << ",\"threshold\":" << row.tolerance + << ",\"relative_error\":" << row.normalized_error + << ",\"reference_scale\":" << row.reference_scale + << ",\"near_zero_band\":" << row.near_zero_band + << ",\"relative_error_applicable\":" + << (row.relative_error_applicable ? "true" : "false") + << ",\"tolerance_branch\":\"" << JsonEscape(row.tolerance_branch) + << "\"" << ",\"blocking\":" << (row.blocking ? "true" : "false") << ",\"within_tolerance\":" << (row.within_tolerance ? "true" : "false") << '}'; @@ -857,15 +926,21 @@ Status Mitc4ReferenceComparison::WriteDeterministicJson( stream << ','; } const auto& metric = report.metrics[index]; - stream << "{\"component\":\"" << JsonEscape(metric.component) - << "\",\"reference_scale\":" << metric.reference_scale - << ",\"tolerance\":" << metric.tolerance + stream << "{\"family_identity\":\"" << JsonEscape(metric.family_identity) + << "\",\"components\":"; + WriteJsonStrings(stream, metric.components); + stream << ",\"blocking\":" << (metric.blocking ? "true" : "false") + << ",\"reference_scale\":" << metric.reference_scale + << ",\"near_zero_band\":" << metric.near_zero_band + << ",\"near_zero_count\":" << metric.near_zero_count + << ",\"row_count\":" << metric.row_count << ",\"maximum_absolute_error\":" << metric.maximum_absolute_error - << ",\"maximum_normalized_error\":" - << metric.maximum_normalized_error - << ",\"rms_error\":" << metric.rms_error - << ",\"vector_norm_error\":" << metric.vector_norm_error - << ",\"worst_row\":" << metric.worst_row << '}'; + << ",\"relative_rms\":" << metric.relative_rms + << ",\"worst_row\":" << metric.worst_row + << ",\"rms_passed\":" << (metric.rms_passed ? "true" : "false") + << ",\"passed\":" << (metric.passed ? "true" : "false") + << ",\"diagnostic_code\":\"" << JsonEscape(metric.diagnostic_code) + << "\"}"; } stream << "],\"vector_metrics\":["; for (std::size_t index = 0U; index < report.vector_metrics.size(); diff --git a/tests/reference/mitc4_reference_comparison.h b/tests/reference/mitc4_reference_comparison.h index 2431d27..2fb07b8 100644 --- a/tests/reference/mitc4_reference_comparison.h +++ b/tests/reference/mitc4_reference_comparison.h @@ -29,19 +29,28 @@ struct Mitc4RowDecision { double absolute_error; double tolerance; double normalized_error; + double reference_scale; + double near_zero_band; + bool relative_error_applicable; + std::string tolerance_branch; bool blocking; bool within_tolerance; }; struct Mitc4ComponentMetrics { - std::string component; + std::string family_identity; + std::vector components; + bool blocking; double reference_scale; - double tolerance; + double near_zero_band; + std::size_t near_zero_count; double maximum_absolute_error; - double maximum_normalized_error; - double rms_error; - double vector_norm_error; + double relative_rms; std::size_t worst_row; + bool rms_passed; + bool passed; + std::string diagnostic_code; + std::size_t row_count{}; }; struct Mitc4VectorMetrics { diff --git a/tests/reference/mitc4_reference_comparison_test.cpp b/tests/reference/mitc4_reference_comparison_test.cpp index a2dc50b..a65397f 100644 --- a/tests/reference/mitc4_reference_comparison_test.cpp +++ b/tests/reference/mitc4_reference_comparison_test.cpp @@ -458,11 +458,11 @@ const fesa::test::Mitc4RowDecision* FindRow( const fesa::test::Mitc4ComponentMetrics* FindMetric( const fesa::test::Mitc4ComparisonReport& report, - const std::string& component) { + const std::string& family_identity) { const auto found = std::find_if(report.metrics.begin(), report.metrics.end(), [&](const fesa::test::Mitc4ComponentMetrics& metric) { - return metric.component == component; + return metric.family_identity == family_identity; }); return found == report.metrics.end() ? nullptr : &*found; } @@ -496,6 +496,9 @@ TEST(Mitc4ReferenceComparison, EXPECT_EQ(report.rows[0U].component, "U1"); EXPECT_EQ(report.rows[5U].component, "UR3"); EXPECT_EQ(report.rows[6U].source_node_label, 2); + ASSERT_EQ(report.metrics.size(), 2U); + EXPECT_EQ(report.metrics[0U].row_count, 6U); + EXPECT_EQ(report.metrics[1U].row_count, 6U); EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(), [](const fesa::test::Mitc4RowDecision& row) { return row.case_id == "shell-contract" && @@ -588,7 +591,7 @@ TEST(Mitc4ReferenceComparison, RejectsInvalidInventoryBeforeNumericComparison) { // MITC4-REF-003 TEST(Mitc4ReferenceComparison, - AppliesFixedAbsoluteToleranceWithoutScaleClampOrRowDenominator) { + SharesTranslationScaleAcrossUComponentsAndUsesNearZeroBranches) { ContractFixture fixture{"tolerance"}; auto reference = DefaultRows(); reference[0U].values[0U] = 0.0; @@ -597,10 +600,10 @@ TEST(Mitc4ReferenceComparison, reference[1U].values[1U] = 0.0; WriteCsv(fixture.Csv(), reference); auto fesa_values = reference; - fesa_values[0U].values[0U] = 0.9999e-5; - fesa_values[1U].values[0U] += 1.0001e-5; - fesa_values[0U].values[1U] = 0.9999e-5; - fesa_values[1U].values[1U] = 1.0001e-5; + fesa_values[0U].values[0U] = 0.999 * 8.0e-2; + fesa_values[1U].values[0U] += 1.001e-1; + fesa_values[0U].values[1U] = 0.999 * 8.0e-2; + fesa_values[1U].values[1U] = 1.001 * 8.0e-2; WriteHdf5(fixture.Results(), fixture.Input(), fesa_values); auto result = @@ -616,25 +619,21 @@ TEST(Mitc4ReferenceComparison, ASSERT_NE(u1_scaled, nullptr); ASSERT_NE(u2_near, nullptr); ASSERT_NE(u2_over, nullptr); - EXPECT_DOUBLE_EQ(u1_zero->tolerance, 1.0e-5); + EXPECT_DOUBLE_EQ(u1_zero->tolerance, 8.0e-2); EXPECT_TRUE(u1_zero->within_tolerance); - EXPECT_NEAR(u1_zero->normalized_error, 0.9999, 1.0e-12); + EXPECT_FALSE(u1_zero->relative_error_applicable); EXPECT_FALSE(u1_scaled->within_tolerance); - EXPECT_NEAR(u1_scaled->normalized_error, 1.0001, 2.0e-11); - EXPECT_DOUBLE_EQ(u2_near->tolerance, 1.0e-5); + EXPECT_NEAR(u1_scaled->normalized_error, 0.05005, 1.0e-12); + EXPECT_DOUBLE_EQ(u2_near->tolerance, 8.0e-2); EXPECT_TRUE(u2_near->within_tolerance); - EXPECT_NEAR(u2_near->normalized_error, 0.9999, 1.0e-12); + EXPECT_FALSE(u2_near->relative_error_applicable); EXPECT_FALSE(u2_over->within_tolerance); - EXPECT_NEAR(u2_over->normalized_error, 1.0001, 1.0e-12); - for (const auto& row : report.rows) { - EXPECT_DOUBLE_EQ(row.tolerance, 1.0e-5); - } - for (const auto& metric : report.metrics) { - EXPECT_DOUBLE_EQ(metric.tolerance, 1.0e-5); - } - const auto* u1_metric = FindMetric(report, "U1"); - ASSERT_NE(u1_metric, nullptr); - EXPECT_DOUBLE_EQ(u1_metric->reference_scale, 2.0); + EXPECT_DOUBLE_EQ(u2_over->tolerance, 8.0e-2); + const auto* translation = FindMetric(report, "displacement-translation"); + ASSERT_NE(translation, nullptr); + EXPECT_DOUBLE_EQ(translation->reference_scale, 8.0); + EXPECT_EQ(translation->components, + (std::vector{"U1", "U2", "U3"})); } // MITC4-REF-004 @@ -650,8 +649,9 @@ TEST(Mitc4ReferenceComparison, ASSERT_TRUE(result.HasValue()); const auto& report = result.Value(); EXPECT_TRUE(report.passed); - ASSERT_EQ(report.warnings.size(), 1U); + ASSERT_EQ(report.warnings.size(), 2U); EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance"); + EXPECT_EQ(report.warnings[1U].code, "rotation-reference-rms-exceedance"); const auto* row = FindRow(report, 1, "UR1"); ASSERT_NE(row, nullptr); EXPECT_FALSE(row->blocking); @@ -663,6 +663,47 @@ TEST(Mitc4ReferenceComparison, EXPECT_NE(report.warnings[0U].message.find("UR1"), std::string::npos); } +TEST(Mitc4ReferenceComparison, + RotationRmsExceedanceWarnsWhenEveryRotationRowPasses) { + ContractFixture fixture{"rotation-rms"}; + auto fesa_values = DefaultRows(); + fesa_values[1U].values[3U] = 0.1049; + fesa_values[1U].values[4U] = -0.2098; + fesa_values[1U].values[5U] = 0.3147; + WriteHdf5(fixture.Results(), fixture.Input(), fesa_values); + + auto result = + fesa::test::Mitc4ReferenceComparison::Compare(fixture.ReferenceCase()); + ASSERT_TRUE(result.HasValue()); + const auto& report = result.Value(); + EXPECT_TRUE(report.passed); + ASSERT_EQ(report.warnings.size(), 1U); + EXPECT_EQ(report.warnings[0U].code, "rotation-reference-rms-exceedance"); + EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(), + [](const fesa::test::Mitc4RowDecision& row) { + return row.within_tolerance; + })); +} + +TEST(Mitc4ReferenceComparison, + RotationRowAndRmsExceedancesProduceSeparateWarnings) { + ContractFixture fixture{"rotation-row-and-rms"}; + auto fesa_values = DefaultRows(); + fesa_values[0U].values[3U] = 1.0; + WriteHdf5(fixture.Results(), fixture.Input(), fesa_values); + + auto result = + fesa::test::Mitc4ReferenceComparison::Compare(fixture.ReferenceCase()); + ASSERT_TRUE(result.HasValue()); + const auto& report = result.Value(); + EXPECT_TRUE(report.passed); + ASSERT_EQ(report.warnings.size(), 2U); + EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance"); + EXPECT_EQ(report.warnings[1U].code, "rotation-reference-rms-exceedance"); + EXPECT_LT(report.warnings[0U].row, report.rows.size()); + EXPECT_LT(report.warnings[1U].row, report.rows.size()); +} + // MITC4-REF-005 TEST(Mitc4ReferenceComparison, ReportsMetricsVectorsWorstRowAndJsonDeterministically) { @@ -678,7 +719,7 @@ TEST(Mitc4ReferenceComparison, ASSERT_TRUE(result.HasValue()); const auto& report = result.Value(); ASSERT_TRUE(report.passed); - ASSERT_EQ(report.metrics.size(), 6U); + ASSERT_EQ(report.metrics.size(), 2U); ASSERT_EQ(report.vector_metrics.size(), 2U); EXPECT_NEAR(report.vector_metrics[0U].displacement_norm_error, std::sqrt(0.3125) * 1.0e-9, 1.0e-21); @@ -687,10 +728,10 @@ TEST(Mitc4ReferenceComparison, EXPECT_EQ(report.rows[report.worst_row].component, "UR1"); for (const auto& metric : report.metrics) { EXPECT_TRUE(std::isfinite(metric.reference_scale)); + EXPECT_TRUE(std::isfinite(metric.near_zero_band)); EXPECT_TRUE(std::isfinite(metric.maximum_absolute_error)); - EXPECT_TRUE(std::isfinite(metric.maximum_normalized_error)); - EXPECT_TRUE(std::isfinite(metric.rms_error)); - EXPECT_TRUE(std::isfinite(metric.vector_norm_error)); + EXPECT_TRUE(std::isfinite(metric.relative_rms)); + EXPECT_FALSE(metric.components.empty()); EXPECT_LT(metric.worst_row, report.rows.size()); } @@ -704,8 +745,10 @@ TEST(Mitc4ReferenceComparison, .IsOk()); const std::string first = ReadBytes(json_a); EXPECT_EQ(first, ReadBytes(json_b)); - for (const char* key : {"\"rows\"", "\"metrics\"", "\"vector_metrics\"", - "\"warnings\"", "\"worst_row\"", "\"passed\""}) { + for (const char* key : + {"\"rows\"", "\"metrics\"", "\"vector_metrics\"", "\"warnings\"", + "\"family_identity\"", "\"near_zero_band\"", "\"relative_rms\"", + "\"row_count\"", "\"worst_row\"", "\"passed\""}) { EXPECT_NE(first.find(key), std::string::npos) << key; } } diff --git a/tests/reference/reference_comparison.cpp b/tests/reference/reference_comparison.cpp index 5fd1795..5823a38 100644 --- a/tests/reference/reference_comparison.cpp +++ b/tests/reference/reference_comparison.cpp @@ -29,7 +29,7 @@ #include "fesa/fem/dof_manager.h" #include "fesa/io/abaqus/domain_mapper.h" #include "fesa/io/abaqus/input_reader.h" -#include "fesa/results/result_recovery.h" +#include "reference_tolerance_policy.h" namespace fesa::test { namespace { @@ -48,9 +48,6 @@ constexpr const char* kReactionPath = "/steps/Step-1/frames/0/nodal/reaction"; constexpr const char* kSectionPath = "/steps/Step-1/frames/0/element/section_resultant"; constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11"; -constexpr double kKinematicFloor = 1.0e-9; -constexpr double kForceMomentFloor = 1.0e-3; -constexpr double kRelativeCoefficient = 1.0e-6; class ComparisonFailure final : public std::runtime_error { public: @@ -166,6 +163,17 @@ struct ReferenceTable { std::vector rows; }; +struct SectionReferenceRow { + std::string instance_name; + std::int64_t source_element_label; + std::int64_t source_node_label; + std::vector values; +}; + +struct SectionReferenceTable { + std::vector rows; +}; + ReferenceTable ReadReferenceCsv( const std::filesystem::path& path, const std::vector& expected_header) { @@ -223,6 +231,70 @@ ReferenceTable ReadReferenceCsv( return table; } +SectionReferenceTable ReadSectionReferenceCsv( + const std::filesystem::path& path) { + std::ifstream stream{path}; + if (!stream) { + Fail("needs-reference-artifacts", "An approved reference CSV is missing."); + } + std::string line; + if (!std::getline(stream, line)) { + Fail("schema-mismatch", "An approved reference CSV is empty."); + } + if (!line.empty() && line.back() == '\r') { + line.pop_back(); + } + const std::vector expected_header = { + "Frame", "Part Instance Name", + "Element Label", "Node Label", + "SF-SF1", "SM-SM1", + "SM-SM2", "SM-SM3"}; + if (SplitCsvLine(line) != expected_header) { + Fail("schema-mismatch", "An approved section CSV header is not exact."); + } + + SectionReferenceTable table; + while (std::getline(stream, line)) { + if (!line.empty() && line.back() == '\r') { + line.pop_back(); + } + if (line.empty()) { + Fail("schema-mismatch", "Blank reference CSV rows are not allowed."); + } + const auto fields = SplitCsvLine(line); + if (fields.size() != expected_header.size() || + CollapseWhitespace(fields[0U]) != kFrameText || fields[1U].empty()) { + Fail("schema-mismatch", + "A reference section row has invalid schema or frame identity."); + } + SectionReferenceRow row{}; + row.instance_name = fields[1U]; + row.source_element_label = ParsePositiveLabel(fields[2U]); + row.source_node_label = ParsePositiveLabel(fields[3U]); + row.values.reserve(fields.size() - 4U); + for (std::size_t field = 4U; field < fields.size(); ++field) { + row.values.push_back(ParseFiniteDouble(fields[field])); + } + const auto duplicate = std::find_if( + table.rows.begin(), table.rows.end(), + [&](const SectionReferenceRow& existing) { + return AsciiLower(existing.instance_name) == + AsciiLower(row.instance_name) && + existing.source_element_label == row.source_element_label && + existing.source_node_label == row.source_node_label; + }); + if (duplicate != table.rows.end()) { + Fail("schema-mismatch", + "A reference section row identity is duplicated."); + } + table.rows.push_back(std::move(row)); + } + if (table.rows.empty()) { + Fail("schema-mismatch", "An approved reference CSV has no data rows."); + } + return table; +} + void RequireExactArtifactInventory( const std::filesystem::path& legacy_directory) { std::error_code error; @@ -894,76 +966,39 @@ std::vector OrderedRows( return ordered; } -double TableScale(const ReferenceTable& table, const std::size_t value_index) { - double scale = 0.0; - for (const auto& row : table.rows) { - if (value_index >= row.values.size()) { - Fail("schema-mismatch", "A reference row has the wrong component arity."); - } - scale = (std::max)(scale, std::abs(row.values[value_index])); +std::vector OrderedSectionRows( + const SectionReferenceTable& table, const HdfProjection& hdf) { + const std::size_t endpoint_count = hdf.elements.size() * 2U; + if (table.rows.size() != endpoint_count) { + Fail("schema-mismatch", "The FESA/reference section row sets differ."); } - return scale; -} - -std::vector NormalizeStations( - const Domain& domain, const HdfProjection& hdf, - const ReferenceTable& section_table) { - auto model_result = AnalysisModel::Create(domain); - if (!model_result.HasValue()) { - Fail("schema-mismatch", - "The approved input cannot create an analysis view."); - } - const AnalysisModel model = std::move(model_result.Value()); - const std::array tolerances = { - kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 0U), - kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 3U), - kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 1U), - kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 2U)}; - std::vector endpoints; - endpoints.reserve(hdf.elements.size() * 2U); - for (std::size_t element = 0U; element < hdf.elements.size(); ++element) { + std::vector ordered; + ordered.reserve(endpoint_count); + for (const auto& element : hdf.elements) { for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) { - const auto node = static_cast( - hdf.elements[element].node_internal_ids[endpoint]); - std::array values{}; - for (std::size_t component = 0U; component < values.size(); ++component) { - values[component] = - hdf.section_resultants[(element * 2U + endpoint) * 4U + component]; + const std::size_t node_index = + static_cast(element.node_internal_ids[endpoint]); + if (node_index >= hdf.nodes.size()) { + Fail("schema-mismatch", "An HDF5 section endpoint references no node."); } - endpoints.push_back({static_cast(element), - static_cast(endpoint), - domain.Nodes()[node].source_id, - {}, - values}); + const auto& node = hdf.nodes[node_index]; + const auto found = std::find_if( + table.rows.begin(), table.rows.end(), + [&](const SectionReferenceRow& row) { + return AsciiLower(row.instance_name) == + AsciiLower(element.instance_name) && + row.source_element_label == element.source_element_label && + row.source_node_label == node.source_node_label; + }); + if (found == table.rows.end() || + found->instance_name != element.instance_name) { + Fail("schema-mismatch", + "A reference section endpoint identity does not match HDF5."); + } + ordered.push_back(&*found); } } - auto normalized = ResultRecovery::NormalizeSectionResultantsToNodeStations( - model, endpoints, tolerances); - if (!normalized.HasValue()) { - const auto& diagnostics = normalized.GetStatus().Diagnostics(); - const std::string code = - diagnostics.empty() ? std::string{} : diagnostics[0U].code; - if (code == "node-station-tolerance-failure") { - Fail("tolerance-failure", - "Interior endpoint section resultants disagree."); - } - Fail("schema-mismatch", - "A node station is not eligible for legacy projection."); - } - return std::move(normalized.Value()); -} - -const NodeStationResultRow& FindStation( - const std::vector& stations, const HdfNode& node) { - const auto found = std::find_if( - stations.begin(), stations.end(), [&](const NodeStationResultRow& row) { - return row.node.instance_name == node.instance_name && - row.node.source_label == node.source_node_label; - }); - if (found == stations.end()) { - Fail("schema-mismatch", "A projected HDF5 node station is missing."); - } - return *found; + return ordered; } CanonicalComparisonRow CanonicalRow(const HdfNode& node, @@ -977,6 +1012,8 @@ CanonicalComparisonRow CanonicalRow(const HdfNode& node, kFrameIndex, node.instance_name, node.source_node_label, + 0, + -1, quantity, std::move(component), value, @@ -1003,93 +1040,122 @@ void AppendNodalRows(ComparisonReport& report, const HdfProjection& hdf, CanonicalRow(hdf.nodes[node], quantity, components[component], reference[node]->values[component], units[component], "global-cartesian", dataset_path); - report.rows.push_back( - {std::move(fesa), std::move(abaqus), 0.0, 0.0, false}); + report.rows.push_back({std::move(fesa), std::move(abaqus)}); } } } -void AppendSectionRows(ComparisonReport& report, const HdfProjection& hdf, - const std::vector& reference, - const std::vector& stations) { +void AppendSectionRows( + ComparisonReport& report, const HdfProjection& hdf, + const std::vector& reference) { const std::array components = {"N", "T", "My", "Mz"}; const std::array units = {"force", "force*length", "force*length", "force*length"}; const std::array reference_columns = {0U, 3U, 1U, 2U}; - for (std::size_t node = 0U; node < hdf.nodes.size(); ++node) { - const auto& station = FindStation(stations, hdf.nodes[node]); - for (std::size_t component = 0U; component < components.size(); - ++component) { - auto fesa = CanonicalRow( - hdf.nodes[node], ComparisonQuantity::kSectionResultant, - components[component], station.section_resultant[component], - units[component], "beam-local", kSectionPath); - auto abaqus = - CanonicalRow(hdf.nodes[node], ComparisonQuantity::kSectionResultant, - components[component], - reference[node]->values[reference_columns[component]], - units[component], "beam-local", kSectionPath); - report.rows.push_back( - {std::move(fesa), std::move(abaqus), 0.0, 0.0, false}); + std::size_t row = 0U; + for (std::size_t element = 0U; element < hdf.elements.size(); ++element) { + for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint, ++row) { + const auto& element_row = hdf.elements[element]; + const std::size_t node_index = + static_cast(element_row.node_internal_ids[endpoint]); + if (node_index >= hdf.nodes.size()) { + Fail("schema-mismatch", "An HDF5 section endpoint references no node."); + } + const auto& node = hdf.nodes[node_index]; + const auto canonical = [&](const double value, std::string component, + std::string unit) { + return CanonicalComparisonRow{kModelId, + kStepName, + kFrameIndex, + element_row.instance_name, + node.source_node_label, + element_row.source_element_label, + static_cast(endpoint), + ComparisonQuantity::kSectionResultant, + std::move(component), + value, + std::move(unit), + "beam-local", + kSectionPath}; + }; + for (std::size_t component = 0U; component < components.size(); + ++component) { + auto fesa = canonical( + hdf.section_resultants[(element * 2U + endpoint) * 4U + component], + components[component], units[component]); + auto abaqus = + canonical(reference[row]->values[reference_columns[component]], + components[component], units[component]); + report.rows.push_back({std::move(fesa), std::move(abaqus)}); + } } } } -double AbsoluteFloor(const ComparisonQuantity quantity, const std::string&) { - return quantity == ComparisonQuantity::kDisplacement ? kKinematicFloor - : kForceMomentFloor; +const char* BranchName(const ReferenceToleranceBranch branch) { + switch (branch) { + case ReferenceToleranceBranch::kNearZero: + return "near-zero"; + case ReferenceToleranceBranch::kRelative: + return "relative"; + case ReferenceToleranceBranch::kZeroScaleExact: + return "zero-scale-exact"; + } + return "unknown"; } -void EvaluateGroup(ComparisonReport& report, const ComparisonQuantity quantity, - const std::string& component) { +void EvaluateFamily(ComparisonReport& report, const ComparisonQuantity quantity, + const std::string& identity, + const std::vector& components, + const bool blocking) { std::vector row_indices; for (std::size_t index = 0U; index < report.rows.size(); ++index) { if (report.rows[index].reference.quantity == quantity && - report.rows[index].reference.component == component) { + std::find(components.begin(), components.end(), + report.rows[index].reference.component) != components.end()) { row_indices.push_back(index); } } if (row_indices.empty()) { - Fail("schema-mismatch", "A canonical comparison component has no rows."); + Fail("schema-mismatch", "A canonical comparison family has no rows."); } - double reference_scale = 0.0; + ReferenceToleranceFamily family{}; + family.identity = identity; + family.components = components; + family.blocking = blocking; + family.rows.reserve(row_indices.size()); for (const std::size_t index : row_indices) { - reference_scale = (std::max)(reference_scale, - std::abs(report.rows[index].reference.value)); + family.rows.push_back( + {report.rows[index].fesa.value, report.rows[index].reference.value}); } - const double tolerance = AbsoluteFloor(quantity, component) + - kRelativeCoefficient * reference_scale; - double maximum_absolute = -1.0; - double maximum_normalized = 0.0; - std::size_t worst_row = row_indices.front(); - long double squared_error = 0.0L; - for (const std::size_t index : row_indices) { - auto& row = report.rows[index]; - row.absolute_error = std::abs(row.fesa.value - row.reference.value); - if (!std::isfinite(row.absolute_error)) { - Fail("schema-mismatch", "A canonical row error is nonfinite."); - } - row.tolerance = tolerance; - row.passed = row.absolute_error <= tolerance; - report.passed = report.passed && row.passed; - const double normalized = row.absolute_error / tolerance; - if (row.absolute_error > maximum_absolute) { - maximum_absolute = row.absolute_error; - worst_row = index; - } - maximum_normalized = (std::max)(maximum_normalized, normalized); - const long double error = static_cast(row.absolute_error); - squared_error += error * error; + const auto evaluation = + ReferenceTolerancePolicy::Evaluate({std::move(family)}); + if (!evaluation.valid || evaluation.families.size() != 1U) { + Fail("schema-mismatch", "A canonical comparison family is nonfinite."); } - const double norm_error = std::sqrt(static_cast(squared_error)); - const double rms_error = std::sqrt(static_cast( - squared_error / static_cast(row_indices.size()))); - if (!std::isfinite(norm_error) || !std::isfinite(rms_error)) { - Fail("schema-mismatch", "A component aggregate error is nonfinite."); + const auto& policy = evaluation.families.front(); + for (std::size_t local = 0U; local < row_indices.size(); ++local) { + auto& row = report.rows[row_indices[local]]; + const auto& decision = policy.rows[local]; + row.absolute_error = decision.absolute_error; + row.tolerance = decision.threshold; + row.reference_scale = policy.reference_scale; + row.near_zero_band = policy.near_zero_band; + row.relative_error = decision.relative_error; + row.relative_error_applicable = decision.relative_error_applicable; + row.tolerance_branch = BranchName(decision.branch); + row.passed = decision.passed; + } + const std::size_t worst_row = row_indices[policy.worst_row]; + report.metrics.push_back({quantity, policy.identity, policy.components, + policy.reference_scale, policy.near_zero_band, + policy.near_zero_count, + policy.maximum_absolute_error, policy.relative_rms, + worst_row, policy.rms_passed, policy.passed, + policy.diagnostic_code, policy.row_count}); + if (blocking && !policy.passed) { + report.passed = false; } - report.metrics.push_back({quantity, component, reference_scale, - maximum_absolute, maximum_normalized, rms_error, - norm_error, worst_row}); } PhysicsEvidence MakePhysicsEvidence(const Domain& domain, @@ -1214,7 +1280,8 @@ void WriteCanonicalRow(std::ostream& stream, stream << ",\"frame_index\":" << row.frame_index << ",\"instance_name\":"; WriteJsonString(stream, row.instance_name); stream << ",\"source_node_label\":" << row.source_node_label - << ",\"quantity\":"; + << ",\"source_element_label\":" << row.source_element_label + << ",\"endpoint_index\":" << row.endpoint_index << ",\"quantity\":"; WriteJsonString(stream, QuantityName(row.quantity)); stream << ",\"component\":"; WriteJsonString(stream, row.component); @@ -1231,6 +1298,18 @@ void WriteArray(std::ostream& stream, const std::array& values) { stream << '[' << values[0U] << ',' << values[1U] << ',' << values[2U] << ']'; } +void WriteStrings(std::ostream& stream, + const std::vector& values) { + stream << '['; + for (std::size_t index = 0U; index < values.size(); ++index) { + if (index != 0U) { + stream << ','; + } + WriteJsonString(stream, values[index]); + } + stream << ']'; +} + bool FiniteReport(const ComparisonReport& report) { const auto finite_array = [](const std::array& values) { return std::all_of(values.begin(), values.end(), @@ -1245,17 +1324,19 @@ bool FiniteReport(const ComparisonReport& report) { } for (const auto& row : report.rows) { if (!std::isfinite(row.fesa.value) || !std::isfinite(row.reference.value) || - !std::isfinite(row.absolute_error) || !std::isfinite(row.tolerance)) { + !std::isfinite(row.absolute_error) || !std::isfinite(row.tolerance) || + !std::isfinite(row.reference_scale) || + !std::isfinite(row.near_zero_band) || + !std::isfinite(row.relative_error)) { return false; } } return std::all_of(report.metrics.begin(), report.metrics.end(), [](const ComponentMetrics& metric) { return std::isfinite(metric.reference_scale) && + std::isfinite(metric.near_zero_band) && std::isfinite(metric.maximum_absolute_error) && - std::isfinite(metric.maximum_normalized_error) && - std::isfinite(metric.rms_error) && - std::isfinite(metric.norm_error); + std::isfinite(metric.relative_rms); }); } @@ -1276,18 +1357,12 @@ Result ReferenceComparison::Compare( ReadReferenceCsv(legacy_reference_directory / kReactionName, {"Frame", "Part Instance Name", "Node Label", "RF-RF1", "RF-RF2", "RF-RF3", "RM-RM1", "RM-RM2", "RM-RM3"}); - const ReferenceTable section = - ReadReferenceCsv(legacy_reference_directory / kSectionName, - {"Frame", "Part Instance Name", "Node Label", "SF-SF1", - "SM-SM1", "SM-SM2", "SM-SM3"}); + const SectionReferenceTable section = + ReadSectionReferenceCsv(legacy_reference_directory / kSectionName); HdfProjection hdf = ReadHdfProjection(results_hdf5, input, domain); const auto displacement_rows = OrderedRows(displacement, hdf.nodes); const auto reaction_rows = OrderedRows(reaction, hdf.nodes); - const auto section_rows = OrderedRows(section, hdf.nodes); - const auto stations = NormalizeStations(domain, hdf, section); - if (stations.size() != hdf.nodes.size()) { - Fail("schema-mismatch", "The HDF5 node-station row set is incomplete."); - } + const auto section_rows = OrderedSectionRows(section, hdf); ComparisonReport report{}; report.passed = true; @@ -1301,19 +1376,20 @@ Result ReferenceComparison::Compare( {"force", "force", "force", "force*length", "force*length", "force*length"}, hdf.reaction, kReactionPath); - AppendSectionRows(report, hdf, section_rows, stations); + AppendSectionRows(report, hdf, section_rows); - for (const std::string& component : - {"UX", "UY", "UZ", "URX", "URY", "URZ"}) { - EvaluateGroup(report, ComparisonQuantity::kDisplacement, component); - } - for (const std::string& component : - {"RF1", "RF2", "RF3", "RM1", "RM2", "RM3"}) { - EvaluateGroup(report, ComparisonQuantity::kReaction, component); - } - for (const std::string& component : {"N", "T", "My", "Mz"}) { - EvaluateGroup(report, ComparisonQuantity::kSectionResultant, component); - } + EvaluateFamily(report, ComparisonQuantity::kDisplacement, + "displacement-translation", {"UX", "UY", "UZ"}, true); + EvaluateFamily(report, ComparisonQuantity::kDisplacement, + "displacement-rotation", {"URX", "URY", "URZ"}, true); + EvaluateFamily(report, ComparisonQuantity::kReaction, "reaction-force", + {"RF1", "RF2", "RF3"}, true); + EvaluateFamily(report, ComparisonQuantity::kReaction, "reaction-moment", + {"RM1", "RM2", "RM3"}, true); + EvaluateFamily(report, ComparisonQuantity::kSectionResultant, + "section-force", {"N"}, true); + EvaluateFamily(report, ComparisonQuantity::kSectionResultant, + "section-moment", {"T", "My", "Mz"}, true); report.physics_evidence = MakePhysicsEvidence(domain, hdf); report.stress_comparison_applicable = false; report.stress_comparison_reason = @@ -1366,8 +1442,15 @@ Status ReferenceComparison::WriteDeterministicJson( stream << ",\"reference\":"; WriteCanonicalRow(stream, row.reference); stream << ",\"absolute_error\":" << row.absolute_error - << ",\"tolerance\":" << row.tolerance - << ",\"passed\":" << (row.passed ? "true" : "false") << '}'; + << ",\"threshold\":" << row.tolerance + << ",\"reference_scale\":" << row.reference_scale + << ",\"near_zero_band\":" << row.near_zero_band + << ",\"relative_error\":" << row.relative_error + << ",\"relative_error_applicable\":" + << (row.relative_error_applicable ? "true" : "false") + << ",\"tolerance_branch\":"; + WriteJsonString(stream, row.tolerance_branch); + stream << ",\"passed\":" << (row.passed ? "true" : "false") << '}'; } stream << "],\"metrics\":["; for (std::size_t index = 0U; index < report.metrics.size(); ++index) { @@ -1377,15 +1460,22 @@ Status ReferenceComparison::WriteDeterministicJson( const auto& metric = report.metrics[index]; stream << "{\"quantity\":"; WriteJsonString(stream, QuantityName(metric.quantity)); - stream << ",\"component\":"; - WriteJsonString(stream, metric.component); + stream << ",\"family_identity\":"; + WriteJsonString(stream, metric.family_identity); + stream << ",\"components\":"; + WriteStrings(stream, metric.components); stream << ",\"reference_scale\":" << metric.reference_scale + << ",\"near_zero_band\":" << metric.near_zero_band + << ",\"near_zero_count\":" << metric.near_zero_count + << ",\"row_count\":" << metric.row_count << ",\"maximum_absolute_error\":" << metric.maximum_absolute_error - << ",\"maximum_normalized_error\":" - << metric.maximum_normalized_error - << ",\"rms_error\":" << metric.rms_error - << ",\"norm_error\":" << metric.norm_error - << ",\"worst_row\":" << metric.worst_row << '}'; + << ",\"relative_rms\":" << metric.relative_rms + << ",\"worst_row\":" << metric.worst_row + << ",\"rms_passed\":" << (metric.rms_passed ? "true" : "false") + << ",\"passed\":" << (metric.passed ? "true" : "false") + << ",\"diagnostic_code\":"; + WriteJsonString(stream, metric.diagnostic_code); + stream << '}'; } stream << "],\"physics_evidence\":{\"free_residual_norm\":" << report.physics_evidence.free_residual_norm << ",\"applied_force\":"; diff --git a/tests/reference/reference_comparison.h b/tests/reference/reference_comparison.h index 4451b43..1a9c339 100644 --- a/tests/reference/reference_comparison.h +++ b/tests/reference/reference_comparison.h @@ -20,6 +20,8 @@ struct CanonicalComparisonRow { std::size_t frame_index; std::string instance_name; std::int64_t source_node_label; + std::int64_t source_element_label; + int endpoint_index; ComparisonQuantity quantity; std::string component; double value; @@ -33,18 +35,28 @@ struct RowDecision { CanonicalComparisonRow reference; double absolute_error; double tolerance; + double reference_scale; + double near_zero_band; + double relative_error; + bool relative_error_applicable; + std::string tolerance_branch; bool passed; }; struct ComponentMetrics { ComparisonQuantity quantity; - std::string component; + std::string family_identity; + std::vector components; double reference_scale; + double near_zero_band; + std::size_t near_zero_count; double maximum_absolute_error; - double maximum_normalized_error; - double rms_error; - double norm_error; + double relative_rms; std::size_t worst_row; + bool rms_passed; + bool passed; + std::string diagnostic_code; + std::size_t row_count{}; }; struct PhysicsEvidence { diff --git a/tests/reference/reference_comparison_test.cpp b/tests/reference/reference_comparison_test.cpp index bcf5845..f4742a3 100644 --- a/tests/reference/reference_comparison_test.cpp +++ b/tests/reference/reference_comparison_test.cpp @@ -41,8 +41,8 @@ constexpr const char* kSectionName = "cantilever beam elemental forces.csv"; constexpr const char* kInstanceName = "PART-1_1-1"; constexpr std::size_t kNodeCount = 11U; constexpr std::size_t kElementCount = 10U; -constexpr std::size_t kExpectedRowCount = 176U; -constexpr std::size_t kExpectedMetricCount = 16U; +constexpr std::size_t kExpectedRowCount = 212U; +constexpr std::size_t kExpectedMetricCount = 6U; using NodalValues = std::array, kNodeCount>; using EndpointValues = @@ -315,14 +315,29 @@ const fesa::test::RowDecision* FindRow( const fesa::test::ComponentMetrics* FindMetric( const fesa::test::ComparisonReport& report, - const fesa::test::ComparisonQuantity quantity, + const std::string& family_identity) { + const auto found = + std::find_if(report.metrics.begin(), report.metrics.end(), + [&](const fesa::test::ComponentMetrics& metric) { + return metric.family_identity == family_identity; + }); + return found == report.metrics.end() ? nullptr : &*found; +} + +const fesa::test::RowDecision* FindSectionRow( + const fesa::test::ComparisonReport& report, + const std::int64_t source_element_label, const int endpoint_index, const std::string& component) { const auto found = std::find_if( - report.metrics.begin(), report.metrics.end(), - [&](const fesa::test::ComponentMetrics& metric) { - return metric.quantity == quantity && metric.component == component; + report.rows.begin(), report.rows.end(), + [&](const fesa::test::RowDecision& row) { + return row.reference.quantity == + fesa::test::ComparisonQuantity::kSectionResultant && + row.reference.source_element_label == source_element_label && + row.reference.endpoint_index == endpoint_index && + row.reference.component == component; }); - return found == report.metrics.end() ? nullptr : &*found; + return found == report.rows.end() ? nullptr : &*found; } void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) { @@ -381,13 +396,25 @@ void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) { const std::array section_components = {"N", "T", "My", "Mz"}; const std::array section_units = { "force", "force*length", "force*length", "force*length"}; - for (std::size_t node = 0U; node < kNodeCount; ++node) { - for (std::size_t component = 0U; component < section_components.size(); - ++component) { - expect_row(fesa::test::ComparisonQuantity::kSectionResultant, node, - section_components[component], section_units[component], - "beam-local", - "/steps/Step-1/frames/0/element/section_resultant"); + for (std::size_t element = 0U; element < kElementCount; ++element) { + for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) { + for (std::size_t component = 0U; component < section_components.size(); + ++component) { + ASSERT_LT(row_index, report.rows.size()); + const auto& row = report.rows[row_index++]; + for (const auto* side : {&row.fesa, &row.reference}) { + EXPECT_EQ(side->source_element_label, + static_cast(element + 1U)); + EXPECT_EQ(side->endpoint_index, static_cast(endpoint)); + EXPECT_EQ(side->source_node_label, + static_cast(element + endpoint + 1U)); + EXPECT_EQ(side->component, section_components[component]); + EXPECT_EQ(side->unit_dimension, section_units[component]); + EXPECT_EQ(side->coordinate_system, "beam-local"); + EXPECT_EQ(side->hdf5_dataset_path, + "/steps/Step-1/frames/0/element/section_resultant"); + } + } } } EXPECT_EQ(row_index, report.rows.size()); @@ -475,14 +502,46 @@ TEST(ReferenceComparisonContract, fixture.Results(), fixture.Legacy()), "schema-mismatch"); } + { + ContractFixture fixture{"section-missing-endpoint", mismatched_values}; + auto lines = ReadLines(fixture.Legacy() / kSectionName); + ASSERT_EQ(lines.size(), kElementCount * 2U + 1U); + lines.pop_back(); + WriteLines(fixture.Legacy() / kSectionName, lines); + ExpectFailureCode(fesa::test::ReferenceComparison::Compare( + fixture.Results(), fixture.Legacy()), + "schema-mismatch"); + } + { + ContractFixture fixture{"section-duplicate-endpoint", mismatched_values}; + auto lines = ReadLines(fixture.Legacy() / kSectionName); + ASSERT_EQ(lines.size(), kElementCount * 2U + 1U); + lines.push_back(lines[1U]); + WriteLines(fixture.Legacy() / kSectionName, lines); + ExpectFailureCode(fesa::test::ReferenceComparison::Compare( + fixture.Results(), fixture.Legacy()), + "schema-mismatch"); + } + { + ContractFixture fixture{"section-mismatched-endpoint", mismatched_values}; + auto lines = ReadLines(fixture.Legacy() / kSectionName); + ASSERT_GT(lines.size(), 2U); + ReplaceFirst(lines[1U], ",1,1,", ",1,3,"); + WriteLines(fixture.Legacy() / kSectionName, lines); + ExpectFailureCode(fesa::test::ReferenceComparison::Compare( + fixture.Results(), fixture.Legacy()), + "schema-mismatch"); + } } TEST(ReferenceComparisonContract, - AppliesAbaqusOnlyComponentScaleWithoutClampOrDrop) { + AppliesSharedFamilyScaleAndNearZeroBranchesWithoutAnAbsoluteGate) { auto values = ReferenceValues(); - values.displacement[1U][0U] = 0.999e-9; - values.displacement[2U][0U] = 1.001e-9; - values.section_resultants[0U][0U][2U] = 1.0e7 + 9.0; + const double translation_scale = 1.90476272e-2; + const double near_zero_band = 0.01 * translation_scale; + values.displacement[1U][0U] = 0.999 * near_zero_band; + values.displacement[2U][0U] = 1.001 * near_zero_band; + values.section_resultants[0U][0U][2U] = 1.0e7 + 4.999e5; values.section_resultants[9U][1U][2U] = 0.0; ContractFixture fixture{"tolerance", values}; @@ -505,28 +564,28 @@ TEST(ReferenceComparisonContract, ASSERT_NE(deliberate_failure, nullptr); EXPECT_DOUBLE_EQ(zero->reference.value, 0.0); EXPECT_DOUBLE_EQ(zero->fesa.value, 0.0); - EXPECT_DOUBLE_EQ(zero->tolerance, 1.0e-9); + EXPECT_DOUBLE_EQ(zero->tolerance, near_zero_band); EXPECT_TRUE(zero->passed); EXPECT_TRUE(near_zero->passed); EXPECT_FALSE(deliberate_failure->passed); + EXPECT_EQ(near_zero->tolerance_branch, "near-zero"); + EXPECT_FALSE(near_zero->relative_error_applicable); - const auto* my_metric = FindMetric( - report, fesa::test::ComparisonQuantity::kSectionResultant, "My"); + const auto* my_metric = FindMetric(report, "section-moment"); ASSERT_NE(my_metric, nullptr); EXPECT_DOUBLE_EQ(my_metric->reference_scale, 1.0e7); - const auto* scaled = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "My"); - const auto* residue = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 11, "My"); + EXPECT_EQ(my_metric->components, (std::vector{"T", "My", "Mz"})); + const auto* scaled = FindSectionRow(report, 1, 0, "My"); + const auto* residue = FindSectionRow(report, 10, 1, "My"); ASSERT_NE(scaled, nullptr); ASSERT_NE(residue, nullptr); - EXPECT_DOUBLE_EQ(scaled->tolerance, 10.001); - EXPECT_DOUBLE_EQ(scaled->absolute_error, 9.0); + EXPECT_DOUBLE_EQ(scaled->tolerance, 5.0e5); + EXPECT_DOUBLE_EQ(scaled->absolute_error, 4.999e5); EXPECT_TRUE(scaled->passed); - EXPECT_DOUBLE_EQ(residue->reference.value, -1.56e-2); + EXPECT_DOUBLE_EQ(residue->reference.value, -1.5625e-2); EXPECT_DOUBLE_EQ(residue->fesa.value, 0.0); - EXPECT_DOUBLE_EQ(residue->absolute_error, 1.56e-2); - EXPECT_DOUBLE_EQ(residue->tolerance, 10.001); + EXPECT_DOUBLE_EQ(residue->absolute_error, 1.5625e-2); + EXPECT_DOUBLE_EQ(residue->tolerance, 1.0e5); EXPECT_TRUE(residue->passed); } @@ -548,19 +607,18 @@ TEST(ReferenceComparisonContract, report.rows.begin(), report.rows.end(), [](const fesa::test::RowDecision& row) { return row.passed; })); - const auto* metric = - FindMetric(report, fesa::test::ComparisonQuantity::kDisplacement, "UX"); + const auto* metric = FindMetric(report, "displacement-translation"); const auto* worst = FindRow(report, fesa::test::ComparisonQuantity::kDisplacement, 2, "UX"); ASSERT_NE(metric, nullptr); ASSERT_NE(worst, nullptr); - EXPECT_DOUBLE_EQ(metric->reference_scale, 0.0); + EXPECT_EQ(metric->row_count, kNodeCount * 3U); + EXPECT_DOUBLE_EQ(metric->reference_scale, 1.90476272e-2); EXPECT_DOUBLE_EQ(metric->maximum_absolute_error, 1.0e-9); - EXPECT_DOUBLE_EQ(metric->maximum_normalized_error, 1.0); - EXPECT_NEAR(metric->rms_error, - std::sqrt(1.25 / static_cast(kNodeCount)) * 1.0e-9, + EXPECT_NEAR(metric->relative_rms, + std::sqrt(1.25 / static_cast(kNodeCount * 3U)) * 1.0e-9 / + metric->reference_scale, 1.0e-21); - EXPECT_NEAR(metric->norm_error, std::sqrt(1.25) * 1.0e-9, 1.0e-21); EXPECT_EQ(metric->worst_row, static_cast(worst - report.rows.data())); @@ -589,7 +647,10 @@ TEST(ReferenceComparisonContract, const std::string first = ReadBytes(json_a); EXPECT_EQ(first, ReadBytes(json_b)); for (const char* required : - {"\"rows\"", "\"metrics\"", "\"stress_comparison_applicable\":false", + {"\"rows\"", "\"metrics\"", "\"family_identity\"", "\"components\"", + "\"near_zero_band\"", "\"relative_rms\"", "\"row_count\"", + "\"tolerance_branch\"", "\"relative_error_applicable\"", + "\"stress_comparison_applicable\":false", "\"stress_comparison_reason\"", "\"physics_evidence\"", "\"free_residual_norm\"", "\"applied_force\"", "\"reaction_force\"", "\"applied_moment_about_origin\"", "\"reaction_moment_about_origin\"", @@ -598,11 +659,10 @@ TEST(ReferenceComparisonContract, } } -TEST(ReferenceComparisonContract, NormalizesEligibleStationsWithoutAveraging) { +TEST(ReferenceComparisonContract, + PreservesSectionEndpointIdentityWithoutStationNormalization) { auto values = ReferenceValues(); - values.section_resultants[0U][0U] = {2.0e-4, 3.0e-4, 1.0e7, 4.0e-4}; values.section_resultants[0U][1U][2U] = 9.0e6 - 5.0; - values.section_resultants[1U][0U][2U] = 9.0e6 + 5.0; ContractFixture fixture{"stations", values}; auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(), @@ -612,35 +672,47 @@ TEST(ReferenceComparisonContract, NormalizesEligibleStationsWithoutAveraging) { ASSERT_TRUE(report.passed); EXPECT_TRUE(report.physics_evidence.endpoint_consistency_passed); - const auto* n = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "N"); - const auto* t = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "T"); - const auto* my = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "My"); - const auto* mz = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "Mz"); - ASSERT_NE(n, nullptr); - ASSERT_NE(t, nullptr); - ASSERT_NE(my, nullptr); - ASSERT_NE(mz, nullptr); - EXPECT_DOUBLE_EQ(n->fesa.value, 2.0e-4); - EXPECT_DOUBLE_EQ(t->fesa.value, 3.0e-4); - EXPECT_DOUBLE_EQ(my->fesa.value, 1.0e7); - EXPECT_DOUBLE_EQ(mz->fesa.value, 4.0e-4); - - const auto* interior = FindRow( - report, fesa::test::ComparisonQuantity::kSectionResultant, 2, "My"); + const auto* interior = FindSectionRow(report, 1, 1, "My"); ASSERT_NE(interior, nullptr); EXPECT_DOUBLE_EQ(interior->fesa.value, 9.0e6 - 5.0); EXPECT_DOUBLE_EQ(interior->reference.value, 9.0e6); EXPECT_DOUBLE_EQ(interior->absolute_error, 5.0); - - auto mismatch_values = ReferenceValues(); - mismatch_values.section_resultants[0U][1U][2U] = 9.0e6 - 6.0; - mismatch_values.section_resultants[1U][0U][2U] = 9.0e6 + 6.0; - ContractFixture mismatch{"station-mismatch", mismatch_values}; - ExpectFailureCode(fesa::test::ReferenceComparison::Compare(mismatch.Results(), - mismatch.Legacy()), - "tolerance-failure"); + const auto* adjacent = FindSectionRow(report, 2, 0, "My"); + ASSERT_NE(adjacent, nullptr); + EXPECT_DOUBLE_EQ(adjacent->fesa.value, 9.0e6); + EXPECT_DOUBLE_EQ(adjacent->reference.value, 9.0e6); +} + +TEST(ReferenceComparisonContract, + MapsEachSectionCsvComponentToItsDirectEndpointResultant) { + auto values = ReferenceValues(); + values.section_resultants[0U][0U] = {11.0, 22.0, 33.0, 44.0}; + ContractFixture fixture{"section-component-mapping", values}; + auto lines = ReadLines(fixture.Legacy() / kSectionName); + ASSERT_EQ(lines.size(), kElementCount * 2U + 1U); + lines[1U] = "Increment 1: Step Time = 1.000,PART-1_1-1,1,1,11,33,44,22"; + WriteLines(fixture.Legacy() / kSectionName, lines); + + auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(), + fixture.Legacy()); + ASSERT_TRUE(result.HasValue()); + const auto& report = result.Value(); + ASSERT_TRUE(report.passed); + + const auto* n = FindSectionRow(report, 1, 0, "N"); + const auto* t = FindSectionRow(report, 1, 0, "T"); + const auto* my = FindSectionRow(report, 1, 0, "My"); + const auto* mz = FindSectionRow(report, 1, 0, "Mz"); + ASSERT_NE(n, nullptr); + ASSERT_NE(t, nullptr); + ASSERT_NE(my, nullptr); + ASSERT_NE(mz, nullptr); + EXPECT_DOUBLE_EQ(n->reference.value, 11.0); + EXPECT_DOUBLE_EQ(t->reference.value, 22.0); + EXPECT_DOUBLE_EQ(my->reference.value, 33.0); + EXPECT_DOUBLE_EQ(mz->reference.value, 44.0); + EXPECT_DOUBLE_EQ(n->fesa.value, 11.0); + EXPECT_DOUBLE_EQ(t->fesa.value, 22.0); + EXPECT_DOUBLE_EQ(my->fesa.value, 33.0); + EXPECT_DOUBLE_EQ(mz->fesa.value, 44.0); } diff --git a/tests/reference/reference_tolerance_policy.cpp b/tests/reference/reference_tolerance_policy.cpp new file mode 100644 index 0000000..e6b750d --- /dev/null +++ b/tests/reference/reference_tolerance_policy.cpp @@ -0,0 +1,115 @@ +#include "reference_tolerance_policy.h" + +#include +#include +#include + +namespace fesa::test { +namespace { + +bool IsFiniteFamily(const ReferenceToleranceFamily& family) { + if (family.identity.empty() || family.components.empty() || + family.rows.empty()) { + return false; + } + return std::all_of(family.rows.begin(), family.rows.end(), + [](const ReferenceToleranceRow& row) { + return std::isfinite(row.fesa_value) && + std::isfinite(row.reference_value); + }); +} + +} + +ReferenceToleranceEvaluation ReferenceTolerancePolicy::Evaluate( + const std::vector& families) { + ReferenceToleranceEvaluation evaluation{true, {}, true, {}}; + evaluation.families.reserve(families.size()); + + for (const auto& family : families) { + if (!IsFiniteFamily(family)) { + return {false, "nonfinite-comparison-value", false, {}}; + } + + ReferenceToleranceFamilyDecision decision{}; + decision.identity = family.identity; + decision.components = family.components; + decision.blocking = family.blocking; + decision.row_count = family.rows.size(); + decision.rows.reserve(family.rows.size()); + for (const auto& row : family.rows) { + decision.reference_scale = + (std::max)(decision.reference_scale, std::abs(row.reference_value)); + } + decision.near_zero_band = kNearZeroRatio * decision.reference_scale; + + double error_norm = 0.0; + for (std::size_t index = 0U; index < family.rows.size(); ++index) { + const auto& row = family.rows[index]; + const double absolute_error = + std::abs(row.fesa_value - row.reference_value); + if (!std::isfinite(absolute_error)) { + return {false, "nonfinite-comparison-value", false, {}}; + } + error_norm = std::hypot(error_norm, absolute_error); + if (!std::isfinite(error_norm)) { + return {false, "nonfinite-comparison-value", false, {}}; + } + if (absolute_error > decision.maximum_absolute_error) { + decision.maximum_absolute_error = absolute_error; + decision.worst_row = index; + } + + ReferenceToleranceRowDecision row_decision{}; + row_decision.absolute_error = absolute_error; + if (decision.reference_scale == 0.0) { + ++decision.near_zero_count; + row_decision.threshold = 0.0; + row_decision.branch = ReferenceToleranceBranch::kZeroScaleExact; + row_decision.passed = absolute_error == 0.0; + } else if (std::abs(row.reference_value) <= decision.near_zero_band) { + ++decision.near_zero_count; + row_decision.threshold = decision.near_zero_band; + row_decision.branch = ReferenceToleranceBranch::kNearZero; + row_decision.passed = absolute_error <= row_decision.threshold; + } else { + row_decision.threshold = + kRelativeTolerance * std::abs(row.reference_value); + row_decision.relative_error = + absolute_error / std::abs(row.reference_value); + row_decision.relative_error_applicable = true; + row_decision.branch = ReferenceToleranceBranch::kRelative; + row_decision.passed = row_decision.relative_error <= kRelativeTolerance; + } + decision.rows.push_back(row_decision); + } + + const bool rows_passed = std::all_of( + decision.rows.begin(), decision.rows.end(), + [](const ReferenceToleranceRowDecision& row) { return row.passed; }); + if (decision.reference_scale == 0.0) { + decision.relative_rms = 0.0; + decision.rms_passed = rows_passed; + if (!rows_passed) { + decision.diagnostic_code = "zero-reference-scale-nonzero-error"; + } + } else { + const double rms = + error_norm / std::sqrt(static_cast(family.rows.size())); + decision.relative_rms = rms / decision.reference_scale; + if (!std::isfinite(decision.relative_rms)) { + return {false, "nonfinite-comparison-value", false, {}}; + } + decision.rms_passed = decision.relative_rms <= kRelativeRmsTolerance; + } + decision.passed = rows_passed && decision.rms_passed; + if (decision.blocking && !decision.passed) { + evaluation.passed = false; + } + evaluation.families.push_back(std::move(decision)); + } + + return evaluation; +} + +} diff --git a/tests/reference/reference_tolerance_policy.h b/tests/reference/reference_tolerance_policy.h new file mode 100644 index 0000000..5cda054 --- /dev/null +++ b/tests/reference/reference_tolerance_policy.h @@ -0,0 +1,73 @@ +#ifndef FESA_TESTS_REFERENCE_REFERENCE_TOLERANCE_POLICY_H_ +#define FESA_TESTS_REFERENCE_REFERENCE_TOLERANCE_POLICY_H_ + +#include +#include +#include + +namespace fesa::test { + +enum class ReferenceToleranceBranch { + kNearZero, + kRelative, + kZeroScaleExact, +}; + +struct ReferenceToleranceRow { + double fesa_value; + double reference_value; +}; + +struct ReferenceToleranceFamily { + std::string identity; + std::vector components; + bool blocking; + std::vector rows; +}; + +struct ReferenceToleranceRowDecision { + double absolute_error; + double threshold; + double relative_error; + bool relative_error_applicable; + ReferenceToleranceBranch branch; + bool passed; +}; + +struct ReferenceToleranceFamilyDecision { + std::string identity; + std::vector components; + bool blocking; + double reference_scale; + double near_zero_band; + std::size_t near_zero_count; + double relative_rms; + double maximum_absolute_error; + std::size_t worst_row; + bool rms_passed; + bool passed; + std::string diagnostic_code; + std::vector rows; + std::size_t row_count{}; +}; + +struct ReferenceToleranceEvaluation { + bool valid; + std::string diagnostic_code; + bool passed; + std::vector families; +}; + +class ReferenceTolerancePolicy { + public: + static constexpr double kNearZeroRatio = 0.01; + static constexpr double kRelativeTolerance = 0.05; + static constexpr double kRelativeRmsTolerance = 0.01; + + static ReferenceToleranceEvaluation Evaluate( + const std::vector& families); +}; + +} + +#endif diff --git a/tests/reference/reference_tolerance_policy_test.cpp b/tests/reference/reference_tolerance_policy_test.cpp new file mode 100644 index 0000000..90335d0 --- /dev/null +++ b/tests/reference/reference_tolerance_policy_test.cpp @@ -0,0 +1,118 @@ +#include "reference_tolerance_policy.h" + +#include + +#include +#include +#include +#include +#include +#include + +namespace { + +using fesa::test::ReferenceToleranceBranch; +using fesa::test::ReferenceToleranceFamily; +using fesa::test::ReferenceTolerancePolicy; +using fesa::test::ReferenceToleranceRow; + +ReferenceToleranceFamily MakeFamily(std::string identity, + std::vector components, + bool blocking, + std::vector rows) { + return {std::move(identity), std::move(components), blocking, + std::move(rows)}; +} + +TEST(ReferenceTolerancePolicy, + AppliesRelativeAndNearZeroThresholdsAtTheirInclusiveBoundaries) { + const auto evaluation = ReferenceTolerancePolicy::Evaluate({MakeFamily( + "displacement-translation", {"UX", "UY", "UZ"}, true, + {{105.0, 100.0}, {105.001, 100.0}, {2.0, 1.0}, {2.001, 1.0}})}); + + ASSERT_TRUE(evaluation.valid); + ASSERT_EQ(evaluation.families.size(), 1U); + const auto& family = evaluation.families.front(); + EXPECT_EQ(family.row_count, 4U); + EXPECT_DOUBLE_EQ(family.reference_scale, 100.0); + EXPECT_DOUBLE_EQ(family.near_zero_band, 1.0); + EXPECT_EQ(family.near_zero_count, 2U); + EXPECT_FALSE(family.passed); + ASSERT_EQ(family.rows.size(), 4U); + EXPECT_EQ(family.rows[0U].branch, ReferenceToleranceBranch::kRelative); + EXPECT_TRUE(family.rows[0U].relative_error_applicable); + EXPECT_DOUBLE_EQ(family.rows[0U].relative_error, 0.05); + EXPECT_DOUBLE_EQ(family.rows[0U].threshold, 5.0); + EXPECT_TRUE(family.rows[0U].passed); + EXPECT_FALSE(family.rows[1U].passed); + EXPECT_EQ(family.rows[2U].branch, ReferenceToleranceBranch::kNearZero); + EXPECT_FALSE(family.rows[2U].relative_error_applicable); + EXPECT_DOUBLE_EQ(family.rows[2U].threshold, 1.0); + EXPECT_TRUE(family.rows[2U].passed); + EXPECT_FALSE(family.rows[3U].passed); +} + +TEST(ReferenceTolerancePolicy, + RejectsFamiliesWhoseScaleRelativeRmsExceedsOnePercent) { + const auto evaluation = ReferenceTolerancePolicy::Evaluate({MakeFamily( + "reaction-force", {"RF1", "RF2", "RF3"}, true, + {{101.1, 100.0}, {101.1, 100.0}, {101.1, 100.0}, {101.1, 100.0}})}); + + ASSERT_TRUE(evaluation.valid); + const auto& family = evaluation.families.front(); + EXPECT_EQ(family.row_count, 4U); + EXPECT_TRUE(std::all_of(family.rows.begin(), family.rows.end(), + [](const auto& row) { return row.passed; })); + EXPECT_NEAR(family.relative_rms, 0.011, 1.0e-15); + EXPECT_FALSE(family.rms_passed); + EXPECT_FALSE(family.passed); + EXPECT_FALSE(evaluation.passed); +} + +TEST(ReferenceTolerancePolicy, + FailsClosedForZeroScaleNonzeroErrorWithoutNonfiniteMetrics) { + const auto evaluation = ReferenceTolerancePolicy::Evaluate( + {MakeFamily("section-force", {"N"}, true, {{0.0, 0.0}, {1.0, 0.0}})}); + + ASSERT_TRUE(evaluation.valid); + const auto& family = evaluation.families.front(); + EXPECT_EQ(family.row_count, 2U); + EXPECT_DOUBLE_EQ(family.reference_scale, 0.0); + EXPECT_DOUBLE_EQ(family.near_zero_band, 0.0); + EXPECT_EQ(family.near_zero_count, 2U); + EXPECT_EQ(family.diagnostic_code, "zero-reference-scale-nonzero-error"); + EXPECT_EQ(family.rows[0U].branch, ReferenceToleranceBranch::kZeroScaleExact); + EXPECT_TRUE(family.rows[0U].passed); + EXPECT_FALSE(family.rows[1U].passed); + EXPECT_FALSE(family.rows[1U].relative_error_applicable); + EXPECT_TRUE(std::isfinite(family.relative_rms)); + EXPECT_DOUBLE_EQ(family.relative_rms, 0.0); + EXPECT_FALSE(family.rms_passed); + EXPECT_FALSE(family.passed); +} + +TEST(ReferenceTolerancePolicy, + KeepsWarningOnlyFamilyExceedanceOutOfBlockingVerdict) { + const auto evaluation = ReferenceTolerancePolicy::Evaluate( + {MakeFamily("translation", {"U1", "U2", "U3"}, true, {{0.0, 0.0}}), + MakeFamily("rotation", {"UR1", "UR2", "UR3"}, false, {{1.0, 0.0}})}); + + ASSERT_TRUE(evaluation.valid); + ASSERT_EQ(evaluation.families.size(), 2U); + EXPECT_TRUE(evaluation.families[0U].passed); + EXPECT_FALSE(evaluation.families[1U].passed); + EXPECT_TRUE(evaluation.passed); +} + +TEST(ReferenceTolerancePolicy, + RejectsNonfiniteInputsBeforeToleranceEvaluation) { + const auto evaluation = ReferenceTolerancePolicy::Evaluate( + {MakeFamily("displacement-translation", {"UX", "UY", "UZ"}, true, + {{std::numeric_limits::quiet_NaN(), 0.0}})}); + + EXPECT_FALSE(evaluation.valid); + EXPECT_EQ(evaluation.diagnostic_code, "nonfinite-comparison-value"); + EXPECT_TRUE(evaluation.families.empty()); +} + +}