modify tolerance policy

This commit is contained in:
KOKO\Mimi
2026-08-18 02:29:26 +09:00
parent 123a1b6ccc
commit 624ea4872c
36 changed files with 1909 additions and 672 deletions
+14
View File
@@ -15,6 +15,8 @@
- 프로젝트의 배경, 목적, 사용자, 범위는 `docs/PRD.md`에서 확인한다. - 프로젝트의 배경, 목적, 사용자, 범위는 `docs/PRD.md`에서 확인한다.
- 전체 아키텍처와 모듈별 책임은 `docs/ARCHITECTURE.md`에서 확인한다. - 전체 아키텍처와 모듈별 책임은 `docs/ARCHITECTURE.md`에서 확인한다.
- 주요 아키텍처 결정과 그 이유 및 트레이드오프는 `docs/ADR.md`에서 확인한다. - 주요 아키텍처 결정과 그 이유 및 트레이드오프는 `docs/ADR.md`에서 확인한다.
- 외부 reference tolerance 값, 판정 과정, 보고서와 변경 관리는
`docs/TOLERANCE.md`를 따른다.
- AI 에이전트는 설계나 구현 결정을 내리기 전에 PRD에서 제품 범위를, ARCHITECTURE에서 소유권과 데이터 흐름을, ADR에서 이미 결정된 트레이드오프를 먼저 확인한다. 기능별 요구조건, 연구, 정식화, 수치/reference 검토, I/O, 구현, 검증, 물리 및 release 의미는 `docs/<feature-id>/`의 승인 문서를 source of truth로 삼는다. - AI 에이전트는 설계나 구현 결정을 내리기 전에 PRD에서 제품 범위를, ARCHITECTURE에서 소유권과 데이터 흐름을, ADR에서 이미 결정된 트레이드오프를 먼저 확인한다. 기능별 요구조건, 연구, 정식화, 수치/reference 검토, I/O, 구현, 검증, 물리 및 release 의미는 `docs/<feature-id>/`의 승인 문서를 source of truth로 삼는다.
- Harness는 솔버 자체가 아니라 요구조건, TDD, phase 실행, 검증을 통제하는 개발 운영 인프라이다. 전체 실행 흐름은 `docs/HARNESS_WORKFLOW.md`, 설치와 설정은 `docs/HARNESS.md`를 따른다. - Harness는 솔버 자체가 아니라 요구조건, TDD, phase 실행, 검증을 통제하는 개발 운영 인프라이다. 전체 실행 흐름은 `docs/HARNESS_WORKFLOW.md`, 설치와 설정은 `docs/HARNESS.md`를 따른다.
- FESA는 Abaqus와 독립적인 솔버다. 문서와 구현은 full Abaqus compatibility뿐 아니라 - FESA는 Abaqus와 독립적인 솔버다. 문서와 구현은 full Abaqus compatibility뿐 아니라
@@ -32,6 +34,18 @@
- 비교기는 필요한 source row identity와 component를 결정적으로 대응시키고 누락, 추가, - 비교기는 필요한 source row identity와 component를 결정적으로 대응시키고 누락, 추가,
중복, nonfinite 값을 tolerance 전에 거부한다. Reference artifact는 rename, rewrite 또는 중복, nonfinite 값을 tolerance 전에 거부한다. Reference artifact는 rename, rewrite 또는
보정하지 않는다. 보정하지 않는다.
- 모든 외부 reference comparison은 `docs/TOLERANCE.md`의 공통 정책을 사용한다.
설계 근거는 `docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md`
ADR-022에 보존한다. 동일 logical quantity, unit dimension, coordinate system과 blocking behavior의
component family에서 Abaqus-only scale `S=max(abs(reference))`를 구한다. `abs(reference)`
`0.01*S` 이하인 행은 `abs(fesa-reference) <= 0.01*S`, 그 외 행은 상대오차 `<=0.05`
판정하고, family scale-relative RMS `RMS(error)/S <= 0.01`도 함께 통과해야 한다. 별도
absolute-error gate는 사용하지 않으며 zero-scale family는 FESA도 exact zero일 때만
통과한다.
- B33 section-resultant reference는 `(instance, element label, endpoint node label,
component)`로 HDF5 element endpoint와 직접 대응한다. Node-station collapse 또는 평균을
사용하지 않는다. MITC4 `U1/U2/U3`는 blocking, `UR1/UR2/UR3`는 warning-only 분류를
유지하면서 같은 공통 수치 정책을 사용한다.
- CSV는 FESA 공식 output이 아니며, FESA HDF5에서 추출한 deterministic CSV view는 비교 디버깅/검토용 보조 artifact로만 둔다. - CSV는 FESA 공식 output이 아니며, FESA HDF5에서 추출한 deterministic CSV view는 비교 디버깅/검토용 보조 artifact로만 둔다.
## FESA 개발의 핵심 원칙 ## FESA 개발의 핵심 원칙
+40
View File
@@ -129,6 +129,9 @@ linear-static hook 사이에 조건문으로 새 physics를 삽입하지 않는
### ADR-014: B33 cantilever baseline은 component-scale 혼합 tolerance로 비교한다 ### ADR-014: B33 cantilever baseline은 component-scale 혼합 tolerance로 비교한다
**상태**: Artifact inventory와 component mapping의 역사적 결정은 유지하며 tolerance와
node-station projection 결정은 ADR-022로 대체됨.
**결정**: `reference/cantilever beam/`의 B33 input, displacement CSV, reaction CSV, elemental-force CSV를 V0 reference baseline으로 사용한다. Displacement와 reaction은 node identity로, section resultant는 CSV node station과 정규화된 FESA element endpoint로 비교한다. 같은 model, step/frame, quantity, component에 대해 `reference_scale = max(abs(reference rows))`를 계산하고 모든 row에 `absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation absolute floor는 `1e-9`, force/moment floor는 `1e-3`이다. Beam stress는 출력하되 Abaqus stress reference comparison은 N/A다. **결정**: `reference/cantilever beam/`의 B33 input, displacement CSV, reaction CSV, elemental-force CSV를 V0 reference baseline으로 사용한다. Displacement와 reaction은 node identity로, section resultant는 CSV node station과 정규화된 FESA element endpoint로 비교한다. 같은 model, step/frame, quantity, component에 대해 `reference_scale = max(abs(reference rows))`를 계산하고 모든 row에 `absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation absolute floor는 `1e-9`, force/moment floor는 `1e-3`이다. Beam stress는 출력하되 Abaqus stress reference comparison은 N/A다.
**이유**: 자유단 moment처럼 이론적으로 0인 값에는 행별 상대오차가 정의되지 않으며 Abaqus 결과에 작은 수치 잔차가 남을 수 있다. Component scale을 사용하면 전체 물리량 크기에 비해 작은 잔차를 허용하면서 각 row를 결정적으로 판정할 수 있다. **이유**: 자유단 moment처럼 이론적으로 0인 값에는 행별 상대오차가 정의되지 않으며 Abaqus 결과에 작은 수치 잔차가 남을 수 있다. Component scale을 사용하면 전체 물리량 크기에 비해 작은 잔차를 허용하면서 각 row를 결정적으로 판정할 수 있다.
@@ -232,6 +235,9 @@ false match와 결과 보정을 방지한다. 더 강한 provenance가 필요한
### ADR-020: MITC4 displacement reference는 고정 절대오차로 판정한다 ### ADR-020: MITC4 displacement reference는 고정 절대오차로 판정한다
**상태**: S4 artifact 선택과 U blocking/UR warning-only 분류는 유지하며 고정 절대오차
결정은 ADR-022로 대체됨.
**결정**: Full-integration FESA-MITC4의 sole S4 reference comparison은 matched global **결정**: Full-integration FESA-MITC4의 sole S4 reference comparison은 matched global
`U1/U2/U3` row에 고정 절대오차 `1.0e-5`를 적용해 pass/fail을 판정한다. `U1/U2/U3` row에 고정 절대오차 `1.0e-5`를 적용해 pass/fail을 판정한다.
`UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하지만 초과는 deterministic warning만 `UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하지만 초과는 deterministic warning만
@@ -306,3 +312,37 @@ formulation, 연산·reduction 순서, sign, units, coordinates와 result identi
리팩터링보다 우선하며 HDF5 schema, reference artifact와 ADR-014/ADR-020 tolerance는 리팩터링보다 우선하며 HDF5 schema, reference artifact와 ADR-014/ADR-020 tolerance는
변경하지 않는다. MITC3, solid, dynamic과 plastic behavior는 별도 feature gate 전까지 변경하지 않는다. MITC3, solid, dynamic과 plastic behavior는 별도 feature gate 전까지
구현된 것으로 간주하지 않는다. 구현된 것으로 간주하지 않는다.
### ADR-022: 외부 reference comparison은 공통 family-scale 정책을 사용한다
운영 상수, 검증 순서, report schema와 변경 관리는 `docs/TOLERANCE.md`를 따른다.
**결정**: B33, MITC4와 이후 기능의 외부 reference comparison은 동일한 무차원 수치
정책을 사용한다. 같은 model/case, step/frame, logical quantity, unit dimension, coordinate
system과 blocking behavior의 component family에 대해 reference-only scale
`S=max(abs(reference))`를 계산한다. `abs(reference)<=0.01*S`인 행은
`abs(fesa-reference)<=0.01*S`, 나머지 행은 `abs(fesa-reference)/abs(reference)<=0.05`
만족해야 한다. 모든 행과 함께 family scale-relative RMS `RMS(error)/S<=0.01`
통과해야 한다. 독립 absolute-error gate는 두지 않으며 `S=0`이면 모든 FESA 값도 exact
zero일 때만 통과한다.
B33 family는 translation, rotation, reaction force, reaction moment, section force와
section moment로 구분한다. 2026-08-18에 다시 생성된 elemental-force CSV의
`Element Label``Node Label`을 사용해 `(instance, element, endpoint node, component)`
HDF5 endpoint에 직접 대응하고 node-station collapse나 평균을 사용하지 않는다. MITC4는
translation `U1/U2/U3` family를 blocking으로, rotation `UR1/UR2/UR3` family를
warning-only로 유지한다.
**이유**: Feature별 absolute floor와 fixed absolute tolerance는 모델 단위와 크기에 따라
서로 다른 엄격도를 만들었다. 개별 component scale은 물리적으로 zero-like인 MITC4
`U1/U2` residue를 자체 scale로 만들어 잘못 실패시킨다. 동일 차원의 family scale,
near-zero 대체 분기와 relative RMS를 결합하면 zero row의 불안정한 raw relative error를
피하면서 모든 행과 전체 오차 수준을 함께 검사할 수 있다. B33의 element-endpoint identity는
reference와 HDF5가 제공하는 실제 source identity를 보존한다.
**트레이드오프**: Family 최대값이 작은 component의 허용폭을 결정하므로 행별 relative
gate와 family RMS gate를 모두 유지해야 한다. Reference family 전체가 zero이면 별도
절대 scale이 없으므로 exact-zero만 허용하는 엄격한 fail-closed 정책이 된다. 기존 B33과
MITC4 verification report 및 comparator schema는 새 정책에 맞춰 다시 생성해야 하지만
reference artifact, HDF5 schema, element formulation과 blocking/warning 분류는 변경하지
않는다.
+16 -12
View File
@@ -419,14 +419,19 @@ Schema requirements:
`[N,T,My,Mz]`, generalized strain/resultant는 두 Gauss point에 기록한다. `[N,T,My,Mz]`, generalized strain/resultant는 두 Gauss point에 기록한다.
- General beam section stress는 section point의 axial `S11`만 복구한다. Section point가 - General beam section stress는 section point의 axial `S11`만 복구한다. Section point가
없으면 centroid `(0,0)``source=fesa-default`로 기록한다. 없으면 centroid `(0,0)``source=fesa-default`로 기록한다.
- 승인된 `reference/cantilever beam/cantilever beam elemental forces.csv`node station - 승인된 `reference/cantilever beam/cantilever beam elemental forces.csv`각 B33
기준 `SF1/SM1/SM2/SM3`을 제공한다. FESA endpoint를 동일한 section-cut 부호로 element의 두 endpoint에 `Element Label`, `Node Label``SF1/SM1/SM2/SM3`을 제공한다.
정규화하고 interior node의 두 endpoint가 tolerance 안에서 일치하는지 먼저 확인한 뒤 `(instance, element label, endpoint node label, component)` identity로 HDF5
`SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`로 비교한다. `[element,endpoint,N/T/My/Mz]`에 직접 대응하고 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`,
- Reference tolerance는 같model, step/frame, quantity, component의 Abaqus rows에서 `SM3 -> T`로 비교한다. External reference comparisonnode-station collapse나 평균을
`reference_scale = max(abs(reference_value))`를 구하고 각 row에 사용하지 않는다.
`absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation floor는 - Reference tolerance는 같은 model/case, step/frame, logical quantity, unit dimension,
`1e-9`, force/moment floor는 `1e-3`이다. coordinate system과 blocking behavior의 component family에서 Abaqus-only scale
`S=max(abs(reference))`를 구한다. `abs(reference)<=0.01*S`인 행은 absolute error
`<=0.01*S`, 그 외 행은 relative error `<=0.05`로 판정하며 family scale-relative RMS
`RMS(error)/S<=0.01`도 통과해야 한다. 독립 absolute-error gate는 사용하지 않고
zero-scale family는 FESA도 exact zero일 때만 통과한다. 전체 판정 과정과 report/change
management contract는 `docs/TOLERANCE.md`를 따른다.
- Beam stress는 HDF5 schema와 unit/analytical test로 검증하지만 Abaqus reference - Beam stress는 HDF5 schema와 unit/analytical test로 검증하지만 Abaqus reference
comparison은 N/A다. comparison은 N/A다.
@@ -469,9 +474,8 @@ stabilization만 둔다. Drilling calibration, artificial-energy policy와 별
dataset은 이 기능 범위가 아니다. dataset은 이 기능 범위가 아니다.
Full-integration FESA-MITC4의 reference comparison은 `reference/shell/` S4의 기존 input 및 Full-integration FESA-MITC4의 reference comparison은 `reference/shell/` S4의 기존 input 및
displacement CSV만 사용한다. Global `U1/U2/U3`만 blocking이고 모든 matched row에 고정 displacement CSV만 사용한다. Global `U1/U2/U3` translation family만 blocking이고
절대오차 `1.0e-5`를 적용한다. `UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하되 `UR1/UR2/UR3` rotation family는 warning-only evidence다. 두 family 모두 ADR-022의 공통
warning-only evidence다. MITC4 판정에는 component scale을 사용하지 않으며 B33의 기존 family-scale row/RMS 정책을 사용한다. S4R은
혼합 tolerance는 변경하지 않는다. S4R은
같은 kernel을 선택하는 source mapping과 metadata를 unit/integration tests로 검증하며 같은 kernel을 선택하는 source mapping과 metadata를 unit/integration tests로 검증하며
`reference/shellR/` artifact는 acceptance comparison에 포함하지 않는다. `reference/shellR/` artifact는 acceptance comparison에 포함하지 않는다.
+7 -7
View File
@@ -113,7 +113,7 @@ Parser keyword, element kernel, solver backend 또는 output dataset 중 하나
| FESA-PRD-014 | 선형 정적 pipeline은 stiffness factorization과 load substitution을 분리해야 한다. | `Kff` factorization이 load vector assembly보다 먼저 수행되고, `rhs=Ff-Kfc*dc` substitution으로 full displacement를 복구한다. | orchestration test, solver-adapter test | | FESA-PRD-014 | 선형 정적 pipeline은 stiffness factorization과 load substitution을 분리해야 한다. | `Kff` factorization이 load vector assembly보다 먼저 수행되고, `rhs=Ff-Kfc*dc` substitution으로 full displacement를 복구한다. | orchestration test, solver-adapter test |
| FESA-PRD-015 | FESA는 V0 결과와 diagnostic을 안정된 외부 계약으로 출력해야 한다. | CLI가 `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5` exit code를 사용하고 diagnostic field가 계약과 일치하며 HDF5가 displacement, reaction, end force, section resultant, generalized strain/resultant, axial `S11`을 포함하고 실패 시 불완전한 최종 파일을 남기지 않는다. | CLI integration test, HDF5 schema/atomicity test | | FESA-PRD-015 | FESA는 V0 결과와 diagnostic을 안정된 외부 계약으로 출력해야 한다. | CLI가 `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5` exit code를 사용하고 diagnostic field가 계약과 일치하며 HDF5가 displacement, reaction, end force, section resultant, generalized strain/resultant, axial `S11`을 포함하고 실패 시 불완전한 최종 파일을 남기지 않는다. | CLI integration test, HDF5 schema/atomicity test |
| FESA-PRD-016 | Dense와 sparse math storage 및 backend 경계를 분리해야 한다. | `Vector`는 contiguous, `Matrix`는 row-major contiguous storage와 MKL CBLAS를 사용하고 `SparseMatrix`는 별도 0-based CSR 타입이며 MKL 타입이 public core API에 노출되지 않는다. | math unit test, dependency review | | FESA-PRD-016 | Dense와 sparse math storage 및 backend 경계를 분리해야 한다. | `Vector`는 contiguous, `Matrix`는 row-major contiguous storage와 MKL CBLAS를 사용하고 `SparseMatrix`는 별도 0-based CSR 타입이며 MKL 타입이 public core API에 노출되지 않는다. | math unit test, dependency review |
| FESA-PRD-017 | B33 reference comparison은 component-scale 혼합 tolerance를 사용해야 한다. | 모든 matched row가 `abs_error <= absolute_floor + 1e-6 * reference_scale`을 만족하고 missing/extra/nonfinite row는 comparison 전에 실패한다. | reference comparison unit/integration test, verification report | | FESA-PRD-017 | 모든 외부 reference comparison은 승인된 공통 family-scale tolerance를 사용해야 한다. | matched row가 near-zero 대체 조건 또는 상대오차 `0.05`를 통과하고 각 family의 scale-relative RMS가 `0.01` 이하이며 missing/extra/duplicate/nonfinite row는 comparison 전에 실패한다. | reference comparison unit/integration test, verification report |
| FESA-PRD-018 | 승인된 B33 reference artifact는 현재 경로의 read-only baseline으로 유지해야 한다. | `reference/cantilever beam/` 파일을 rename, rewrite 또는 보정하지 않고 기능이 선언한 exact path에서 읽는다. | artifact inventory, Git diff review | | FESA-PRD-018 | 승인된 B33 reference artifact는 현재 경로의 read-only baseline으로 유지해야 한다. | `reference/cantilever beam/` 파일을 rename, rewrite 또는 보정하지 않고 기능이 선언한 exact path에서 읽는다. | artifact inventory, Git diff review |
| FESA-PRD-019 | B33 beam section과 local axis를 Abaqus 의미에 맞게 매핑해야 한다. | `n1 -> local y`, `t x n1 -> local z`, `Iy=I11`, `Iz=I22`, `I12=0`을 적용하고 nonpositive property, zero-length element, tangent-parallel guide vector를 구조화된 model diagnostic으로 거부한다. | section-mapping unit test, element geometry test | | FESA-PRD-019 | B33 beam section과 local axis를 Abaqus 의미에 맞게 매핑해야 한다. | `n1 -> local y`, `t x n1 -> local z`, `Iy=I11`, `Iz=I22`, `I12=0`을 적용하고 nonpositive property, zero-length element, tangent-parallel guide vector를 구조화된 model diagnostic으로 거부한다. | section-mapping unit test, element geometry test |
@@ -121,12 +121,12 @@ Parser keyword, element kernel, solver backend 또는 output dataset 중 하나
- MSVC x64 Debug 환경에서 configure, build, CTest를 검증한다. - MSVC x64 Debug 환경에서 configure, build, CTest를 검증한다.
- reference test 결과는 deterministic해야 한다. - reference test 결과는 deterministic해야 한다.
- HDF5 schema는 versioned contract로 관리한다. - HDF5 schema는 versioned contract로 관리한다.
- B33 reference tolerance는 model, step/frame, quantity, component별 Abaqus scale만 사용한다. - 모든 외부 reference comparison은 동일 logical quantity와 unit dimension의 component
- 승인된 SI B33 bundle의 absolute floor는 displacement/rotation `1e-9`, force/moment family별 Abaqus-only maximum scale을 사용한다. Near-zero ratio `0.01`, 일반 행
`1e-3`이며 relative coefficient는 `1e-6`이다. 상대오차는 `0.05`, family scale-relative RMS는 `0.01`이며 독립 absolute-error gate는
- MITC4 U/UR comparison은 고정 절대오차 `1.0e-5`를 사용한다. `U1/U2/U3` 사용하지 않는다.
blocking이고 `UR1/UR2/UR3` 초과는 warning-only다. B33의 component-scale 혼합 - B33 section resultant는 source element label과 endpoint node label로 직접 대응한다.
tolerance는 별도 기존 계약으로 유지한다. MITC4 `U1/U2/U3`만 blocking이고 `UR1/UR2/UR3` 초과는 warning-only다.
- parser, solver, HDF5 writer는 실패 원인을 구조화된 diagnostic으로 보고한다. - parser, solver, HDF5 writer는 실패 원인을 구조화된 diagnostic으로 보고한다.
- oneMKL, oneTBB, HDF5는 CMake에서 명시 탐지하고 실패 원인을 분류한다. - oneMKL, oneTBB, HDF5는 CMake에서 명시 탐지하고 실패 원인을 분류한다.
- 대규모 모델 성능 최적화보다 V0의 명확성, 테스트 가능성, 검증 traceability를 우선한다. - 대규모 모델 성능 최적화보다 V0의 명확성, 테스트 가능성, 검증 traceability를 우선한다.
+2 -1
View File
@@ -148,7 +148,8 @@ gap은 Coordinator가 해당 owner 단계로 되돌린다.
Reference comparison은 source identity와 component로 행을 결정적으로 대응시키며 missing, Reference comparison은 source identity와 component로 행을 결정적으로 대응시키며 missing,
extra, duplicate 또는 nonfinite required row를 tolerance 전에 거부한다. CSV는 외부 reference이고 extra, duplicate 또는 nonfinite required row를 tolerance 전에 거부한다. CSV는 외부 reference이고
FESA의 authoritative output은 `results.h5`다. FESA의 authoritative output은 `results.h5`다. 공통 tolerance 값, family 구성, row/RMS
판정, report evidence와 변경 관리는 `docs/TOLERANCE.md`를 따른다.
## 요구사항 단위 산출물 구조 ## 요구사항 단위 산출물 구조
+431
View File
@@ -0,0 +1,431 @@
# FESA Reference Tolerance Policy
## 문서 정보
- policy_id: `common-reference-tolerance`
- status: `approved-and-implemented`
- effective_date: `2026-08-18`
- scope: Abaqus CSV와 FESA HDF5 사이의 외부 reference comparison
- decision_record: `docs/ADR.md`의 ADR-022
- design_record: `docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md`
## 목적
이 문서는 FESA의 외부 reference tolerance를 관리하는 운영 기준이다. 공통 tolerance
상수뿐 아니라 row identity 사전검사, family 구성, 행별 판정, relative RMS, 최종 verdict,
보고서 evidence와 변경 절차를 하나의 source of truth로 제공한다.
이 정책은 Abaqus와 FESA의 내부 알고리즘 동등성을 판단하지 않는다. 승인된 같은 입력
모델에서 나온 외부 수치 결과가 합의된 오차 범위 안에 있는지만 검증한다.
## 적용 범위
### 적용 대상
- Abaqus reference CSV와 FESA `results.h5`의 deterministic comparison
- B33 displacement, rotation, reaction, section resultant
- MITC4 displacement 및 rotation
- 이후 requirements에서 family와 row identity가 승인된 reference quantity
### 적용하지 않는 대상
다음 검증은 이 문서의 `0.01/0.05/0.01` 값을 사용하지 않는다. 각 feature의 formulation,
requirements, numerical-review 또는 physics contract가 별도 tolerance를 정한다.
- element stiffness symmetry, rank, rigid-body mode
- analytical solution, patch test와 mesh convergence
- linear-system residual과 global equilibrium
- energy, stress/strain sanity
- geometry, Jacobian과 director validation
- parser, HDF5 schema와 exact identity 검증
Schema와 identity 검증은 수치 tolerance보다 먼저 수행하며 tolerance로 완화할 수 없다.
## 용어
| 용어 | 정의 |
| --- | --- |
| FESA 값 \(f_i\) | Authoritative `results.h5`에서 deterministic projection으로 얻은 값 |
| Reference 값 \(r_i\) | 승인된 read-only Abaqus CSV에서 얻은 값 |
| Matched row | Model/case, step/frame, source identity, quantity와 component가 정확히 대응된 한 행 |
| Family | 같은 logical quantity, unit dimension, coordinate system과 blocking behavior를 공유하는 component 집합 |
| Reference scale \(S\) | Family reference 값만으로 계산한 \(\max_i |r_i|\) |
| Near-zero band \(B\) | \(0.01S\) |
| Blocking family | 행 또는 RMS가 실패하면 전체 reference gate를 실패시키는 family |
| Warning-only family | 동일한 실패를 warning으로 기록하지만 전체 gate를 실패시키지 않는 family |
## 공통 상수
모든 외부 reference comparison은 다음 세 상수를 사용한다.
```text
near_zero_ratio = 0.01
relative_tolerance = 0.05
relative_rms_tolerance = 0.01
```
- `near_zero_ratio`: family 최대 reference 크기의 1% 이하를 near-zero row로 분류
- `relative_tolerance`: 일반 row의 최대 상대오차 5%
- `relative_rms_tolerance`: family error RMS가 reference scale의 1% 이하
독립적인 absolute-error gate는 사용하지 않는다. Maximum absolute error는 진단 정보이며
그 자체로 pass/fail을 결정하지 않는다.
## Comparison family
Family는 다음 identity를 모두 포함한다.
```text
model_or_case
step_and_frame
logical_quantity
unit_dimension
coordinate_system
blocking_behavior
```
서로 다른 물리 차원, 좌표계 또는 blocking behavior는 같은 scale을 공유하지 않는다.
Component가 작다는 이유만으로 별도 family로 분리하지 않는다.
### 현재 승인된 family
| Feature | Family | Components | Dimension | Coordinates | Behavior |
| --- | --- | --- | --- | --- | --- |
| B33 | displacement-translation | `UX, UY, UZ` | length | global Cartesian | blocking |
| B33 | displacement-rotation | `URX, URY, URZ` | rotation | global Cartesian | blocking |
| B33 | reaction-force | `RF1, RF2, RF3` | force | global Cartesian | blocking |
| B33 | reaction-moment | `RM1, RM2, RM3` | force × length | global Cartesian | blocking |
| B33 | section-force | `N` | force | beam local | blocking |
| B33 | section-moment | `T, My, Mz` | force × length | beam local | blocking |
| MITC4 | displacement-translation | `U1, U2, U3` | length | global Cartesian | blocking |
| MITC4 | displacement-rotation | `UR1, UR2, UR3` | rotation | global Cartesian | warning-only |
## 전체 검증 과정
```mermaid
flowchart TD
A[승인 input과 read-only reference CSV 확인] --> B[FESA로 같은 input 해석]
B --> C[authoritative results.h5 생성]
C --> D[CSV와 HDF5를 canonical row로 projection]
D --> E{row identity, schema, finite precheck}
E -- fail --> X[수치 비교 전 hard failure]
E -- pass --> F[comparison family 구성]
F --> G[reference-only scale S와 near-zero band 계산]
G --> H[모든 row에 near-zero 또는 relative gate 적용]
H --> I[family scale-relative RMS 계산]
I --> J[blocking 또는 warning-only behavior 적용]
J --> K[deterministic JSON과 report 기록]
```
### 1. Reference artifact 확인
기능별 `reference-model.md``io.md`가 지정한 정확한 input/CSV 경로만 사용한다.
Comparator와 agent는 reference artifact를 rename, rewrite, 보정 또는 zero-clamp하지 않는다.
Metadata, README 또는 canonical filename은 feature requirement가 수치 계약으로 명시하지
않는 한 readiness 조건이 아니다.
### 2. FESA 결과 생성
Reference와 같은 `.inp`를 FESA로 해석하고 authoritative `results.h5`를 생성한다.
CSV로 다시 내보낸 FESA 값은 디버깅용 view일 뿐 authoritative comparison source가 아니다.
### 3. Canonical row projection
CSV와 HDF5 값을 기능 계약의 stable source identity와 component로 canonical row에 투영한다.
Internal index 또는 equation index를 source identity 대신 사용하지 않는다.
- B33 displacement/reaction:
`(model, Step-1, frame 0, instance, source node label, quantity, component)`
- B33 section resultant:
`(model, Step-1, frame 0, instance, source element label, endpoint source node label,
quantity, component)`
- MITC4 displacement/rotation:
`(case, Step-1, frame 0, instance, source node label, quantity, component)`
B33 section mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`다.
Element endpoint를 node station으로 collapse하거나 대표 endpoint를 선택하거나 평균하지
않는다.
### 4. 수치 비교 전 hard precheck
다음 중 하나라도 발견되면 tolerance 계산을 시작하지 않고 comparison을 실패시킨다.
- required file 또는 dataset 누락
- CSV/HDF5 header, shape, component inventory 불일치
- missing 또는 extra canonical row
- duplicate row identity
- source instance/node/element/endpoint identity 불일치
- element endpoint와 input connectivity 불일치
- nonfinite FESA/reference 값 또는 계산 중 nonfinite error
이 오류는 warning-only family에 속하더라도 hard failure다. Tolerance는 schema나 identity
문제를 숨기는 수단이 아니다.
### 5. Reference scale과 near-zero band
각 family \(g\)에 대해 read-only reference 값만 사용한다.
\[
S_g = \max_i |r_i|
\]
\[
B_g = 0.01S_g
\]
FESA 값을 scale 계산에 사용하지 않는다. `max(1, ...)`, 임의 floor, unit별 absolute
tolerance 또는 reference zero clamp를 추가하지 않는다.
### 6. 행별 판정
먼저 absolute error를 계산한다.
\[
e_i = |f_i-r_i|
\]
#### Near-zero row
\[
|r_i| \le B_g
\]
이면 다음을 만족할 때 통과한다.
\[
e_i \le B_g
\]
#### 일반 relative row
\[
|r_i| > B_g
\]
이면 다음을 만족할 때 통과한다.
\[
\frac{e_i}{|r_i|} \le 0.05
\]
Near-zero 분류와 두 threshold는 경계값을 포함한다. 모든 matched row를 판정하며 작은
reference 값이나 zero row를 삭제하지 않는다.
### 7. Zero-scale family
\(S_g=0\)이면 family의 모든 reference 값이 정확히 0이다. 독립 absolute floor가 없으므로
다음 fail-closed 규칙을 적용한다.
- 모든 FESA 값도 정확히 0이면 모든 row와 family RMS가 통과한다.
- 하나라도 0이 아닌 FESA 값이 있으면 해당 row와 family가 실패한다.
- Report에는 NaN/Inf 대신 `zero-reference-scale-nonzero-error`를 기록한다.
### 8. Family scale-relative RMS
Family의 모든 absolute error로 error RMS를 계산하고 같은 reference scale로 정규화한다.
\[
\operatorname{relative\_rms}_g =
\frac{\sqrt{\frac{1}{n_g}\sum_i e_i^2}}{S_g}
\]
\[
\operatorname{relative\_rms}_g \le 0.01
\]
이 값은 row-relative error들의 RMS가 아니다. Family의 모든 row gate와 RMS gate가 모두
통과해야 family가 통과한다.
### 9. 최종 verdict
| 상태 | 전체 reference verdict |
| --- | --- |
| Precheck failure | 항상 실패 |
| Blocking family row failure | 실패 |
| Blocking family RMS failure | 실패 |
| Warning-only family row failure | 전체 verdict 유지, row warning 생성 |
| Warning-only family RMS failure | 전체 verdict 유지, RMS warning 생성 |
| Warning-only row와 RMS 동시 실패 | 두 종류의 deterministic warning을 모두 생성 |
| 모든 blocking family 통과 | Reference gate 통과 |
Warning-only family도 동일한 수치 규칙과 report schema를 사용한다. 차이는 전체 verdict에
미치는 영향뿐이다.
### 10. Evidence report
Report는 사람이 원본 값에서 판정을 재현할 수 있어야 한다.
#### Row별 필수 항목
- model/case와 feature contract가 정한 step/frame
- instance와 source node/element/endpoint identity
- quantity와 component
- FESA 값, reference 값, absolute error
- reference scale과 near-zero band
- 적용 branch: `near-zero`, `relative` 또는 `zero-scale-exact`
- 적용 threshold와, 해당되는 경우 row relative error
- blocking 여부와 row pass/fail
Units와 coordinate system은 family와 feature reference/I/O contract에 반드시 정의한다.
Row JSON이 이를 직접 반복하지 않더라도 report가 해당 contract와 family identity를
결정적으로 가리켜야 한다.
#### Family별 필수 항목
- family identity와 component 목록
- total `row_count``near_zero_count`
- reference scale과 near-zero band
- maximum absolute error와 deterministic worst row
- scale-relative RMS와 RMS pass/fail
- family pass/fail과 diagnostic code
#### 전체 필수 항목
- missing/extra/duplicate/nonfinite precheck 결과
- blocking failure와 warning-only warning 목록
- overall `passed` verdict
- 사용한 input/reference/HDF5 identity
Report에는 nonfinite metric을 기록하지 않는다. Historical report의 수치나 판정을 새
정책에 맞춰 소급 수정하지 않고 새 comparison evidence를 별도 생성한다.
## 계산 예시
### 일반 relative row
Family scale이 \(S=100\)이면 near-zero band는 \(B=1\)이다.
- \(r=50\), \(f=52.4\): 상대오차 \(2.4/50=0.048\), 통과
- \(r=50\), \(f=52.6\): 상대오차 \(2.6/50=0.052\), 실패
### Near-zero row
같은 family에서 \(S=100\), \(B=1\)일 때:
- \(r=0.2\), \(e=0.9\): \(0.9 \le 1\), 통과
- \(r=0.2\), \(e=1.1\): \(1.1 > 1\), 실패
### RMS gate
네 reference 값이 모두 \(100\)이고 \(S=100\)일 때, 각 error가 \(0.8\)이면 relative
RMS는 \(0.008\)로 통과한다. 각 error가 \(1.1\)이면 각 row의 상대오차는 \(0.011\)로
통과하지만 relative RMS도 \(0.011\)이므로 family는 실패한다.
### Zero-scale family
- Reference `[0,0,0]`, FESA `[0,0,0]` → 통과
- Reference `[0,0,0]`, FESA `[0,1e-12,0]` → exact-zero 규칙 실패,
`zero-reference-scale-nonzero-error`
## 구현 위치와 검증 진입점
| 책임 | 위치 |
| --- | --- |
| 공통 evaluator | `tests/reference/reference_tolerance_policy.h/.cpp` |
| 공통 boundary tests | `tests/reference/reference_tolerance_policy_test.cpp` |
| B33 comparator와 JSON | `tests/reference/reference_comparison.h/.cpp` |
| B33 tests | `tests/reference/reference_comparison_test.cpp`, `tests/reference/b33_reference_comparison_test.cpp` |
| MITC4 comparator와 JSON | `tests/reference/mitc4_reference_comparison.h/.cpp` |
| MITC4 tests | `tests/reference/mitc4_reference_comparison_test.cpp`, `tests/reference/mitc4_reference_cases_test.cpp` |
Focused verification:
```powershell
cmake --build .harness/build --config Debug --target fesa_reference_tests
ctest --test-dir .harness/build -C Debug -R 'ReferenceTolerancePolicy|ReferenceComparisonContract|B33ReferenceComparison|Mitc4ReferenceComparison|Mitc4S4Reference' --output-on-failure
```
전체 verification:
```powershell
cmake --build .harness/build --config Debug
ctest --test-dir .harness/build -C Debug --show-only=json-v1
ctest --test-dir .harness/build -C Debug --output-on-failure
```
Generated JSON ledger는 `.harness/build/reference/` 아래에 두며 Git에 commit하지 않는다.
Feature별 current evidence는 해당 `docs/<feature-id>/reference-comparison.md`에 기록한다.
## 규범 요구조건
- **FESA-TOL-001** — Comparator shall use only the exact feature-approved read-only reference
artifacts and authoritative FESA HDF5 values.
- **FESA-TOL-002** — Comparator shall reject missing, extra, duplicate, nonfinite, schema- or
identity-mismatched rows before numerical tolerance evaluation.
- **FESA-TOL-003** — Comparator shall group matched rows only by the approved family identity,
unit dimension, coordinate system and blocking behavior.
- **FESA-TOL-004** — Every nonzero-scale family shall use near-zero ratio `0.01`, row relative
tolerance `0.05` and scale-relative RMS tolerance `0.01`.
- **FESA-TOL-005** — A zero-scale family shall pass only when every FESA value is exactly zero and
shall otherwise report `zero-reference-scale-nonzero-error` without NaN/Inf.
- **FESA-TOL-006** — Every matched row and family RMS shall be evaluated; no row shall be clamped,
omitted, averaged or hidden by an independent absolute-error gate.
- **FESA-TOL-007** — Blocking-family failure shall fail the reference gate; warning-only failure
shall emit deterministic row/RMS warnings without changing the blocking verdict.
- **FESA-TOL-008** — The report shall contain every row decision, family row count, scale,
near-zero inventory, worst error, relative RMS, diagnostic and overall verdict.
- **FESA-TOL-009** — Repeated comparison of the same artifacts and HDF5 shall preserve row,
family, diagnostic and JSON ordering.
- **FESA-TOL-010** — Tolerance constants, family membership, row identity or blocking behavior
shall not change without explicit approval and fresh comparison evidence.
## Requirement Verification Matrix
| ID | Verification method | Acceptance criteria | Evidence |
| --- | --- | --- | --- |
| FESA-TOL-001 | Reference integration tests and run-local before/after content audit | Exact declared paths used; comparison run이 reference를 변경하지 않음 | Feature reference tests; run-local hash/diff evidence |
| FESA-TOL-002 | Missing/extra/duplicate/nonfinite/identity negative tests | Deterministic failure before row metrics | Comparator contract tests |
| FESA-TOL-003 | Family membership and component tests | Exact approved components, units, coordinates and behavior | JSON family metrics |
| FESA-TOL-004 | Inclusive boundary, near-zero and RMS tests | `0.01/0.05/0.01` gates match this document | Common evaluator tests |
| FESA-TOL-005 | All-zero and one-nonzero zero-scale tests | Finite metrics and exact diagnostic | Common evaluator and JSON tests |
| FESA-TOL-006 | Row inventory and no-clamp/no-average tests | Every canonical row has one decision | Comparator row-count/identity tests |
| FESA-TOL-007 | Blocking, row-warning, RMS-warning and combined-warning tests | Correct verdict and deterministic warnings | MITC4 comparator tests |
| FESA-TOL-008 | JSON schema assertions | Required row/family/overall fields present and finite | B33/MITC4 deterministic JSON tests |
| FESA-TOL-009 | Repeated writer/comparison tests | Byte-stable ordering for the same inputs | Deterministic report tests |
| FESA-TOL-010 | ADR/doc/diff/revalidation review | All affected contracts and evidence updated together | ADR-022, feature reports, review record |
## 새 reference quantity 추가 절차
새 기능이나 quantity는 comparator code부터 추가하지 않는다.
1. Feature requirements에 exact input/CSV paths와 blocking/warning-only quantity를 정의한다.
2. Units, coordinate system, source row identity, component mapping과 result location을 정의한다.
3. Existing family에 포함할지 새 family를 만들지 dimension/behavior 기준으로 결정한다.
4. Numerical review와 reference-model에서 row precheck와 readiness를 승인한다.
5. I/O 문서에서 CSV column을 authoritative HDF5 dataset/component에 연결한다.
6. RED test로 identity, family, boundary, zero-scale, RMS와 report contract를 고정한다.
7. Comparator를 구현하고 focused/full MSVC CTest를 통과한다.
8. Fresh reference-comparison report를 생성하고 이후 physics/release gate를 진행한다.
## Tolerance 변경 관리
Tolerance 상수, 수식, family membership, row identity 또는 blocking behavior를 변경하려면
다음 항목을 함께 처리한다.
1. 사용자 명시 승인
2. 이 문서와 ADR의 decision, rationale과 tradeoff 갱신
3. Common design 및 영향받는 feature requirements/reference-model/I/O 문서 갱신
4. Boundary와 failure semantics를 먼저 고정한 RED test
5. 공통 evaluator 및 영향받는 comparator의 최소 수정
6. Focused reference tests와 전체 MSVC/CTest
7. 모든 영향 feature의 fresh reference comparison
8. 기존 physics/release evidence의 유효성 재검토
과거 report의 원본 hash, command, row count와 metric table은 소급 수정하지 않는다.
Historical status를 보존하고 새 정책으로 생성한 evidence를 별도 섹션 또는 새 report에
추가한다. Reference CSV는 tolerance 변경을 맞추기 위해 수정하지 않는다.
## 검토 체크리스트
- [ ] Exact reference input/CSV와 authoritative HDF5를 사용했는가?
- [ ] Canonical row identity와 component mapping이 기능 계약과 일치하는가?
- [ ] Missing/extra/duplicate/nonfinite row를 tolerance 전에 거부했는가?
- [ ] Family가 units, coordinates와 blocking behavior를 섞지 않는가?
- [ ] Scale을 reference 값만으로 계산했는가?
- [ ] 모든 row에 near-zero 또는 relative branch가 기록됐는가?
- [ ] Zero-scale에서 exact-zero와 finite diagnostic을 보장하는가?
- [ ] 모든 family의 scale-relative RMS를 판정했는가?
- [ ] Warning-only row/RMS failure를 deterministic warning으로 기록했는가?
- [ ] Report에 row count, scale, worst row, RMS와 overall verdict가 있는가?
- [ ] Reference artifact와 historical evidence를 변경하지 않았는가?
- [ ] Focused 및 full verification evidence가 있는가?
+19 -12
View File
@@ -1311,9 +1311,10 @@ $$
가 일치해야 한다. 따라서 이 V0 조건의 equilibrium end action은 요소 끝에 작용하는 가 일치해야 한다. 따라서 이 V0 조건의 equilibrium end action은 요소 끝에 작용하는
outward action이고, endpoint section resultant는 동일 section을 양의 local $x$ 면으로 outward action이고, endpoint section resultant는 동일 section을 양의 local $x$ 면으로
자른 값이다. Abaqus node-station force row에는 equilibrium end action이 아니라 section 자른 값이다. Abaqus element-endpoint force row에는 equilibrium end action이 아니라
resultant를 동일한 section-cut 부호로 정규화해 비교하며, interior node의 두 endpoint section resultant를 동일한 section-cut 부호로 정규화해 비교한다. Reference의 element
승인 tolerance 안에서 먼저 일치해야 한다. 승인된 component mapping은 label과 endpoint node label을 HDF5 row에 직접 대응하며 node station collapse, 대표 endpoint
선택 또는 평균은 하지 않는다. 승인된 component mapping은
`SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`이고, CSV에 없는 transverse `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`이고, CSV에 없는 transverse
`SF2`, `SF3`는 reference comparison 대상이 아니다. `SF2`, `SF3`는 reference comparison 대상이 아니다.
@@ -1529,19 +1530,25 @@ $$
\sum\mathbf M_{\mathrm{reaction}}\right\|\le\epsilon_M. \sum\mathbf M_{\mathrm{reaction}}\right\|\le\epsilon_M.
$$ $$
승인된 B33 reference row는 같은 model, step/frame, quantity, componentAbaqus 승인된 B33 reference row는 같은 model, step/frame, quantity, component로 family를 만들고
reference 값만 사용해 Abaqus reference 값만 사용해
$$ $$
\operatorname{reference\_scale}=\max_i|r_i|,\qquad S=\max_i|r_i|
\operatorname{row\_tolerance}=\operatorname{absolute\_floor}
+10^{-6}\operatorname{reference\_scale}
$$ $$
적용한다. 승인된 SI bundle의 displacement/rotation floor는 $10^{-9}$, 계산한다. $|r_i|\le0.01S$인 near-zero row는 $|f_i-r_i|\le0.01S$로,
force/moment floor는 $10^{-3}$이다. Reference 값을 zero-clamp하거나 row를 제거하지 나머지 row는 $|f_i-r_i|/|r_i|\le0.05$로 판정한다. 또한
않으며 missing, extra, duplicate, nonfinite, schema/identity mismatch는 수치 판정 전에
실패한다. 이 reference policy는 위 formulation/analytical tolerance를 대체하지 않는다. $$
\frac{\sqrt{n^{-1}\sum_i(f_i-r_i)^2}}{S}\le0.01
$$
을 family 전체에서 만족해야 한다. $S=0$이면 모든 FESA 값이 정확히 0일 때만 통과하고,
그렇지 않으면 zero-reference-scale-nonzero-error로 실패한다. 독립적인 절대오차 gate는
사용하지 않는다. Reference 값을 zero-clamp하거나 row를 제거하지 않으며 missing, extra,
duplicate, nonfinite, schema/identity mismatch는 수치 판정 전에 실패한다. 이 reference
policy는 위 formulation/analytical tolerance를 대체하지 않는다.
## 19. Numerical Risks ## 19. Numerical Risks
+47 -46
View File
@@ -12,7 +12,7 @@
- owner_agent: `io-definition-agent` - owner_agent: `io-definition-agent`
- date: `2026-08-09` - date: `2026-08-09`
- authoritative_output: `results.h5` - authoritative_output: `results.h5`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` - reference_baseline: exact read-only artifacts under `reference/cantilever beam/`
이 문서는 승인된 V0의 semantic I/O contract만 정의한다. Parser, HDF5 writer, 이 문서는 승인된 V0의 semantic I/O contract만 정의한다. Parser, HDF5 writer,
comparison tooling의 C++ API나 구현 구조는 정의하지 않으며 Abaqus full compatibility를 comparison tooling의 C++ API나 구현 구조는 정의하지 않으며 Abaqus full compatibility를
@@ -336,7 +336,7 @@ CSV는 `SF1/SM1/SM2/SM3`만 제공하고 section-cut result와 비교하므로 `
0으로 만들거나 reference row로 합성하지 않는다. HDF5의 transverse end action은 0으로 만들거나 reference row로 합성하지 않는다. HDF5의 transverse end action은
unit/analytical test와 physics sanity 대상이다. unit/analytical test와 physics sanity 대상이다.
### Frame, instance, and node-station normalization ### Frame, instance, and element-endpoint normalization
- Legacy `Frame` value `Increment 1: Step Time = 1.000`은 canonical `(Step-1, frame 0)`으로 - Legacy `Frame` value `Increment 1: Step Time = 1.000`은 canonical `(Step-1, frame 0)`으로
변환한다. 승인 bundle의 다른 increment/time string은 `schema-mismatch`다. 변환한다. 승인 bundle의 다른 increment/time string은 `schema-mismatch`다.
@@ -344,29 +344,27 @@ unit/analytical test와 physics sanity 대상이다.
`instance_name`과 case-insensitive lookup 후 raw identity 일치 여부를 확인한다. `instance_name`과 case-insensitive lookup 후 raw identity 일치 여부를 확인한다.
- `Node Label`은 instance 안의 preserved source node label로 resolve한다. Displacement와 - `Node Label`은 instance 안의 preserved source node label로 resolve한다. Displacement와
reaction key는 `(model_id,Step-1,0,instance_name,source_node_label,quantity,component)`다. reaction key는 `(model_id,Step-1,0,instance_name,source_node_label,quantity,component)`다.
- Elemental-force CSV는 element label이 없으므로 HDF5 endpoint section resultants를 - Elemental-force CSV`Element Label`은 instance 안의 preserved source element label로,
source node station으로 project한다. Boundary station은 유일한 incident endpoint를 쓴다. `Node Label`은 해당 B33 connectivity의 정확한 endpoint source node로 resolve한다.
- Interior station collapse는 정확히 두 incident B33 endpoints, 동일 section/local-axis - Section-resultant key는
orientation, 일관된 chain connectivity, 해당 node의 concentrated force/moment가 없는 `(model_id,Step-1,0,instance_name,source_element_label,source_node_label,quantity,component)`다.
경우에만 허용한다. 두 positive-face section-cut 값이 아래 승인 component tolerance 한 element에는 connectivity와 일치하는 두 endpoint 행이 정확히 존재해야 한다.
안에서 먼저 일치해야 한다. - Missing, extra, duplicate 또는 element-connectivity-mismatched endpoint row는
- Interior 값이 일치하면 stable internal element ID가 작은 endpoint를 deterministic `schema-mismatch`로 tolerance 전에 실패한다. Source node station collapse,
representative 선택한다. 두 값을 평균하지 않는다. 불일치는 `tolerance-failure`다. deterministic representative 선택 또는 endpoint 평균은 사용하지 않는다.
- Reversed connectivity, local-axis discontinuity, section jump, branch 또는 loaded interior
station은 element label 없는 legacy schema로 collapse할 수 없다. 이 approved bundle
밖에서는 element-aware reference row가 필요하며 legacy projection은 `schema-mismatch`
중단한다.
Projected canonical comparison row는 다음 fields를 가진다. Projected canonical comparison row는 다음 fields를 가진다.
```text ```text
model_id, step_name, frame_index, instance_name, source_node_label, model_id, step_name, frame_index, instance_name,
quantity, component, value, unit_dimension, coordinate_system, hdf5_dataset_path [source_element_label], source_node_label, quantity, component, value,
unit_dimension, coordinate_system, hdf5_dataset_path
``` ```
Stable ordering은 quantity inventory order, instance declaration order, stable source node `source_element_label`은 section-resultant row에 필수이고 nodal row에는 적용하지 않는다.
order, component order다. Approved model ID는 `cantilever-beam-b33`; nodal quantities는 Stable ordering은 quantity inventory order, instance declaration order, stable source
global Cartesian, section resultants는 beam local이다. node/element-endpoint order, component order다. Approved model ID는 `cantilever-beam-b33`;
nodal quantities는 global Cartesian, section resultants는 beam local이다.
### Row-set precheck and tolerance ### Row-set precheck and tolerance
@@ -376,27 +374,32 @@ identity-mismatched row가 하나라도 있으면 tolerance 계산 전에 실패
mandatory인 `SF2/SF3` 대응 end action, generalized results와 `S11`은 계약상 비교 대상이 mandatory인 `SF2/SF3` 대응 end action, generalized results와 `S11`은 계약상 비교 대상이
아니므로 extra reference row가 아니다. 아니므로 extra reference row가 아니다.
Matched rows는 같은 `model_id`, step/frame, quantity, component로 group한다. Matched rows는 같은 model/case, step/frame, logical quantity, unit dimension, coordinate
system과 blocking behavior의 component family로 group한다. B33 family는 translation
`UX/UY/UZ`, rotation `URX/URY/URZ`, reaction force `RF1/RF2/RF3`, reaction moment
`RM1/RM2/RM3`, section force `N`, section moment `T/My/Mz`다.
```text ```text
reference_scale = max(abs(reference_value_i)) S = max(abs(reference_value_i))
row_tolerance = absolute_floor + 1e-6 * reference_scale error_i = abs(fesa_value_i-reference_value_i)
row_pass = abs(fesa_value_i - reference_value_i) <= row_tolerance near_zero_band = 0.01*S
if abs(reference_value_i) <= near_zero_band:
row_pass = error_i <= near_zero_band
else:
row_pass = error_i/abs(reference_value_i) <= 0.05
relative_rms = sqrt(mean(error_i^2))/S
family_pass = all(row_pass) and relative_rms <= 0.01
``` ```
즉 exact policy는 `absolute_floor + 1e-6 * reference_scale`이다. `reference_scale` `S`는 read-only Abaqus values만 사용하고 FESA 값으로 조정하지 않는다. 독립적인
read-only Abaqus values만 사용하고 FESA 값으로 조정하지 않는다. Scale이 zero면 relative absolute-error gate와 zero clamp는 사용하지 않는다. `S=0`이면 모든 FESA 값도 exact
term은 zero다. Reference value나 작은 residue를 zero-clamp하지 않고 모든 row를 판정한다. zero일 때 relative RMS를 0으로 기록하고 통과하며, 하나라도 nonzero이면 NaN/Inf 대신
`zero-reference-scale-nonzero-error`로 실패한다. Verification report는 모든 row의 적용
| approved SI component class | absolute floor | branch와 pass/fail, family identity/components, scale, near-zero band/count, max absolute
| --- | ---: | error, scale-relative RMS와 worst row/component를 기록한다. 이 reference tolerance는
| displacement and rotation | `1e-9` | analytical/formulation tolerance를 대체하지 않는다.
| force and moment | `1e-3` |
Interior endpoint consistency도 해당 CSV quantity/component의 same Abaqus-only scale과
floor를 사용한다. Verification report는 모든 row pass/fail과 quantity별 max absolute
error, component-scale normalized error, RMS error, norm error, worst row/component를
기록한다. 이 reference tolerance는 analytical/formulation tolerance를 대체하지 않는다.
## CLI and Diagnostics Contract ## CLI and Diagnostics Contract
@@ -445,17 +448,15 @@ release approval을 이 status가 의미하지 않는다.
### Resolved numerical-review handoff ### Resolved numerical-review handoff
`NR-O03-STATION-NORMALIZATION`approved legacy bundle에 대해 unloaded, consistently `NR-O03-STATION-NORMALIZATION`2026-08-18 regenerated elemental-force CSV의
oriented two-endpoint interior station만 collapse하고, tolerance check 후 smaller stable `Element Label`로 해소됐다. Comparator는 `(instance, element label, endpoint node label,
element ID를 선택하는 규칙으로 구체화했다. Reversed/branched/loaded/jumped station은 component)`를 직접 대응하며 station collapse, 대표 endpoint 선택 또는 평균을 하지 않는다.
element-aware reference가 없는 한 비교하지 않는다.
### Reference Model Agent ### Reference Model Agent
- Exact legacy inventory, generator `Abaqus/CAE Learning Edition 2024`, source commit - Exact legacy inventory, current row schema와 stress comparison N/A를 계약에 고정한다.
`2b34d0b`, external SI provenance와 stress comparison N/A를 계약에 고정한다. - 추가 reference case의 exact paths와 수치 비교 계약은 해당 feature requirement가 정하며,
- 추가 reference model은 canonical filenames와 metadata를 사용하며 이 legacy file을 canonical naming이나 optional metadata를 readiness 조건으로 추가하지 않는다.
변경하지 않는다.
### Implementation Planning Agent ### Implementation Planning Agent
@@ -467,7 +468,7 @@ element-aware reference가 없는 한 비교하지 않는다.
### Reference Verification Agent ### Reference Verification Agent
- Artifact precheck 뒤 HDF5-to-legacy projection, node-station eligibility, row-set equality, - Artifact precheck 뒤 HDF5-to-CSV source identity projection, direct element-endpoint row-set
component-scale comparison 순서를 유지한다. equality, common family-scale comparison 순서를 유지한다.
- Missing/extra/nonfinite row를 무시하거나 `SF2/SF3`/stress reference row를 합성하지 않는다. - Missing/extra/nonfinite row를 무시하거나 `SF2/SF3`/stress reference row를 합성하지 않는다.
@@ -191,8 +191,13 @@ Confirmed defects, risks, and open issues are separated.
2. `NR-O02-DETERMINISTIC-REDUCTION`: stable COO sort and duplicate-summation rules are project policy and must be made explicit before NR-T11. 2. `NR-O02-DETERMINISTIC-REDUCTION`: stable COO sort and duplicate-summation rules are project policy and must be made explicit before NR-T11.
3. `NR-O03-STATION-NORMALIZATION`: reversed connectivity/local-axis orientation and legitimate jumps at loaded interior nodes need an explicit downstream row-normalization/eligibility rule. The legacy baseline may use its documented stable orientation and unloaded interior stations, but mismatch must never be averaged. NR-T07 covers element signs. 3. `NR-O03-STATION-NORMALIZATION`: reversed connectivity/local-axis orientation and legitimate jumps at loaded interior nodes need an explicit downstream row-normalization/eligibility rule. The legacy baseline may use its documented stable orientation and unloaded interior stations, but mismatch must never be averaged. NR-T07 covers element signs.
2026-08-18 amendment: `NR-O03-STATION-NORMALIZATION` is resolved and superseded for the
approved case. The regenerated elemental-force CSV supplies `Element Label`, so comparison uses
direct `(instance, element label, endpoint node label, component)` identity and never collapses,
selects or averages adjacent endpoints.
No open issue requires formulation revision. NR-O01/NR-O02 are implementation-planning No open issue requires formulation revision. NR-O01/NR-O02 are implementation-planning
handoffs; NR-O03 belongs to I/O and reference-model contracts. handoffs; the resolved NR-O03 identity is fixed by the I/O and reference-model contracts.
## Required Revisions ## Required Revisions
@@ -206,7 +211,7 @@ handoffs; NR-O03 belongs to I/O and reference-model contracts.
### Reference Model Agent ### Reference Model Agent
- Make NR-O03 orientation and unloaded-interior assumptions explicit without modifying the approved legacy artifacts. - Enforce the resolved NR-O03 direct element-endpoint identity without modifying reference artifacts.
## Downstream Handoff ## Downstream Handoff
@@ -10,11 +10,45 @@
- source_io_definition: `docs/linear-static-3d-euler-beam/io.md` - source_io_definition: `docs/linear-static-3d-euler-beam/io.md`
- source_implementation_plan: `docs/linear-static-3d-euler-beam/implementation-plan.md` - source_implementation_plan: `docs/linear-static-3d-euler-beam/implementation-plan.md`
- source_implementation_report: `docs/linear-static-3d-euler-beam/implementation-report.md` - source_implementation_report: `docs/linear-static-3d-euler-beam/implementation-report.md`
- status: `pass-for-physics-evaluation` - historical_status: `pass-for-physics-evaluation`
- current_status: `pass-for-physics-evaluation`
- superseded_on: `2026-08-18`
- revalidated_on: `2026-08-18`
- owner_agent: `reference-verification-agent` - owner_agent: `reference-verification-agent`
- date: `2026-08-09` - date: `2026-08-09`
- review_fix_date: `2026-08-10` - review_fix_date: `2026-08-10`
The 2026-08-09 evidence below is preserved as a historical record of the former 11-station,
component-scale comparison. ADR-022 and the regenerated 20-row element-endpoint CSV supersede
that identity and tolerance. The following revalidation is the current reference-gate evidence.
## 2026-08-18 Common-policy Revalidation
`cmake --build .harness/build --config Debug` passed, followed by
`ctest --test-dir .harness/build -C Debug --output-on-failure`: 214/214 tests passed.
The B33 reference test generated
`.harness/build/reference/cantilever-beam-b33/comparison.json` from the exact read-only
reference paths.
- exact canonical rows: 212 = 66 displacement + 66 reaction + 80 direct element-endpoint
section-resultant rows
- exact families: 6; all row gates and family RMS gates passed
- direct section identity: 20 endpoint rows, 10 B33 elements x 2 connectivity-matched endpoints
- warnings, identity/schema/nonfinite failures: 0
- overall verdict: `passed=true`
| family | rows | scale | near-zero rows | max absolute error | scale-relative RMS |
| --- | ---: | ---: | ---: | ---: | ---: |
| translation `UX/UY/UZ` | 33 | `1.90476272e-2` | 23 | `5.333229170789711e-10` | `8.459623217000381e-9` |
| rotation `URX/URY/URZ` | 33 | `2.85714399e-3` | 23 | `1.000013943180944e-10` | `1.1723557271997219e-8` |
| reaction force `RF1/RF2/RF3` | 33 | `1.0e6` | 32 | `8.195638656616211e-7` | `2.0922817757699256e-13` |
| reaction moment `RM1/RM2/RM3` | 33 | `1.0e7` | 32 | `5.0514936447143555e-6` | `8.811160558471221e-14` |
| section force `N` | 20 | `0` | 20 | `0` | `0` |
| section moment `T/My/Mz` | 60 | `1.0e7` | 41 | `1.2499958951957524e-1` | `2.2910664449287597e-9` |
All values above come from the generated JSON ledger. The historical report body starts below
and must not be read as the current identity/tolerance evidence.
The prerequisite build/test report has status The prerequisite build/test report has status
`pass-for-reference-verification`. This report applies only the approved Abaqus `pass-for-reference-verification`. This report applies only the approved Abaqus
B33 reference tolerance. It does not approve physics sanity or release readiness. B33 reference tolerance. It does not approve physics sanity or release readiness.
@@ -12,9 +12,10 @@
- status: `ready-for-implementation-planning` - status: `ready-for-implementation-planning`
- owner_agent: `reference-model-agent` - owner_agent: `reference-model-agent`
- date: `2026-08-09` - date: `2026-08-09`
- amended_on: `2026-08-18`
- approved_reference_model: `cantilever-beam-b33` - approved_reference_model: `cantilever-beam-b33`
- approved_reference_schema: `abaqus-cae-report-csv-v0` - approved_reference_identity: direct source-node and element-endpoint rows
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` - reference_baseline: exact current files under `reference/cantilever beam/`
이 문서는 구현 전에 필요한 code verification, analytical solution verification 및 이 문서는 구현 전에 필요한 code verification, analytical solution verification 및
approved B33 reference comparison의 모델·artifact 계약을 정의한다. 이 status는 모델과 approved B33 reference comparison의 모델·artifact 계약을 정의한다. 이 status는 모델과
@@ -45,7 +46,7 @@ test fixtures이며 reference artifact bundle로 가장하지 않는다.
논리 모델 `cantilever-beam-b33`의 exact read-only legacy bundle만 사용하여 FESA 논리 모델 `cantilever-beam-b33`의 exact read-only legacy bundle만 사용하여 FESA
`results.h5`의 displacement, reaction 및 endpoint section resultant를 Abaqus/CAE report `results.h5`의 displacement, reaction 및 endpoint section resultant를 Abaqus/CAE report
CSV row와 비교한다. Artifact precheck와 exact row-set matching이 먼저 통과해야 하며, CSV row와 비교한다. Artifact precheck와 exact row-set matching이 먼저 통과해야 하며,
수치 비교는 component-scale mixed tolerance를 사용한다. Axial `S11` output은 필수지만 수치 비교는 ADR-022의 common family-scale tolerance를 사용한다. Axial `S11` output은 필수지만
Abaqus beam stress comparison은 명시적 N/A다. Abaqus beam stress comparison은 명시적 N/A다.
### Excluded validation scope ### Excluded validation scope
@@ -136,8 +137,8 @@ single-step deck and changes only the named condition.
- boundary_conditions: source node 1, DOFs 1 through 6 fixed - boundary_conditions: source node 1, DOFs 1 through 6 fixed
- load: source node 11, global DOF 3, magnitude `-1e6` N - load: source node 11, global DOF 3, magnitude `-1e6` N
- model_id: `cantilever-beam-b33` - model_id: `cantilever-beam-b33`
- logical_schema: `abaqus-cae-report-csv-v0` - historical_schema_record: `abaqus-cae-report-csv-v0` for the 2026-08-09 inventory only
- source_commit: `2b34d0b` - historical_source_commit: `2b34d0b` for the 2026-08-09 inventory only
- generator: `Abaqus/CAE Learning Edition 2024` - generator: `Abaqus/CAE Learning Edition 2024`
- units: SI - units: SI
- nodal_coordinate_system: global Cartesian - nodal_coordinate_system: global Cartesian
@@ -145,7 +146,7 @@ single-step deck and changes only the named condition.
- step_name: `Step-1` - step_name: `Step-1`
- increment: `1` - increment: `1`
- step_time: `1.0` - step_time: `1.0`
- artifact_status: all four exact paths present; structural precheck observed; FESA comparison not run - artifact_status: all four exact paths present; 20-row element-endpoint comparison passed on 2026-08-18
- stress: N/A for Abaqus reference comparison; mandatory FESA `S11` remains covered by unit/analytical and HDF5 schema tests - stress: N/A for Abaqus reference comparison; mandatory FESA `S11` remains covered by unit/analytical and HDF5 schema tests
The input and CSV numeric reference values are not recalculated, repaired, rounded, clamped or The input and CSV numeric reference values are not recalculated, repaired, rounded, clamped or
@@ -193,93 +194,44 @@ For this approved legacy bundle:
- `README.md`: N/A - `README.md`: N/A
- stress CSV: N/A because beam stress reference comparison is outside the approved V0 scope - stress CSV: N/A because beam stress reference comparison is outside the approved V0 scope
The approved design and this contract record model ID, provenance, generator, source commit, This contract records the exact required paths, row identities, comparison quantities, tolerance
units, coordinate systems, step/frame identity, logical CSV schema, exact inventory, tolerance policy and the stress N/A reason. Historical schema/provenance fields are retained as dated
policy and the stress N/A reason. The optional metadata file's absence and the approved legacy inventory only and are not readiness gates. The optional metadata file's absence and the approved
README/stress exclusions therefore do not change the record to `needs-reference-artifacts`. legacy README/stress exclusions do not change the record to `needs-reference-artifacts`.
### Future reference bundles ### Later reference cases and optional metadata
Every later reference model shall use this structure unless its approved requirement explicitly Each later feature requirement names its exact input and comparison CSV paths, required row
marks a quantity N/A: identity/components and N/A quantities. Canonical filenames, `README.md`, `metadata.json`,
generator/version, provenance, duplicated unit/schema fields and a portfolio-wide directory layout
```text are not readiness gates unless that feature explicitly makes one part of its numerical comparison
reference/ contract. No agent may invent unknown provenance. If optional metadata exists, inspect it read-only
<model-id>/ and report disagreement without rewriting the reference artifacts.
model.inp
metadata.json # optional
<model-id>_displacements.csv
<model-id>_reactions.csv
<model-id>_internalforces.csv
<model-id>_stresses.csv
README.md
```
CSV names are canonical `<model-id>_*.csv` names. `README.md` is mandatory for later bundles;
`metadata.json` is optional. A quantity CSV may be omitted only when the upstream acceptance
contract explicitly records N/A and gives its verification replacement. Missing required files
or required Reference Model Contract provenance keep that model at `needs-reference-artifacts`.
## Reference Metadata Contract
This document is the required source of truth for the following metadata. A later bundle may
optionally duplicate it in `metadata.json` using at least this schema:
```json
{
"feature_id": "linear-static-3d-euler-beam",
"model_id": "<model-id>",
"artifact_status": "needs-reference-artifacts | ready-for-verification",
"input_file": "model.inp",
"abaqus_version": "<exact generator/version>",
"generation_owner": "<person or approved procedure>",
"generation_date": "<YYYY-MM-DD>",
"source_commit": "<commit>",
"units": "<consistent unit system>",
"coordinate_system": "<nodal and element result systems>",
"analysis_type": "single linear static",
"element_types": ["B33"],
"step_name": "Step-1",
"increment": 1,
"step_time": 1.0,
"output_requests": ["U", "RF", "SF"],
"reference_csv_schema_version": "<approved schema>",
"reference_csv_files": ["<canonical filenames>"],
"tolerance_policy": "<approved quantity/component policy>",
"limitations": ["<known limitations and explicit N/A quantities>"]
}
```
No agent may invent unknown provenance fields or mark a bundle ready merely because filenames
exist. An absent `metadata.json` is allowed. If the file exists, inventory it read-only and report
any disagreement with this contract or stored artifacts as an upstream contract/provenance issue.
## Abaqus Reference CSV Requirements ## Abaqus Reference CSV Requirements
Header comparison trims whitespace around each comma-separated field but does not rename fields. Header comparison trims whitespace around each comma-separated field but does not rename fields.
For every file, `Frame` must normalize exactly from For every file, `Frame` must normalize exactly from
`Increment 1: Step Time = 1.000` to `(Step-1, frame 0)`, `Part Instance Name` must resolve to the `Increment 1: Step Time = 1.000` to `(Step-1, frame 0)`, `Part Instance Name` must resolve to the
preserved instance identity, `Node Label` must be a unique source-node station, and all projected preserved instance identity, every declared source identity must be unique, and all projected
numeric values must be finite. numeric values must be finite.
| exact legacy path | expected trimmed header | unique row key | observed inventory | | exact legacy path | expected trimmed header | unique row key | observed inventory |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `reference/cantilever beam/cantilever beam displacements.csv` | `Frame, Part Instance Name, Node Label, U-U1, U-U2, U-U3, UR-UR1, UR-UR2, UR-UR3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed | | `reference/cantilever beam/cantilever beam displacements.csv` | `Frame, Part Instance Name, Node Label, U-U1, U-U2, U-U3, UR-UR1, UR-UR2, UR-UR3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed |
| `reference/cantilever beam/cantilever beam reactions.csv` | `Frame, Part Instance Name, Node Label, RF-RF1, RF-RF2, RF-RF3, RM-RM1, RM-RM2, RM-RM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed | | `reference/cantilever beam/cantilever beam reactions.csv` | `Frame, Part Instance Name, Node Label, RF-RF1, RF-RF2, RF-RF3, RM-RM1, RM-RM2, RM-RM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed |
| `reference/cantilever beam/cantilever beam elemental forces.csv` | `Frame, Part Instance Name, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed | | `reference/cantilever beam/cantilever beam elemental forces.csv` | `Frame, Part Instance Name, Element Label, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3` | `(Frame, Part Instance Name, Element Label, Node Label)` | 20 rows; 10 B33 elements x 2 connectivity-matched endpoints; header/key/finite/arity checks observed |
The wide-row key becomes unique canonical component rows after adding `quantity` and `component`. The wide-row key becomes unique canonical component rows after adding `quantity` and `component`.
Missing, extra, duplicate, nonfinite, header/schema or identity mismatch stops comparison as Missing, extra, duplicate, nonfinite, header/schema or identity mismatch stops comparison as
`needs-reference-artifacts` or `schema-mismatch`. No bad or near-zero row may be silently dropped. `needs-reference-artifacts` or `schema-mismatch`. No bad or near-zero row may be silently dropped.
For the elemental-force CSV, source node station is not an element-end identity. A boundary For the elemental-force CSV, `(Part Instance Name, Element Label, Node Label)` is the source
station uses its only incident endpoint. An interior station may collapse exactly two endpoints element-endpoint identity. Each B33 element must have exactly two rows whose node labels match its
only when chain connectivity, section and local axes are consistent and the station has no ordered input connectivity. Every row maps directly to the corresponding HDF5
concentrated force/moment. The two positive-local-x section-cut values must first agree within the `[element,endpoint,N/T/My/Mz]` row. Missing, extra, duplicate or connectivity-mismatched endpoint
approved component tolerance. If they agree, choose the endpoint with smaller stable internal identity fails as `schema-mismatch` before tolerance. The comparator does not collapse endpoints to
element ID; never average. Reversed orientation, branch, section jump, local-axis discontinuity or a node station, choose a representative or average values.
loaded interior station requires an element-aware future schema and is a `schema-mismatch` under
this legacy schema.
## Coverage Matrix ## Coverage Matrix
@@ -288,20 +240,20 @@ the canonical V0 step identity.
| verification quantity | requirement ids | model_id | FESA HDF5 dataset | legacy CSV and components | row identity/location | tolerance | verification method | status | | verification quantity | requirement ids | model_id | FESA HDF5 dataset | legacy CSV and components | row identity/location | tolerance | verification method | status |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| nodal displacement/rotation | 003, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/displacement` | `reference/cantilever beam/cantilever beam displacements.csv`: `U-U1/U-U2/U-U3 -> UX/UY/UZ`, `UR-UR1/UR-UR2/UR-UR3 -> URX/URY/URZ` | preserved instance + source node; global nodal | `1e-9 + 1e-6*reference_scale` per displacement/rotation component | HDF5-to-read-only CSV after artifact/row-set precheck | ready for implementation planning; comparison not run | | nodal displacement/rotation | 003, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/displacement` | `reference/cantilever beam/cantilever beam displacements.csv`: `U-U1/U-U2/U-U3 -> UX/UY/UZ`, `UR-UR1/UR-UR2/UR-UR3 -> URX/URY/URZ` | preserved instance + source node; global nodal | common translation/rotation family policy | HDF5-to-read-only CSV after artifact/row-set precheck | passed 2026-08-18 |
| nodal reaction force | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RF-RF1/RF-RF2/RF-RF3 -> RF1/RF2/RF3` | preserved instance + source node; global nodal | `1e-3 + 1e-6*reference_scale` per force component | HDF5-to-read-only CSV plus global equilibrium | ready for implementation planning; comparison not run | | nodal reaction force | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RF-RF1/RF-RF2/RF-RF3 -> RF1/RF2/RF3` | preserved instance + source node; global nodal | common reaction-force family policy | HDF5-to-read-only CSV plus global equilibrium | passed 2026-08-18 |
| nodal reaction moment | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RM-RM1/RM-RM2/RM-RM3 -> RM1/RM2/RM3` | preserved instance + source node; global nodal | `1e-3 + 1e-6*reference_scale` per moment component | HDF5-to-read-only CSV plus moment equilibrium | ready for implementation planning; comparison not run | | nodal reaction moment | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RM-RM1/RM-RM2/RM-RM3 -> RM1/RM2/RM3` | preserved instance + source node; global nodal | common reaction-moment family policy | HDF5-to-read-only CSV plus moment equilibrium | passed 2026-08-18 |
| section axial force | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SF-SF1 -> N` | positive-local-x endpoint projected to eligible source node station | `1e-3 + 1e-6*reference_scale` for `N` | endpoint consistency, deterministic station selection, HDF5-to-CSV | ready for implementation planning; comparison not run | | section axial force | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SF-SF1 -> N` | direct instance + source element + endpoint node; beam local | common section-force family policy | direct HDF5-to-endpoint CSV comparison | passed 2026-08-18 |
| section moments/torsion | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SM-SM1 -> My`, `SM-SM2 -> Mz`, `SM-SM3 -> T` | positive-local-x endpoint projected to eligible source node station; beam local | `1e-3 + 1e-6*reference_scale` separately for `My`, `Mz`, `T` | endpoint consistency, deterministic station selection, HDF5-to-CSV | ready for implementation planning; comparison not run | | section moments/torsion | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SM-SM1 -> My`, `SM-SM2 -> Mz`, `SM-SM3 -> T` | direct instance + source element + endpoint node; beam local | common section-moment family policy | direct HDF5-to-endpoint CSV comparison | passed 2026-08-18 |
| equilibrium end action | 031, 035, 043 | analytical models and physics portfolio | `/steps/Step-1/frames/0/element/end_force_local` | Abaqus CSV N/A for direct outward-action comparison | element endpoint `xi=-1,+1`; local outward action `[FX,FY,FZ,MX,MY,MZ]` | analytical normalized `1e-12`; residual `1e-10` | unit/analytical end-sign tests and later physics sanity | planned | | equilibrium end action | 031, 035, 043 | analytical models and physics portfolio | `/steps/Step-1/frames/0/element/end_force_local` | Abaqus CSV N/A for direct outward-action comparison | element endpoint `xi=-1,+1`; local outward action `[FX,FY,FZ,MX,MY,MZ]` | analytical normalized `1e-12`; residual `1e-10` | unit/analytical end-sign tests and later physics sanity | planned |
| generalized strain/resultant | 029, 031, 035 | code and analytical models | `/steps/Step-1/frames/0/element/generalized_strain` and `/steps/Step-1/frames/0/element/generalized_resultant` | Abaqus CSV N/A | two Gauss points; beam local | matrix/formulation normalized `1e-12`, analytical relative `1e-9` | formulation/unit/HDF5 schema tests | planned | | generalized strain/resultant | 029, 031, 035 | code and analytical models | `/steps/Step-1/frames/0/element/generalized_strain` and `/steps/Step-1/frames/0/element/generalized_resultant` | Abaqus CSV N/A | two Gauss points; beam local | matrix/formulation normalized `1e-12`, analytical relative `1e-9` | formulation/unit/HDF5 schema tests | planned |
| axial stress | 029, 032, 035 | axial/local-z analytical models | `/steps/Step-1/frames/0/element/stress_s11` | stress CSV N/A; Abaqus beam stress reference comparison N/A | element, Gauss point, input section point or `fesa-default` centroid | analytical relative `1e-9`; exact unit/row schema | unit/analytical recovery and HDF5 schema tests | planned; reference N/A | | axial stress | 029, 032, 035 | axial/local-z analytical models | `/steps/Step-1/frames/0/element/stress_s11` | stress CSV N/A; Abaqus beam stress reference comparison N/A | element, Gauss point, input section point or `fesa-default` centroid | analytical relative `1e-9`; exact unit/row schema | unit/analytical recovery and HDF5 schema tests | planned; reference N/A |
For every matched reference group, For every matched family, `S=max(abs(Abaqus reference rows))` uses only read-only reference values.
`reference_scale=max(abs(Abaqus reference rows))` for the same model, step/frame, quantity and Rows with `abs(reference)<=0.01*S` use `error<=0.01*S`; other rows use relative error `<=0.05`;
component, and `row_tolerance=absolute_floor+1e-6*reference_scale`. Abaqus values alone set the the family also requires `RMS(error)/S<=0.01`. A zero-scale family requires exact-zero FESA values.
scale. A zero scale uses only the floor. Every row decision and max absolute, component-scale Every row branch/decision and family scale, near-zero band/count, max absolute error,
normalized, RMS, norm and worst-row/component metrics must be reported. scale-relative RMS, worst row/component and zero-scale diagnostic must be reported.
### Complete must-requirement coverage ### Complete must-requirement coverage
@@ -346,19 +298,20 @@ it does not waive the requirement.
| `FESA-REQ-LS3DEB-033` | `smoke-b33-cli`, diagnostic negative cases | CLI exit-code/field/order integration tests; CSV N/A | | `FESA-REQ-LS3DEB-033` | `smoke-b33-cli`, diagnostic negative cases | CLI exit-code/field/order integration tests; CSV N/A |
| `FESA-REQ-LS3DEB-034` | every implementation model/test | per-step RED/GREEN/VERIFY and full MSVC/CTest evidence; model CSV N/A | | `FESA-REQ-LS3DEB-034` | every implementation model/test | per-step RED/GREEN/VERIFY and full MSVC/CTest evidence; model CSV N/A |
| `FESA-REQ-LS3DEB-035` | `NR-T01` through `NR-T11` and analytical inventory | exact numerical criteria in model records | | `FESA-REQ-LS3DEB-035` | `NR-T01` through `NR-T11` and analytical inventory | exact numerical criteria in model records |
| `FESA-REQ-LS3DEB-036` | `cantilever-beam-b33` and comparison-policy unit fixtures | Abaqus-only component scale and exact formula tests | | `FESA-REQ-LS3DEB-036` | `cantilever-beam-b33` and comparison-policy unit fixtures | exact family membership and Abaqus-only scale tests |
| `FESA-REQ-LS3DEB-037` | same comparison fixtures, including zero-scale groups | exact SI floors and zero-scale tests | | `FESA-REQ-LS3DEB-037` | same comparison fixtures, including boundary and zero-scale families | near-zero, relative-row, scale-relative RMS and zero-scale tests |
| `FESA-REQ-LS3DEB-038` | malformed reference cases in `neg-b33-input-contract` | pre-tolerance fail-fast and no-clamp/no-drop tests | | `FESA-REQ-LS3DEB-038` | malformed reference cases in `neg-b33-input-contract` | pre-tolerance fail-fast and no-clamp/no-drop tests |
| `FESA-REQ-LS3DEB-039` | verification-report schema fixture | every row decision and aggregate/worst metrics test | | `FESA-REQ-LS3DEB-039` | verification-report schema fixture | every row branch/decision and family aggregate/worst metrics test |
| `FESA-REQ-LS3DEB-040` | exact legacy Artifact Bundle Contract | inventory plus `git diff --exit-code -- reference/` process check | | `FESA-REQ-LS3DEB-040` | exact legacy Artifact Bundle Contract | inventory plus `git diff --exit-code -- reference/` process check |
| `FESA-REQ-LS3DEB-041` | approved artifact precheck | four files, B33, exact headers, unique keys, finite values | | `FESA-REQ-LS3DEB-041` | approved artifact precheck | four files, B33, exact headers, unique keys, finite values |
| `FESA-REQ-LS3DEB-042` | three comparison quantities and station normalization | exact component mapping, endpoint consistency and no-average tests | | `FESA-REQ-LS3DEB-042` | three comparison quantities and direct endpoint projection | exact component mapping plus missing/extra/duplicate/connectivity-mismatch endpoint tests |
| `FESA-REQ-LS3DEB-043` | gate audit; later physics portfolio | reference execution N/A at this step; enforce build/test -> comparison -> physics -> release order | | `FESA-REQ-LS3DEB-043` | gate audit; later physics portfolio | reference execution N/A at this step; enforce build/test -> comparison -> physics -> release order |
| `FESA-REQ-LS3DEB-044` | process/Git diff audit and limitations review | no reference execution/mutation and no out-of-scope support claim | | `FESA-REQ-LS3DEB-044` | process/Git diff audit and limitations review | no reference execution/mutation and no out-of-scope support claim |
## Artifact Acceptance Checklist ## Artifact Acceptance Checklist
Read-only inventory inspection on `2026-08-09` established the following pre-implementation facts: The following is historical 2026-08-09 pre-implementation inventory evidence. Its 11-row
elemental-force observation is superseded by the 2026-08-18 regenerated 20-row endpoint file:
- all four exact legacy paths exist; - all four exact legacy paths exist;
- the input declares `TYPE=B33`; - the input declares `TYPE=B33`;
@@ -370,6 +323,11 @@ Read-only inventory inspection on `2026-08-09` established the following pre-imp
- absent `metadata.json` is allowed by project-wide policy; legacy `README.md` and stress CSV are accepted N/A exceptions; - absent `metadata.json` is allowed by project-wide policy; legacy `README.md` and stress CSV are accepted N/A exceptions;
- no reference value was recalculated and no comparison was performed. - no reference value was recalculated and no comparison was performed.
Read-only inspection on `2026-08-18` confirms that the elemental-force CSV has 20 rows,
exactly two connectivity-matched endpoint rows for each of the 10 B33 elements. Displacement and
reaction CSVs remain 11-row source-node tables. The fresh comparison passed on 2026-08-18;
current metrics are recorded in `reference-comparison.md`.
Before an actual comparison, tooling must repeat all artifact checks, verify the exact Frame and Before an actual comparison, tooling must repeat all artifact checks, verify the exact Frame and
instance identities, compare the complete projected row sets, and stop on any failure. The Step AC instance identities, compare the complete projected row sets, and stop on any failure. The Step AC
must also show no working-tree diff under `reference/`. Passing this checklist is not a reference must also show no working-tree diff under `reference/`. Passing this checklist is not a reference
@@ -383,8 +341,8 @@ comparison pass.
model. model.
- `NR-O01` (official oneMKL PARDISO contract) and `NR-O02` (deterministic duplicate-reduction - `NR-O01` (official oneMKL PARDISO contract) and `NR-O02` (deterministic duplicate-reduction
algorithm) remain implementation-planning inputs, not reference artifact defects. algorithm) remain implementation-planning inputs, not reference artifact defects.
- Future reversed, branched, loaded-interior or section-jump reference models require an - Future reversed, branched, loaded-interior or section-jump reference models use the same
element-aware canonical CSV schema; the legacy node-station schema must not be generalized. element-endpoint identity and require their own approved reference coverage.
### Implementation Planning Agent ### Implementation Planning Agent
@@ -403,8 +361,8 @@ artifacts for code/analytical fixtures.
### Reference Verification Agent ### Reference Verification Agent
Run `ARTIFACT CHECK -> HDF5 ROW PROJECTION -> EXACT ROW-SET CHECK -> ENDPOINT CONSISTENCY -> Run `ARTIFACT CHECK -> HDF5 ROW PROJECTION -> EXACT ROW-SET CHECK -> DIRECT ENDPOINT MATCH ->
COMPONENT-SCALE COMPARE -> REPORT`. Use only the exact legacy files and the HDF5 paths/component COMMON FAMILY-SCALE COMPARE -> REPORT`. Use only the exact legacy files and the HDF5 paths/component
mappings in the Coverage Matrix. Do not synthesize `SF2/SF3` or stress rows, clamp values, omit mappings in the Coverage Matrix. Do not synthesize `SF2/SF3` or stress rows, clamp values, omit
rows, average interior endpoints, or change the approved tolerance. rows, average interior endpoints, or change the approved tolerance.
+7 -1
View File
@@ -19,11 +19,17 @@
- reference_model_id: `cantilever-beam-b33` - reference_model_id: `cantilever-beam-b33`
- reference_schema: `abaqus-cae-report-csv-v0` - reference_schema: `abaqus-cae-report-csv-v0`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` - reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b`
- status: `ready-for-release` - historical_status: `ready-for-release`
- current_status: `superseded-pending-revalidation`
- superseded_on: `2026-08-18`
- owner_agent: `release-agent` - owner_agent: `release-agent`
- date: `2026-08-10` - date: `2026-08-10`
- final_review_source_head: `b7a1258ce0f36a85b888e23470cf9d936a7595cd` - final_review_source_head: `b7a1258ce0f36a85b888e23470cf9d936a7595cd`
The readiness evidence below is historical. The 2026-08-18 build/reference revalidation passed,
but fresh physics and release audits are still required before a current `ready-for-release`
verdict may be issued.
This is an internal feature-readiness verdict. It authorizes no publish, deploy, package, tag, This is an internal feature-readiness verdict. It authorizes no publish, deploy, package, tag,
push, external release, or reference-artifact operation. push, external release, or reference-artifact operation.
@@ -10,7 +10,7 @@
- approved_design: `docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md` - approved_design: `docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md`
- approval_basis: design `status: approved`, user approval on `2026-08-08`, and amendment on `2026-08-09` - approval_basis: design `status: approved`, user approval on `2026-08-08`, and amendment on `2026-08-09`
- source_formulation: `docs/linear-static-3d-euler-beam/formulation.md` - source_formulation: `docs/linear-static-3d-euler-beam/formulation.md`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b` - reference_baseline: exact read-only artifacts under `reference/cantilever beam/`
## Purpose ## Purpose
@@ -113,7 +113,7 @@ compatibility, 새로운 수학 계약 또는 새로운 구현 정책을 추가
- nodal_displacement: required, global six components, HDF5-to-Abaqus CSV comparison - nodal_displacement: required, global six components, HDF5-to-Abaqus CSV comparison
- reaction: required, global six components plus global force/moment equilibrium - reaction: required, global six components plus global force/moment equilibrium
- equilibrium_end_action: required, local endpoint six components; unit/analytical and physics tests - equilibrium_end_action: required, local endpoint six components; unit/analytical and physics tests
- section_resultant: required, endpoint `[N,T,My,Mz]`, node-station-normalized reference comparison - section_resultant: required, endpoint `[N,T,My,Mz]`, element-endpoint reference comparison
- generalized_strain_and_resultant: required at two Gauss points; formulation and schema tests - generalized_strain_and_resultant: required at two Gauss points; formulation and schema tests
- stress: axial `S11` required; Abaqus reference comparison N/A - stress: axial `S11` required; Abaqus reference comparison N/A
- residual: required, free-DOF and normalized global equilibrium checks - residual: required, free-DOF and normalized global equilibrium checks
@@ -122,21 +122,22 @@ compatibility, 새로운 수학 계약 또는 새로운 구현 정책을 추가
## Tolerance Policy ## Tolerance Policy
- **FESA-REQ-LS3DEB-036** — The approved B33 reference comparison shall group rows by the same model, step/frame, quantity, and component, compute `reference_scale` only from read-only Abaqus values, and apply `absolute_floor + 1e-6 * reference_scale` to every matched row. - **FESA-REQ-LS3DEB-036** — The approved B33 reference comparison shall group rows into translation, rotation, reaction-force, reaction-moment, section-force and section-moment families and compute each family scale `S=max(abs(reference))` only from read-only Abaqus values.
- **FESA-REQ-LS3DEB-037**For the approved SI bundle, displacement and rotation shall use `absolute_floor=1e-9`, force and moment shall use `absolute_floor=1e-3`, and a zero component scale shall use the applicable absolute floor alone. - **FESA-REQ-LS3DEB-037**A matched row with `abs(reference)<=0.01*S` shall pass when `abs(fesa-reference)<=0.01*S`; every other row shall pass when its relative error is at most `0.05`; each family shall also satisfy `RMS(error)/S<=0.01`. No independent absolute-error gate is used, and a zero-scale family passes only when every FESA value is exactly zero.
- **FESA-REQ-LS3DEB-038** — Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation. - **FESA-REQ-LS3DEB-038** — Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation.
- **FESA-REQ-LS3DEB-039** — The verification report shall record every row decision and maximum absolute error, component-scale normalized error, RMS error, norm error, and worst row/component for each compared quantity. - **FESA-REQ-LS3DEB-039** — The verification report shall record every row decision and comparison branch, family identity/components, reference scale, near-zero band/count, maximum absolute error, scale-relative RMS, worst row/component and zero-scale diagnostic for each compared family.
## Reference Artifact Requirements ## Reference Artifact Requirements
The approved logical model is `cantilever-beam-b33`, schema is The approved logical model is `cantilever-beam-b33`. The input header records
`abaqus-cae-report-csv-v0`, source commit is `2b34d0b`, generator is
`Abaqus/CAE Learning Edition 2024`, and the coordinate/output contract is global Cartesian `Abaqus/CAE Learning Edition 2024`, and the coordinate/output contract is global Cartesian
nodal output plus beam-local section-force output at `Step-1`, increment 1, step time 1.0. nodal output plus beam-local section-force output at `Step-1`, increment 1, step time 1.0.
The historical `abaqus-cae-report-csv-v0` and source commit `2b34d0b` inventory is not a
readiness requirement for the regenerated elemental-force CSV.
- **FESA-REQ-LS3DEB-040** — The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle. - **FESA-REQ-LS3DEB-040** — The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle.
- **FESA-REQ-LS3DEB-041** — Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start. - **FESA-REQ-LS3DEB-041** — Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start.
- **FESA-REQ-LS3DEB-042** — Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and node-station-normalized section resultant by `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; adjacent interior endpoints shall first agree within approved tolerance and shall not be averaged to hide a mismatch. - **FESA-REQ-LS3DEB-042** — Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and section resultants by direct `(instance, element label, endpoint node label, component)` identity using `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; missing, extra, duplicate or connectivity-mismatched endpoint rows shall fail before tolerance and shall not be collapsed or averaged.
- **FESA-REQ-LS3DEB-043** — Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations. - **FESA-REQ-LS3DEB-043** — Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations.
- **FESA-REQ-LS3DEB-044** — FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior. - **FESA-REQ-LS3DEB-044** — FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior.
@@ -180,13 +181,13 @@ nodal output plus beam-local section-force output at `Step-1`, increment 1, step
| FESA-REQ-LS3DEB-033 | The CLI shall support `fesa.exe <model.inp> --output <results.h5>`, default output to the current directory's `results.h5`, use exit codes `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5`, and emit `severity`, `code`, `file`, `line`, `keyword`, `entity_identity`, and `message` diagnostics to stderr in deterministic order. | output | Stabilize automation and failure classification. | Approved design §10 | must | CLI integration and diagnostic ordering tests | Default/explicit output works and every failure class returns its exact code and complete ordered fields. | Exact codes, fields, and order | io-definition-agent; implementation-planning-agent | approved | | FESA-REQ-LS3DEB-033 | The CLI shall support `fesa.exe <model.inp> --output <results.h5>`, default output to the current directory's `results.h5`, use exit codes `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5`, and emit `severity`, `code`, `file`, `line`, `keyword`, `entity_identity`, and `message` diagnostics to stderr in deterministic order. | output | Stabilize automation and failure classification. | Approved design §10 | must | CLI integration and diagnostic ordering tests | Default/explicit output works and every failure class returns its exact code and complete ordered fields. | Exact codes, fields, and order | io-definition-agent; implementation-planning-agent | approved |
| FESA-REQ-LS3DEB-034 | Every production C++ behavior shall be developed in one step as GoogleTest `RED -> GREEN -> VERIFY`, with a related C++ test file, focused CTest evidence, full MSVC x64 Debug build/CTest evidence, at least one discovered test, and no new warning under the FESA target's `/W4 /WX` policy. | verification | Enforce project TDD and warning policy. | Approved design §§11.1, 11.4; ADR-012 | must | Implementation report; build/CTest logs | The related test fails first, then focused/full tests pass, discovery finds tests, and FESA emits no warning. | Zero test failures and new warnings | implementation-planning-agent; implementation-agent; build-test-executor-agent | approved | | FESA-REQ-LS3DEB-034 | Every production C++ behavior shall be developed in one step as GoogleTest `RED -> GREEN -> VERIFY`, with a related C++ test file, focused CTest evidence, full MSVC x64 Debug build/CTest evidence, at least one discovered test, and no new warning under the FESA target's `/W4 /WX` policy. | verification | Enforce project TDD and warning policy. | Approved design §§11.1, 11.4; ADR-012 | must | Implementation report; build/CTest logs | The related test fails first, then focused/full tests pass, discovery finds tests, and FESA emits no warning. | Zero test failures and new warnings | implementation-planning-agent; implementation-agent; build-test-executor-agent | approved |
| FESA-REQ-LS3DEB-035 | Numerical tests shall satisfy normalized `1e-12` for stiffness symmetry and two-point-Gauss/closed-form agreement, normalized `1e-10` for rigid-mode and linear-system residual, and relative `1e-9` for analytical solutions, while checking six rigid modes, rank 6, positive deformation energy, transformation orthogonality/energy invariance, prescribed-displacement recovery, and axial/torsion/two-plane bending benchmarks. | verification | Detect sign, integration, rank, and transform defects. | Approved design §§11.2, 11.3; formulation §18 | must | Unit, analytical, and orchestration tests | Every listed invariant and analytical case passes at its stated threshold. | `1e-12` matrix; `1e-10` residual; `1e-9` analytical | formulation-agent; numerical-review-agent; implementation-planning-agent | approved | | FESA-REQ-LS3DEB-035 | Numerical tests shall satisfy normalized `1e-12` for stiffness symmetry and two-point-Gauss/closed-form agreement, normalized `1e-10` for rigid-mode and linear-system residual, and relative `1e-9` for analytical solutions, while checking six rigid modes, rank 6, positive deformation energy, transformation orthogonality/energy invariance, prescribed-displacement recovery, and axial/torsion/two-plane bending benchmarks. | verification | Detect sign, integration, rank, and transform defects. | Approved design §§11.2, 11.3; formulation §18 | must | Unit, analytical, and orchestration tests | Every listed invariant and analytical case passes at its stated threshold. | `1e-12` matrix; `1e-10` residual; `1e-9` analytical | formulation-agent; numerical-review-agent; implementation-planning-agent | approved |
| FESA-REQ-LS3DEB-036 | The approved B33 reference comparison shall group rows by the same model, step/frame, quantity, and component, compute `reference_scale` only from read-only Abaqus values, and apply `absolute_floor + 1e-6 * reference_scale` to every matched row. | tolerance | Give zero and nonzero rows one deterministic rule. | Approved design §11.3; ADR-014 | must | Comparison unit/integration test; report review | Every group uses the Abaqus-only maximum absolute scale and every matched row uses the exact formula. | Relative coefficient `1e-6` | reference-model-agent; reference-verification-agent | approved | | FESA-REQ-LS3DEB-036 | The approved B33 reference comparison shall group rows into translation, rotation, reaction-force, reaction-moment, section-force and section-moment families and compute `S=max(abs(reference))` from read-only Abaqus values. | tolerance | Give zero-like components a dimensionally compatible reference scale. | Common tolerance design; ADR-022 | must | Comparison unit/integration test; report review | Every family uses the exact declared components and Abaqus-only maximum scale. | Family scale | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-037 | For the approved SI bundle, displacement and rotation shall use `absolute_floor=1e-9`, force and moment shall use `absolute_floor=1e-3`, and a zero component scale shall use the applicable absolute floor alone. | tolerance | Preserve dimensional meaning near zero. | Approved design §11.3; ADR-014 | must | Comparison tests with zero/near-zero rows | Each quantity uses its exact SI floor and zero-scale groups use no relative contribution. | SI `1e-9` displacement/rotation; `1e-3` force/moment | reference-model-agent; reference-verification-agent | approved | | FESA-REQ-LS3DEB-037 | Near-zero rows shall use the `0.01*S` fallback, other rows relative error `0.05`, and every family scale-relative RMS `0.01`; zero-scale families require exact-zero FESA values and no independent absolute gate is used. | tolerance | Stabilize zero-like rows while retaining row and aggregate checks. | Common tolerance design; ADR-022 | must | Boundary, zero-scale and RMS comparison tests | Every row and family uses the exact common constants and branch rules without NaN/Inf. | `0.01`, `0.05`, `0.01` | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-038 | Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation. | tolerance | Prevent false passes through omission or clamping. | Approved design §§11.3, 12; ADR-014 | must | Negative comparison tests | Every listed invalid case fails before numeric comparison and zero values remain unchanged. | No ignored invalid rows | reference-verification-agent | approved | | FESA-REQ-LS3DEB-038 | Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation. | tolerance | Prevent false passes through omission or clamping. | Approved design §§11.3, 12; ADR-014 | must | Negative comparison tests | Every listed invalid case fails before numeric comparison and zero values remain unchanged. | No ignored invalid rows | reference-verification-agent | approved |
| FESA-REQ-LS3DEB-039 | The verification report shall record every row decision and maximum absolute error, component-scale normalized error, RMS error, norm error, and worst row/component for each compared quantity. | tolerance | Make the pass/fail decision auditable. | Approved design §11.3; ADR-014 | must | Verification report schema/review | Per-row decisions and all required aggregate/worst metrics are present for every quantity. | Report completeness | reference-verification-agent; release-agent | approved | | FESA-REQ-LS3DEB-039 | The verification report shall record every row branch/decision, family identity/components, scale, near-zero band/count, maximum absolute error, scale-relative RMS, worst row/component and zero-scale diagnostic. | tolerance | Make the pass/fail decision auditable. | Common tolerance design; ADR-022 | must | Verification report schema/review | Per-row decisions and all required family metrics are present without nonfinite report values. | Report completeness | reference-verification-agent; release-agent | approved |
| FESA-REQ-LS3DEB-040 | The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle. | reference | Protect the approved correctness baseline. | Approved design §12; ADR-010, ADR-014 | must | Artifact inventory; Git diff review | Exact filenames exist and no reference file is added, removed, renamed, or content-modified. | Exact path/content identity | reference-model-agent; reference-verification-agent; release-agent | approved | | FESA-REQ-LS3DEB-040 | The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle. | reference | Protect the approved correctness baseline. | Approved design §12; ADR-010, ADR-014 | must | Artifact inventory; Git diff review | Exact filenames exist and no reference file is added, removed, renamed, or content-modified. | Exact path/content identity | reference-model-agent; reference-verification-agent; release-agent | approved |
| FESA-REQ-LS3DEB-041 | Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start. | reference | Detect stale B31 or malformed evidence. | Approved design §§6.3, 12 | must | Artifact-check integration test | All checks pass before comparison and every failure uses an approved classification. | Exact inventory and schema | reference-model-agent; reference-verification-agent | approved | | FESA-REQ-LS3DEB-041 | Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start. | reference | Detect stale B31 or malformed evidence. | Approved design §§6.3, 12 | must | Artifact-check integration test | All checks pass before comparison and every failure uses an approved classification. | Exact inventory and schema | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-042 | Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and node-station-normalized section resultant by `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; adjacent interior endpoints shall first agree within approved tolerance and shall not be averaged to hide a mismatch. | reference | Compare equivalent quantities despite legacy station rows. | Approved design §§8.2, 12 | must | Reference comparison test/report | All components match by source identity and interior endpoints pass before deterministic representative selection. | Requirements 036 and 037 policy | io-definition-agent; reference-model-agent; reference-verification-agent | approved | | FESA-REQ-LS3DEB-042 | Reference verification shall compare displacement and reaction by source-node identity and section resultants by direct `(instance, element label, endpoint node label, component)` identity using `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; endpoint rows shall not be collapsed or averaged. | reference | Preserve the source identity supplied by the element-endpoint CSV and HDF5. | Common tolerance design; ADR-022 | must | Reference comparison test/report | All 20 B33 endpoint rows map one-to-one and malformed endpoint identity fails before tolerance. | Requirements 036 and 037 policy | io-definition-agent; reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-043 | Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations. | governance | Keep numerical similarity distinct from physical/release approval. | Approved design §§11, 12, 13 | must | Gate evidence audit | Each downstream report cites the preceding pass and physics evidence covers all six checks before release review. | Relevant upstream tolerances | coordinator-agent; physics-evaluation-agent; release-agent | approved | | FESA-REQ-LS3DEB-043 | Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations. | governance | Keep numerical similarity distinct from physical/release approval. | Approved design §§11, 12, 13 | must | Gate evidence audit | Each downstream report cites the preceding pass and physics evidence covers all six checks before release review. | Relevant upstream tolerances | coordinator-agent; physics-evaluation-agent; release-agent | approved |
| FESA-REQ-LS3DEB-044 | FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior. | governance | Prevent baseline contamination and scope inflation. | Approved design §§2.2, 12; ADR-010 | must | Process audit; Git diff; release documentation review | No reference execution or artifact mutation occurs and every exclusion is recorded without a support claim. | N/A | coordinator-agent; reference-model-agent; release-agent | approved | | FESA-REQ-LS3DEB-044 | FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior. | governance | Prevent baseline contamination and scope inflation. | Approved design §§2.2, 12; ADR-010 | must | Process audit; Git diff; release documentation review | No reference execution or artifact mutation occurs and every exclusion is recorded without a support claim. | N/A | coordinator-agent; reference-model-agent; release-agent | approved |
@@ -1540,11 +1540,11 @@ or drilling-energy warning is part of this check.
### 17.5 Reference-comparison boundary ### 17.5 Reference-comparison boundary
Abaqus comparison uses only the declared full-integration S4 case and blocks only Abaqus comparison uses only the declared full-integration S4 case. Matched global
on matched global `U1/U2/U3` rows under the fixed absolute criterion `U1/U2/U3` families are blocking and `UR1/UR2/UR3` families are warning-only. Both use
`abs(fesa-reference) <= 1.0e-5`. `UR1/UR2/UR3` uses the same fixed absolute value but the common family-scale row rule (`S=max(abs(reference))`, near-zero ratio `0.01`, relative
an exceedance emits only a deterministic nonblocking warning. A reported reference tolerance `0.05`) and scale-relative RMS tolerance `0.01`. `S=0` requires exact-zero FESA
scale is diagnostic only and does not enter the MITC4 decision or normalization. values and there is no independent absolute-error gate.
FESA `S4` and `S4R` inputs must produce the same internal numerical rows for identical FESA `S4` and `S4R` inputs must produce the same internal numerical rows for identical
supported models while preserving distinct source metadata. This common-path property supported models while preserving distinct source metadata. This common-path property
is verified without consuming an S4R Abaqus artifact; Abaqus S4R is not an acceptance is verified without consuming an S4R Abaqus artifact; Abaqus S4R is not an acceptance
@@ -1631,7 +1631,7 @@ research brief remain the project source of truth.
| `024-029` | deterministic element buffers, partitioned linear lifecycle, full-residual reaction | planning | | `024-029` | deterministic element buffers, partitioned linear lifecycle, full-residual reaction | planning |
| `039-048` | nodal/global and shell/local recovery inventory, units, identities, physical shell energy | I/O schema | | `039-048` | nodal/global and shell/local recovery inventory, units, identities, physical shell energy | I/O schema |
| `049-057` | normalized invariants, patches, fixed drilling, declared S4 reference and equilibrium | Numerical Review/reference/physics | | `049-057` | normalized invariants, patches, fixed drilling, declared S4 reference and equilibrium | Numerical Review/reference/physics |
| `058-064` | fixed absolute `1.0e-5`; U blocking and UR warning-only | reference verification | | `058-064` | common family-scale row/RMS policy; U blocking and UR warning-only | reference verification |
| `065-072` | exact existing S4 paths, S4R reference non-consumption, immutability and displacement-only boundary | reference model | | `065-072` | exact existing S4 paths, S4R reference non-consumption, immutability and displacement-only boundary | reference model |
### 20.1 Numerical Review revision traceability ### 20.1 Numerical Review revision traceability
@@ -1685,7 +1685,8 @@ items or an expanded reference portfolio as missing evidence.
without claiming element equivalence. without claiming element equivalence.
- Do not consume `reference/shellR/` in acceptance comparison; preserve S4R support - Do not consume `reference/shellR/` in acceptance comparison; preserve S4R support
through source-mapping/common-kernel/metadata tests. through source-mapping/common-kernel/metadata tests.
- Use the fixed absolute MITC4 tolerance `1.0e-5`; do not alter the separate B33 tolerance or add administrative metadata or portfolio gates. - Use the project-wide common family-scale row/RMS tolerance; do not add an independent
absolute-error gate, administrative metadata or portfolio gates.
- Do not create, repair, rename, or run reference artifacts during this formulation - Do not create, repair, rename, or run reference artifacts during this formulation
gate. gate.
+26 -17
View File
@@ -36,8 +36,8 @@ The user approved the following I/O-specific decisions on `2026-08-12`:
retaining the existing common metadata, nodal-result, diagnostic, CLI, and retaining the existing common metadata, nodal-result, diagnostic, CLI, and
failure-atomicity conventions. failure-atomicity conventions.
The approved requirements fix the drilling rule and the MITC4 fixed absolute The approved requirements fix the drilling rule and the project-wide common
displacement tolerance `1.0e-5`. family-scale reference tolerance.
Drilling calibration/output, `NR-O03`, `NR-O04`, bundle-administration metadata, and Drilling calibration/output, `NR-O03`, `NR-O04`, bundle-administration metadata, and
an expanded reference portfolio are outside this contract. an expanded reference portfolio are outside this contract.
@@ -515,22 +515,31 @@ Before tolerance evaluation:
Only `U1/U2/U3` affect pass/fail. `UR1/UR2/UR3` are always compared and reported but Only `U1/U2/U3` affect pass/fail. `UR1/UR2/UR3` are always compared and reported but
can emit only an approved deterministic nonblocking warning. can emit only an approved deterministic nonblocking warning.
For every matched displacement row: Matched U rows form one blocking translation family and matched UR rows form one warning-only
rotation family. For each family:
```text ```text
row_tolerance = 1.0e-5 S = max(abs(reference_value_i))
row_pass = abs(fesa_value-reference_value) <= row_tolerance error_i = abs(fesa_value_i-reference_value_i)
near_zero_band = 0.01*S
if abs(reference_value_i) <= near_zero_band:
row_pass = error_i <= near_zero_band
else:
row_pass = error_i/abs(reference_value_i) <= 0.05
relative_rms = sqrt(mean(error_i^2))/S
family_pass = all(row_pass) and relative_rms <= 0.01
``` ```
The `1.0e-5` value is in the user-consistent length unit for U and dimensionless for Reference values alone define `S`; values are not zero-clamped or omitted and there is no
UR. No reference or result value is zero-clamped and neither component scale nor a independent absolute-error gate. A zero-scale family passes only if every FESA value is exactly
row-specific denominator changes the fixed value. A reference scale may remain in the zero, otherwise it fails without emitting NaN/Inf. U row or RMS exceedance fails; the same UR
report as diagnostic information only. U exceedance fails; UR exceedance emits a exceedance emits a deterministic warning only.
deterministic warning only. The separate B33 mixed tolerance is unchanged.
The report records every U/UR row, blocking/nonblocking decision, absolute error, The report records every U/UR row, blocking/nonblocking decision, family scale,
fixed-tolerance-normalized error, RMS error, displacement/rotation vector-norm near-zero branch, absolute and applicable row-relative error, scale-relative RMS,
error, worst source row/component, and every UR warning. worst source row/component, and every UR warning.
## 8. CLI and Diagnostics Contract ## 8. CLI and Diagnostics Contract
@@ -615,7 +624,7 @@ physics review, or release status follows from approval of this document alone.
| `031-038` | source-independent MITC4 identity, fixed drilling stabilization and full-integration policy; no drilling output | Implementation Planning tests | | `031-038` | source-independent MITC4 identity, fixed drilling stabilization and full-integration policy; no drilling output | Implementation Planning tests |
| `039-048` | additive HDF5 v0 paths, mandatory quantities, location identity, atomic output | Reference Model and Implementation Planning | | `039-048` | additive HDF5 v0 paths, mandatory quantities, location identity, atomic output | Reference Model and Implementation Planning |
| `049-057` | diagnostic/schema hooks and required verification-metric/physical-energy evidence | Numerical Review and planning | | `049-057` | diagnostic/schema hooks and required verification-metric/physical-energy evidence | Numerical Review and planning |
| `058-064` | normalized U/UR rows, fixed absolute `1.0e-5`, blocking/warning behavior, report inventory | Reference Verification | | `058-064` | common-policy U/UR families, blocking/warning behavior, report inventory | Reference Verification |
| `065-072` | exact current S4 paths, S4R reference non-consumption, immutability and displacement-only gate | Reference Model | | `065-072` | exact current S4 paths, S4R reference non-consumption, immutability and displacement-only gate | Reference Model |
## 11. Open Issues and Downstream Handoff ## 11. Open Issues and Downstream Handoff
@@ -623,8 +632,8 @@ physics review, or release status follows from approval of this document alone.
### 11.1 Numerical Review boundary ### 11.1 Numerical Review boundary
No I/O-owned calibration value remains open. Numerical Review shall verify the exact No I/O-owned calibration value remains open. Numerical Review shall verify the exact
fixed drilling rule, basic geometry predicates, required HDF5 inventory, and fixed fixed drilling rule, basic geometry predicates, required HDF5 inventory, and common
absolute MITC4 tolerance mapping. Drilling calibration/energy output, `NR-O03`, `NR-O04`, bundle family-scale row/RMS tolerance mapping. Drilling calibration/energy output, `NR-O03`, `NR-O04`, bundle
administration and reference-portfolio expansion are removed scope. administration and reference-portfolio expansion are removed scope.
### 11.2 Reference Model Agent ### 11.2 Reference Model Agent
@@ -632,7 +641,7 @@ administration and reference-portfolio expansion are removed scope.
- Write `docs/linear-static-mitc4-shell/reference-model.md` using - Write `docs/linear-static-mitc4-shell/reference-model.md` using
this exact keyword/HDF5/reference-row contract. this exact keyword/HDF5/reference-row contract.
- Record only the two exact existing input/displacement pairs, comparison components, - Record only the two exact existing input/displacement pairs, comparison components,
HDF5 projection, source-row identity, fixed absolute MITC4 tolerance and immutability rule. HDF5 projection, source-row identity, common family-scale tolerance and immutability rule.
- Treat reaction/stress artifacts as nonblocking review evidence and do not create - Treat reaction/stress artifacts as nonblocking review evidence and do not create
location-equivalence claims absent from this contract. location-equivalence claims absent from this contract.
@@ -75,7 +75,7 @@ confirmed mathematical defect는 없다.
| `NR-O02` drilling-energy ratio | `removed from scope` | Drilling energy는 내부 quadratic identity일 뿐 physical energy나 mandatory output이 아니며 ratio/warning threshold도 요구하지 않는다. | | `NR-O02` drilling-energy ratio | `removed from scope` | Drilling energy는 내부 quadratic identity일 뿐 physical energy나 mandatory output이 아니며 ratio/warning threshold도 요구하지 않는다. |
| `NR-O03` smooth-director calibration | `removed from scope` | Pairwise positive incident-normal orientation, finite/nonzero averaging 및 duplicate-node fold modeling이 승인된 exact predicate다. 별도 angle calibration은 gate가 아니다. | | `NR-O03` smooth-director calibration | `removed from scope` | Pairwise positive incident-normal orientation, finite/nonzero averaging 및 duplicate-node fold modeling이 승인된 exact predicate다. 별도 angle calibration은 gate가 아니다. |
| `NR-O04` distortion/warp calibration | `removed from scope` | Basic topology, finite/nonzero surface measure 및 required-point `J>0`가 승인된 predicate다. Quality sweep이나 cutoff는 gate가 아니다. | | `NR-O04` distortion/warp calibration | `removed from scope` | Basic topology, finite/nonzero surface measure 및 required-point `J>0`가 승인된 predicate다. Quality sweep이나 cutoff는 gate가 아니다. |
| `NR-O05` U/UR tolerance | `resolved` | 모든 관련 문서가 sole S4 reference에 고정 절대오차 `1.0e-5`, U blocking, UR warning-only를 동일하게 정의한다. Reference scale은 판정에 사용하지 않으며 S4R은 reference gate가 아닌 common-path evidence다. | | `NR-O05` U/UR tolerance | `resolved` | 모든 관련 문서가 sole S4 reference에 공통 family-scale row/RMS 규칙, U blocking, UR warning-only를 동일하게 정의한다. S4R은 reference gate가 아닌 common-path evidence다. |
이전의 `needs-reference-model` 판정에 포함됐던 canonical naming, README, 이전의 `needs-reference-model` 판정에 포함됐던 canonical naming, README,
`metadata.json`, provenance, expanded portfolio 및 아직 없는 comparison result는 현재 `metadata.json`, provenance, expanded portfolio 및 아직 없는 comparison result는 현재
@@ -223,9 +223,10 @@ confirmed mathematical defect는 없다.
not emitted, and `S13/S23` point stress is not synthesized. Different natural or not emitted, and `S13/S23` point stress is not synthesized. Different natural or
section locations are never averaged. section locations are never averaged.
- Reference comparison first rejects missing, extra, duplicate, nonfinite or - Reference comparison first rejects missing, extra, duplicate, nonfinite or
identity-mismatched rows. Every U/UR row then uses fixed absolute tolerance identity-mismatched rows. Every U/UR family then uses the common reference-only scale,
`1.0e-5`; no reference-scale decision term, zero clamp or row denominator is near-zero ratio `0.01`, row relative tolerance `0.05` and scale-relative RMS tolerance
introduced. U1/U2/U3 is blocking and UR1/UR2/UR3 is warning-only. `0.01`. `S=0` requires exact-zero FESA values; there is no zero clamp or independent
absolute-error gate. U1/U2/U3 is blocking and UR1/UR2/UR3 is warning-only.
- Source S4 and S4R select the same FESA MITC4 kernel/quadrature/recovery path while - Source S4 and S4R select the same FESA MITC4 kernel/quadrature/recovery path while
preserving source type. This is an input mapping, not an Abaqus formulation, preserving source type. This is an input mapping, not an Abaqus formulation,
integration, stabilization or recovery equivalence claim. integration, stabilization or recovery equivalence claim.
@@ -11,10 +11,39 @@
`docs/linear-static-mitc4-shell/reference-model.md` `docs/linear-static-mitc4-shell/reference-model.md`
- source_io_definition: `docs/linear-static-mitc4-shell/io.md` - source_io_definition: `docs/linear-static-mitc4-shell/io.md`
- source_requirements: `docs/linear-static-mitc4-shell/requirements.md` - source_requirements: `docs/linear-static-mitc4-shell/requirements.md`
- status: `pass-for-physics-evaluation` - historical_status: `pass-for-physics-evaluation`
- current_status: `pass-for-physics-evaluation`
- superseded_on: `2026-08-18`
- revalidated_on: `2026-08-18`
- owner_agent: `reference-verification-agent` - owner_agent: `reference-verification-agent`
- date: `2026-08-13` - date: `2026-08-13`
The fixed-absolute-tolerance evidence below is preserved as a historical record. ADR-022
supersedes that decision with the common family-scale row/RMS policy. The following revalidation
is the current reference-gate evidence.
## 2026-08-18 Common-policy Revalidation
`cmake --build .harness/build --config Debug` passed, followed by
`ctest --test-dir .harness/build -C Debug --output-on-failure`: 214/214 tests passed.
The S4 reference test generated
`.harness/build/reference/mitc4-shell-s4-comparison/comparison.json` from the declared
read-only input/CSV pair.
- exact canonical rows: 294
- blocking translation family: 147 rows, all row/RMS gates passed
- warning-only rotation family: 147 rows, all row/RMS gates passed; warnings 0
- identity/schema/nonfinite failures: 0
- overall verdict: `passed=true`
| family | rows | scale | near-zero rows | max absolute error | scale-relative RMS |
| --- | ---: | ---: | ---: | ---: | ---: |
| translation `U1/U2/U3` | 147 | `2.37408203e-5` | 122 | `1.903785349151439e-7` | `1.0332756650397166e-3` |
| rotation `UR1/UR2/UR3` | 147 | `7.60725743e-6` | 107 | `6.882854962740428e-8` | `2.9771372973060382e-3` |
All values above come from the generated JSON ledger. The historical report body starts below
and must not be read as the current identity/tolerance evidence.
The prerequisite build/test report has status `pass-for-reference-verification`. The prerequisite build/test report has status `pass-for-reference-verification`.
This report verifies only the approved full-integration S4 displacement case. It This report verifies only the approved full-integration S4 displacement case. It
does not compare S4R artifacts, claim Abaqus formulation equivalence, approve does not compare S4R artifacts, claim Abaqus formulation equivalence, approve
@@ -8,6 +8,7 @@
- status: `approved-for-implementation-planning` - status: `approved-for-implementation-planning`
- owner_agent: `reference-model-agent` - owner_agent: `reference-model-agent`
- date: `2026-08-13` - date: `2026-08-13`
- comparison_revalidated_on: `2026-08-18`
- artifact_policy: `read-only-existing-files` - artifact_policy: `read-only-existing-files`
- authoritative_fesa_output: `results.h5` - authoritative_fesa_output: `results.h5`
@@ -80,26 +81,31 @@ zero-clamped and mismatched rows are not omitted, averaged, or synthesized.
## 5. Tolerance and decision rule ## 5. Tolerance and decision rule
For every matched row: Matched `U1/U2/U3` rows form one blocking translation family and matched
`UR1/UR2/UR3` rows form one warning-only rotation family. For each family:
```text ```text
tolerance = 1.0e-5 S = max(abs(abaqus_value_i))
absolute_error_i = abs(fesa_value_i - abaqus_value_i) absolute_error_i = abs(fesa_value_i-abaqus_value_i)
near_zero_band = 0.01*S
near-zero row: absolute_error_i <= near_zero_band
other row: absolute_error_i/abs(abaqus_value_i) <= 0.05
relative_rms = sqrt(mean(absolute_error_i^2))/S <= 0.01
``` ```
The fixed `1.0e-5` value is expressed in the model's user-consistent length unit for The scale uses read-only Abaqus values only. No independent absolute-error gate, zero clamp or
`U1/U2/U3` and is dimensionless for `UR1/UR2/UR3`. Neither a component reference scale row omission is permitted. A zero-scale family passes only when all FESA values are exactly zero;
nor a row-specific denominator changes the value. Reference scale may be reported as otherwise it reports `zero-reference-scale-nonzero-error` without NaN/Inf. The U family controls
non-decision diagnostic information. The separate B33 mixed tolerance is unchanged. the verdict and the UR family uses the same numeric rule for deterministic warning-only evidence.
- Every matched `U1/U2/U3` row must satisfy `absolute_error_i <= tolerance`. - Every matched `U1/U2/U3` row and the translation family RMS must satisfy the common
Any U exceedance fails that case and the feature reference comparison. row/RMS rule. Any U row or family-RMS exceedance fails the feature reference comparison.
- `UR1/UR2/UR3` uses the same fixed value. Every exceedance produces a deterministic - `UR1/UR2/UR3` uses the same family-scale row/RMS rule. Every row or family-RMS
warning containing the case, source row, component, error, and tolerance, but does exceedance produces deterministic warning evidence but does not change pass/fail.
not change pass/fail.
The comparison report records every U/UR row decision, maximum absolute error, The comparison report records every U/UR row decision, family scale, near-zero branch,
fixed-tolerance-normalized error, RMS error, vector-norm error, worst source maximum absolute error, row relative error, scale-relative RMS, worst source
row/component, and every UR warning. row/component, and every UR warning.
## 6. Coverage and handoff ## 6. Coverage and handoff
+7 -1
View File
@@ -18,11 +18,17 @@
- source_physics_evaluation_report: `docs/linear-static-mitc4-shell/physics-evaluation.md` - source_physics_evaluation_report: `docs/linear-static-mitc4-shell/physics-evaluation.md`
- audited_source_head: `820ba30c717b3d0e113775608e20dfd5fbc05d53` - audited_source_head: `820ba30c717b3d0e113775608e20dfd5fbc05d53`
- audited_branch: `feat-linear-static-mitc4-shell` - audited_branch: `feat-linear-static-mitc4-shell`
- status: `ready-for-release` - historical_status: `ready-for-release`
- current_status: `superseded-pending-revalidation`
- superseded_on: `2026-08-18`
- owner_agent: `release-agent` - owner_agent: `release-agent`
- date: `2026-08-13` - date: `2026-08-13`
- release_boundary: internal FESA feature release readiness only; no publish, deploy, package, tag, commit, or external release was performed - release_boundary: internal FESA feature release readiness only; no publish, deploy, package, tag, commit, or external release was performed
The readiness evidence below is historical. The 2026-08-18 common-policy reference revalidation
passed, but fresh physics and release audits are still required before a current
`ready-for-release` verdict may be issued.
## Release Scope ## Release Scope
| item | included | excluded | notes | | item | included | excluded | notes |
@@ -184,13 +184,13 @@ Formulation, Numerical Review, I/O, Reference Model, Implementation Planning 및
## Tolerance Policy ## Tolerance Policy
- **FESA-REQ-LSMITC4-058** — Abaqus reference pass/fail shall apply only to matched global `U1/U2/U3` rows using the fixed absolute tolerance `abs(fesa-reference) <= 1.0e-5` for every row. - **FESA-REQ-LSMITC4-058** — Abaqus reference pass/fail shall apply only to matched global `U1/U2/U3` rows grouped as one blocking translation family with Abaqus-only scale `S=max(abs(reference))`.
- **FESA-REQ-LSMITC4-059**MITC4 row tolerance and tolerance-normalized error shall not depend on `reference_scale`, a row-specific denominator, zero clamp or component magnitude; a reference scale may be reported only as non-decision diagnostic information. - **FESA-REQ-LSMITC4-059**A U row with `abs(reference)<=0.01*S` shall pass when `abs(fesa-reference)<=0.01*S`; every other U row shall pass when its relative error is at most `0.05`; the U family shall also satisfy `RMS(error)/S<=0.01`.
- **FESA-REQ-LSMITC4-060**The fixed `1.0e-5` U tolerance is expressed in the model's user-consistent length unit and is independent of the approved B33 component-scale mixed tolerance. - **FESA-REQ-LSMITC4-060**MITC4 shall use the project-wide family-scale policy without an independent absolute-error gate, zero clamp or row omission; a zero-scale family passes only when every FESA value is exactly zero.
- **FESA-REQ-LSMITC4-061** — Global `UR1/UR2/UR3` rows shall use the same fixed absolute value `1.0e-5` and shall be fully reported; an exceedance emits a deterministic nonblocking warning and never changes pass/fail. - **FESA-REQ-LSMITC4-061** — Global `UR1/UR2/UR3` rows shall form one warning-only rotation family using the same near-zero, relative-row and scale-relative RMS constants; an exceedance emits a deterministic nonblocking warning and never changes pass/fail.
- **FESA-REQ-LSMITC4-062** The fixed `1.0e-5` UR tolerance is dimensionless. No separate UR large-error or drilling-energy threshold is required. - **FESA-REQ-LSMITC4-062** — No separate UR large-error, fixed absolute or drilling-energy threshold is required; every row decision and family metric shall remain finite and auditable.
- **FESA-REQ-LSMITC4-063** — Missing, extra, duplicate, nonfinite, schema-mismatched or source-identity-mismatched rows shall fail artifact/schema validation before numeric tolerance evaluation for both U and UR inventories. - **FESA-REQ-LSMITC4-063** — Missing, extra, duplicate, nonfinite, schema-mismatched or source-identity-mismatched rows shall fail artifact/schema validation before numeric tolerance evaluation for both U and UR inventories.
- **FESA-REQ-LSMITC4-064** — The comparison report shall record each U/UR row decision, maximum absolute error, fixed-tolerance-normalized error, RMS error, vector-norm error and worst source row/component; nonblocking UR warnings shall not be omitted from an otherwise passing report. - **FESA-REQ-LSMITC4-064** — The comparison report shall record each U/UR row branch/decision, family identity/components, scale, near-zero band/count, maximum absolute error, scale-relative RMS, vector-norm diagnostic, worst source row/component and zero-scale diagnostic; nonblocking UR warnings shall not be omitted from an otherwise passing report.
## Reference Artifact Requirements ## Reference Artifact Requirements
@@ -226,8 +226,8 @@ without gaps or overlap.
| `031-038` | 5-DOF physics embedded in 6-DOF with fixed drilling stabilization | numerical boundary | User approval; MITC literature and thesis 6-DOF discussion | must | Formulation review, invariant and rank tests | Exact `10^-3` positive rotational-diagonal rule; physical outputs exclude drilling | Fixed by Requirements 033-036 | Formulation; Numerical Review; Implementation Planning | approved | | `031-038` | 5-DOF physics embedded in 6-DOF with fixed drilling stabilization | numerical boundary | User approval; MITC literature and thesis 6-DOF discussion | must | Formulation review, invariant and rank tests | Exact `10^-3` positive rotational-diagonal rule; physical outputs exclude drilling | Fixed by Requirements 033-036 | Formulation; Numerical Review; Implementation Planning | approved |
| `039-048` | Mandatory HDF5 output and failure atomicity | output | User approval; ADR-005/016/018 | must | Recovery, schema, identity, nonfinite and atomicity tests | Every quantity/location/unit/identity exists; failure commits no partial success | Exact component/location inventory; I/O Definition owns schema | Formulation; I/O Definition; Implementation Planning | approved | | `039-048` | Mandatory HDF5 output and failure atomicity | output | User approval; ADR-005/016/018 | must | Recovery, schema, identity, nonfinite and atomicity tests | Every quantity/location/unit/identity exists; failure commits no partial success | Exact component/location inventory; I/O Definition owns schema | Formulation; I/O Definition; Implementation Planning | approved |
| `049-057` | TDD, invariants, patch, declared reference and physics | verification | User approval; shell formulation evidence; project process | must | CTest evidence, analytical/patch tests, the S4 reference case and physics review | Required tests pass; removed calibration/portfolio checks are not reintroduced | `1e-12` symmetry/frame; `1e-10` rigid/residual | Numerical Review; Implementation Planning | approved | | `049-057` | TDD, invariants, patch, declared reference and physics | verification | User approval; shell formulation evidence; project process | must | CTest evidence, analytical/patch tests, the S4 reference case and physics review | Required tests pass; removed calibration/portfolio checks are not reintroduced | `1e-12` symmetry/frame; `1e-10` rigid/residual | Numerical Review; Implementation Planning | approved |
| `058-060` | Translational displacement pass/fail tolerance | tolerance | User approval; ADR-020 | must | Comparator unit/integration tests and report review | Every matched U row uses fixed absolute `1.0e-5` without scale, clamp or omission | Fixed by Requirements 058-060 | Reference Verification | approved | | `058-060` | Translational displacement pass/fail tolerance | tolerance | User approval; ADR-022 | must | Comparator unit/integration tests and report review | Every matched U row and U-family RMS use the common family-scale policy without clamp or omission | `0.01`, `0.05`, `0.01` | Reference Verification | approved |
| `061-062` | Rotational warning-only comparison | tolerance/warning | User approval; ADR-020 | must | Comparator/diagnostic tests and report review | UR never changes pass/fail; fixed absolute `1.0e-5` exceedance emits a deterministic warning | Fixed by Requirements 061-062 | Reference Verification | approved | | `061-062` | Rotational warning-only comparison | tolerance/warning | User approval; ADR-022 | must | Comparator/diagnostic tests and report review | UR never changes pass/fail; common-policy exceedance emits a deterministic warning | `0.01`, `0.05`, `0.01` | Reference Verification | approved |
| `063-064` | Row/schema failure and report completeness | reference verification | User approval; ADR-005/014/018 | must | Negative comparator and report-schema tests | Invalid inventory fails before numeric comparison; all U/UR metrics remain visible | No ignored invalid rows | I/O Definition; Reference Verification | approved | | `063-064` | Row/schema failure and report completeness | reference verification | User approval; ADR-005/014/018 | must | Negative comparator and report-schema tests | Invalid inventory fails before numeric comparison; all U/UR metrics remain visible | No ignored invalid rows | I/O Definition; Reference Verification | approved |
| `065-068` | Exact S4 reference-case inventory, S4R exclusion and row validity | reference | User declaration; ADR-019 | must | Read-only S4 inventory, source-row/component precheck and S4R non-consumption test | Two declared S4 paths exist; required rows are unique, finite and deterministically mapped; S4R artifacts are not required or consumed | Requirements `058-063` | Reference Model; Reference Verification | approved | | `065-068` | Exact S4 reference-case inventory, S4R exclusion and row validity | reference | User declaration; ADR-019 | must | Read-only S4 inventory, source-row/component precheck and S4R non-consumption test | Two declared S4 paths exist; required rows are unique, finite and deterministically mapped; S4R artifacts are not required or consumed | Requirements `058-063` | Reference Model; Reference Verification | approved |
| `069-071` | S4 reference coverage and displacement-only comparison | reference | User approval | must | HDF5-to-CSV comparison | Declared S4 case only; U blocks and UR only warns; S4R mapping remains independently tested | Requirements `058-064` | Reference Verification; Physics Evaluation | approved | | `069-071` | S4 reference coverage and displacement-only comparison | reference | User approval | must | HDF5-to-CSV comparison | Declared S4 case only; U blocks and UR only warns; S4R mapping remains independently tested | Requirements `058-064` | Reference Verification; Physics Evaluation | approved |
@@ -270,7 +270,7 @@ and tangent derivation may remain in the formulation document.
- Record only the exact existing S4 input/displacement CSV paths from Requirement 065 as acceptance artifacts and keep every existing reference artifact read-only. - Record only the exact existing S4 input/displacement CSV paths from Requirement 065 as acceptance artifacts and keep every existing reference artifact read-only.
- Do not consume the S4R bundle in reference verification; route S4R source support to parser/common-kernel/HDF5 tests from Requirement 066. - Do not consume the S4R bundle in reference verification; route S4R source support to parser/common-kernel/HDF5 tests from Requirement 066.
- Define only the HDF5-to-CSV source-node/component projection and the approved fixed absolute MITC4 tolerance `1.0e-5`; do not add bundle administration or portfolio gates. - Define only the HDF5-to-CSV source-node/component projection and the approved common family-scale tolerance; do not add bundle administration or portfolio gates.
### Implementation Planning Agent ### Implementation Planning Agent
@@ -7,11 +7,15 @@
- status: `approved` - status: `approved`
- approved_by: user - approved_by: user
- approved_on: `2026-08-08` - approved_on: `2026-08-08`
- amended_on: `2026-08-09` - amended_on: `2026-08-18`
- source_formulation: `docs/linear-static-3d-euler-beam/formulation.md` - source_formulation: `docs/linear-static-3d-euler-beam/formulation.md`
- reference_baseline: `reference/cantilever beam/` from source commit `2b34d0b` - reference_baseline: exact read-only artifacts under `reference/cantilever beam/`
- implementation_environment: C++17, MSVC, CMake, CTest, GoogleTest, Intel oneMKL, Intel oneTBB, HDF5 - implementation_environment: C++17, MSVC, CMake, CTest, GoogleTest, Intel oneMKL, Intel oneTBB, HDF5
2026-08-18 amendment: the former node-station projection and mixed absolute-floor tolerance are
superseded by the direct element-endpoint identity and common family-scale row/RMS policy in
`2026-08-17-common-reference-tolerance-design.md` and ADR-022.
## 1. 목적 ## 1. 목적
이 설계는 Abaqus `.inp` keyword subset을 읽어 2절점 3차원 이 설계는 Abaqus `.inp` keyword subset을 읽어 2절점 3차원
@@ -288,8 +292,8 @@ node label 또는 node set이며 assembly-level set의 `INSTANCE` parameter를
Abaqus B31은 transverse shear deformation을 포함하는 Timoshenko beam이고 B33은 Abaqus B31은 transverse shear deformation을 포함하는 Timoshenko beam이고 B33은
2절점 cubic EulerBernoulli beam이다. FESA V0는 `TYPE=B33`만 Euler 요소로 매핑한다. 2절점 cubic EulerBernoulli beam이다. FESA V0는 `TYPE=B33`만 Euler 요소로 매핑한다.
`TYPE=B31``unsupported-element-formulation` 오류로 거부한다. `TYPE=B31``unsupported-element-formulation` 오류로 거부한다.
`reference/cantilever beam/cantilever beam.inp`source commit `2b34d0b`에서 `reference/cantilever beam/cantilever beam.inp``TYPE=B33`으로 생성된 승인 reference
`TYPE=B33`으로 생성된 승인 reference input이다. Reference artifact check는 비교 전에 input이다. Reference artifact check는 비교 전에
이 element type을 다시 확인하며 B31로 되돌아간 입력이나 결과를 허용하지 않는다. 이 element type을 다시 확인하며 B31로 되돌아간 입력이나 결과를 허용하지 않는다.
이 결정은 [[Abaqus Structural Element Families]], [[Beam and Frame Finite Elements]], 이 결정은 [[Abaqus Structural Element Families]], [[Beam and Frame Finite Elements]],
@@ -370,19 +374,20 @@ Abaqus internal-force CSV는 equilibrium end action이 아니라 section resulta
| `SM1` | `MY` | | `SM1` | `MY` |
| `SM2` | `MZ` | | `SM2` | `MZ` |
승인된 B33 reference의 `cantilever beam elemental forces.csv`element label 없이 승인된 B33 reference의 `cantilever beam elemental forces.csv``Frame`,
`Frame`, `Part Instance Name`, `Node Label`, `SF1`, `SM1`, `SM2`, `SM3` 기록한다. `Part Instance Name`, `Element Label`, `Node Label`, `SF1`, `SM1`, `SM2`, `SM3`
기록한다.
따라서 이 파일의 comparison은 다음 규칙을 사용한다. 따라서 이 파일의 comparison은 다음 규칙을 사용한다.
- Reference row key는 `(Step-1, frame 0, instance name, source node label, component)`다. - Reference row key는 `(Step-1, frame 0, instance name, source element label,
endpoint source node label, component)`다.
단일 step invariant에 따라 CSV의 `Increment 1: Step Time = 1.000` 단일 step invariant에 따라 CSV의 `Increment 1: Step Time = 1.000`
`(Step-1, frame 0)`으로 정규화한다. `(Step-1, frame 0)`으로 정규화한다.
- FESA endpoint section resultant를 동일한 section-cut 부호로 변환한 뒤 source node - 각 B33 element에는 input connectivity의 두 source node label과 일치하는 endpoint row가
station으로 정규화한다. Interior node의 두 인접 element endpoint 값은 먼저 승인된 정확히 두 개 있어야 하며, 각 row를 해당 HDF5 element endpoint에 직접 대응한다.
tolerance 안에서 서로 일치해야 한다. 불일치는 평균으로 숨기지 않고 Missing, extra, duplicate 또는 connectivity mismatch는 tolerance 전에 실패한다.
`tolerance-failure`로 보고한다. - Comparator는 같은 source node의 인접 endpoint를 node station으로 축약하거나 대표값을
- Interior endpoint가 일치하면 stable internal element ID가 작은 endpoint를 대표 row로 선택하거나 평균하지 않는다.
선택한다. Boundary node는 하나의 endpoint를 사용한다.
- 이 bundle에서 비교하는 mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, - 이 bundle에서 비교하는 mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`,
`SM3 -> T`다. CSV에 없는 `SF2``SF3`는 reference comparison 대상이 아니며, `SM3 -> T`다. CSV에 없는 `SF2``SF3`는 reference comparison 대상이 아니며,
FESA equilibrium end action의 transverse force는 unit/analytical test와 physics sanity로 FESA equilibrium end action의 transverse force는 unit/analytical test와 physics sanity로
@@ -496,8 +501,8 @@ GoogleTest target에는 FESA warning policy를 강제하지 않는다.
- axial, torsion, y/z bending cantilever analytical cases - axial, torsion, y/z bending cantilever analytical cases
- HDF5 schema, identity, component, metadata, atomic finalization - HDF5 schema, identity, component, metadata, atomic finalization
- CLI `.inp -> results.h5` integration - CLI `.inp -> results.h5` integration
- 승인된 B33 CSV header/Frame 정규화, node-station matching, interior endpoint 일치 검사 - 승인된 B33 CSV header/Frame 정규화와 element-endpoint direct identity 검사
- zero 및 near-zero reference row를 포함한 component-scale 혼합 허용오차 검사 - zero 및 near-zero reference row를 포함한 공통 family-scale 상대오차/RMS 검사
### 11.3 수치 tolerance ### 11.3 수치 tolerance
@@ -506,37 +511,33 @@ GoogleTest target에는 FESA warning policy를 강제하지 않는다.
| matrix symmetry and Gauss/closed-form comparison | normalized `1e-12` | | matrix symmetry and Gauss/closed-form comparison | normalized `1e-12` |
| rigid-mode and linear-system residual | normalized `1e-10` | | rigid-mode and linear-system residual | normalized `1e-10` |
| analytical solution tests | relative `1e-9` | | analytical solution tests | relative `1e-9` |
| Abaqus B33 reference comparison | component-scale relative `1e-6` | | Abaqus reference row relative error | `0.05` |
| SI displacement and rotation absolute floor | `1e-9` | | near-zero threshold ratio | reference family scale의 `0.01` |
| SI force and moment absolute floor | `1e-3` | | reference family scale-relative RMS | `0.01` |
Reference row 판정은 zero-reference에서도 의미가 있도록 component-scale 혼합 Reference row 판정은 같은 model, step/frame, quantity, component로 구성한 reference
허용오차를 사용한다. 같은 model, step/frame, quantity, component의 Abaqus reference family별 공통 규칙을 사용한다. Reference 값으로만 family scale을 계산한다.
rows에 대해 다음 값을 계산한다.
```text ```text
reference_scale = max(abs(reference_value_i)) S = max(abs(reference_value_i))
row_tolerance = absolute_floor + 1e-6 * reference_scale near_zero_i = abs(reference_value_i) <= 0.01 * S
row_pass = abs(fesa_value_i - reference_value_i) <= row_tolerance row_pass_i = abs(fesa_value_i - reference_value_i) / abs(reference_value_i) <= 0.05
if not near_zero_i
row_pass_i = abs(fesa_value_i - reference_value_i) <= 0.01 * S
if near_zero_i
relative_rms = sqrt(mean((fesa_value_i - reference_value_i)^2)) / S
family_pass = all(row_pass_i) and relative_rms <= 0.01
``` ```
- Scale은 displacement, rotation, force, moment 및 각 component를 섞지 않는다. - Family는 displacement, rotation, force, moment 및 각 component를 섞지 않는다.
- Scale은 read-only Abaqus reference 값만 사용하며 FESA 결과로 조정하지 않는다. - Scale은 read-only Abaqus reference 값만 사용하며 FESA 결과로 조정하지 않는다.
- Scale이 0이면 absolute floor만 적용한다. - `S == 0`이면 모든 FESA 값이 정확히 0일 때만 통과한다. 하나라도 0이 아니면
`zero-reference-scale-nonzero-error`로 실패한다.
- 모든 row를 개별 판정하며 reference 값을 zero-clamp하거나 row를 제거하지 않는다. - 모든 row를 개별 판정하며 reference 값을 zero-clamp하거나 row를 제거하지 않는다.
- Missing/extra row, nonfinite value, schema 또는 identity mismatch는 tolerance 계산 전에 - Missing/extra row, nonfinite value, schema 또는 identity mismatch는 tolerance 계산 전에
실패한다. 실패한다.
- Report는 max absolute error, component-scale normalized error, RMS error, norm error, - Report는 family scale, max absolute error, max row relative error, scale-relative RMS,
worst row/component를 기록한다. worst row/component와 판정 reason을 기록한다. 독립적인 absolute-error gate는 없다.
현재 B33 `SM1`의 reference scale은 `1.0e7`이고 moment row tolerance는
`1.0e-3 + 1.0e-6 * 1.0e7 = 10.001`이다. 자유단 reference residue `-1.56e-2`
FESA의 이론적 zero 사이 오차 `1.56e-2`는 이 기준을 통과하지만, 예를 들어 `100`
자유단 moment 오차는 실패한다.
승인된 cantilever bundle은 SI absolute floor를 사용한다. 이후 SI가 아닌 reference를
추가하면 quantity별 absolute floor를 해당 단위로 변환하고 변환 근거를 metadata와
verification report에 기록해야 한다.
### 11.4 공통 build/test command ### 11.4 공통 build/test command
@@ -561,9 +562,10 @@ CTest discovery 결과는 한 개 이상의 test를 포함해야 한다.
## 12. Reference 및 release gate ## 12. Reference 및 release gate
`reference/cantilever beam/`을 V0의 승인된 Abaqus B33 reference baseline으로 사용한다. `reference/cantilever beam/`을 V0의 승인된 Abaqus B33 reference baseline으로 사용한다.
논리 model ID는 `cantilever-beam-b33`, reference schema ID는 논리 model ID는 `cantilever-beam-b33`이다. 기존 `abaqus-cae-report-csv-v0`과 source
`abaqus-cae-report-csv-v0`이며 source baseline은 commit `2b34d0b`이다. Input header가 commit `2b34d0b` 기록은 2026-08-09 당시 inventory의 역사적 provenance이며, 2026-08-18
기록한 generator는 `Abaqus/CAE Learning Edition 2024`다. 모델은 SI 단위계, global regenerated elemental-force CSV의 현재 readiness gate가 아니다. Input header가 기록한
generator는 `Abaqus/CAE Learning Edition 2024`다. 모델은 SI 단위계, global
Cartesian nodal output, beam local section-force output, `Step-1`, increment 1, Cartesian nodal output, beam local section-force output, `Step-1`, increment 1,
step time 1.0 계약으로 해석한다. step time 1.0 계약으로 해석한다.
@@ -588,8 +590,8 @@ Reference verification은 build/test gate 통과 후 승인 input으로 FESA `re
- nodal displacement: CSV `U1/U2/U3/UR1/UR2/UR3`과 HDF5 displacement - nodal displacement: CSV `U1/U2/U3/UR1/UR2/UR3`과 HDF5 displacement
- nodal reaction: CSV `RF1/RF2/RF3/RM1/RM2/RM3`과 HDF5 reaction - nodal reaction: CSV `RF1/RF2/RF3/RM1/RM2/RM3`과 HDF5 reaction
- section resultant: CSV `SF1/SM1/SM2/SM3`node station으로 정규화한 HDF5 - section resultant: CSV `SF1/SM1/SM2/SM3`element label 및 endpoint node label로
section resultant 직접 대응한 HDF5 section resultant
- stress: 명시적 N/A - stress: 명시적 N/A
Artifact check는 네 파일의 존재, B33 element type, expected header, 유일한 row key, Artifact check는 네 파일의 존재, B33 element type, expected header, 유일한 row key,
@@ -3,11 +3,16 @@
## Status ## Status
- date: `2026-08-12` - date: `2026-08-12`
- amended_on: `2026-08-18`
- status: `approved` - status: `approved`
- scope: FESA project policy, agents, skills, and `linear-static-mitc4-shell` upstream contracts - scope: FESA project policy, agents, skills, and `linear-static-mitc4-shell` upstream contracts
- implementation_code: out of scope - implementation_code: out of scope
- reference_artifact_mutation: prohibited - reference_artifact_mutation: prohibited
2026-08-18 amendment: the former MITC4 fixed absolute tolerance is superseded by the common
family-scale row/RMS policy in `2026-08-17-common-reference-tolerance-design.md` and ADR-022.
The independent-solver, minimal-reference-case and drilling decisions remain unchanged.
## Decision ## Decision
FESA is an independent finite element solver. It accepts an approved subset of the FESA is an independent finite element solver. It accepts an approved subset of the
@@ -84,19 +89,23 @@ Only matched global `U1/U2/U3` rows are blocking. `UR1/UR2/UR3` rows are compare
reported as warning-only evidence. Reactions and stresses may be inspected but are not reported as warning-only evidence. Reactions and stresses may be inspected but are not
reference pass/fail quantities. reference pass/fail quantities.
For every matched U/UR row: The project-wide common policy now evaluates each matched U/UR component family:
```text ```text
tolerance = 1.0e-5 S = max(abs(reference_value_i))
near_zero_i = abs(reference_value_i) <= 0.01 * S
ordinary_row_pass_i = abs(error_i) / abs(reference_value_i) <= 0.05
near_zero_row_pass_i = abs(error_i) <= 0.01 * S
relative_rms = sqrt(mean(error_i^2)) / S <= 0.01
``` ```
The fixed value is expressed in the model's user-consistent length unit for translation The scale is computed from read-only reference values in the same model, step/frame,
and as a dimensionless rotation value for warning-only UR. Reference scale may remain quantity and component family. If `S == 0`, only exact-zero FESA values pass; otherwise
diagnostic report data but does not enter the tolerance or normalized-error denominator. the reason is `zero-reference-scale-nonzero-error`. There is no independent absolute-error
Values are not zero-clamped. Missing, duplicate, unmatched, or nonfinite required rows gate. Values are not zero-clamped. Missing, duplicate, unmatched, or nonfinite required
fail before numeric tolerance evaluation. U tolerance failure fails reference rows fail before numeric tolerance evaluation. U tolerance failure fails reference
verification; UR tolerance exceedance emits a deterministic warning only. The B33 mixed verification; UR tolerance exceedance emits a deterministic warning only. B33 uses the
tolerance remains unchanged. same row and family-RMS thresholds.
No additional locking, convergence, distorted, curved, director-angle, or invalid No additional locking, convergence, distorted, curved, director-angle, or invalid
geometry reference portfolio is required for this MITC4 implementation acceptance. geometry reference portfolio is required for this MITC4 implementation acceptance.
@@ -147,7 +156,7 @@ For MITC4:
- fixed drilling stabilization closes `NR-O01` and removes `NR-O02` by scope; - fixed drilling stabilization closes `NR-O01` and removes `NR-O02` by scope;
- `NR-O03` and `NR-O04` are removed from the approved verification scope; - `NR-O03` and `NR-O04` are removed from the approved verification scope;
- the fixed absolute MITC4 tolerance `1.0e-5` closes the blocking part of `NR-O05`; - the common family-scale row/RMS policy closes the blocking part of `NR-O05`;
- missing heavyweight reference metadata is `not-required-by-policy`; - missing heavyweight reference metadata is `not-required-by-policy`;
- confirmed formulation defects remain blocking. - confirmed formulation defects remain blocking.
@@ -3,11 +3,15 @@
## Status ## Status
- date: `2026-08-17` - date: `2026-08-17`
- status: `approved-design-pending-implementation` - status: `implemented-and-verified`
- verified_on: `2026-08-18`
- scope: 모든 FESA 외부 reference comparison - scope: 모든 FESA 외부 reference comparison
- production_solver_behavior: 변경 없음 - production_solver_behavior: 변경 없음
- reference_artifact_mutation: 금지 - reference_artifact_mutation: 금지
현재 운영 상수, 검증 순서, report schema와 변경 관리는 `docs/TOLERANCE.md`를 source of
truth로 사용한다. 이 문서는 승인 결정의 설계 근거와 대안 검토를 보존한다.
## 목적 ## 목적
B33, MITC4 및 이후 추가되는 기능이 서로 다른 수치 tolerance 식을 만들지 않도록 공통 B33, MITC4 및 이후 추가되는 기능이 서로 다른 수치 tolerance 식을 만들지 않도록 공통
@@ -67,6 +71,26 @@ S_g = \max_{i \in g}|R_i|
dimension, components 및 behavior를 명시해 family에 배치한다. 물리 차원이 같다는 이유만으로 dimension, components 및 behavior를 명시해 family에 배치한다. 물리 차원이 같다는 이유만으로
서로 다른 logical quantity를 자동으로 합치지 않는다. 서로 다른 logical quantity를 자동으로 합치지 않는다.
### B33 element-endpoint identity amendment
2026-08-18에 사용자가 다시 생성한
`reference/cantilever beam/cantilever beam elemental forces.csv``Element Label`
`Node Label`을 모두 포함하고 B33 요소마다 두 endpoint 행을 제공한다. Section-resultant
comparison은 다음 key로 각 CSV 행을 HDF5 `[element, endpoint, component]` 행에 직접
대응시킨다.
```text
(model_id, step/frame, instance_name, source_element_label,
source_node_label, quantity, component)
```
Component mapping은 `SF-SF1 -> N`, `SM-SM1 -> My`, `SM-SM2 -> Mz`,
`SM-SM3 -> T`다. Comparator는 이 CSV를 source-node station으로 collapse하거나 인접
endpoint를 평균하지 않는다. Missing, extra, duplicate 또는 connectivity-mismatched
element-endpoint identity는 tolerance 전에 실패한다. Production
`ResultRecovery::NormalizeSectionResultantsToNodeStations` 계약과 그 단위 테스트는 이 외부
reference identity 변경의 범위가 아니다.
## 행별 판정 ## 행별 판정
각 대응 행에 대해 다음 absolute error를 계산한다. 각 대응 행에 대해 다음 absolute error를 계산한다.
@@ -149,15 +173,16 @@ Comparator report는 최소한 다음 항목을 보존한다.
## 현재 B33/MITC4 Evidence ## 현재 B33/MITC4 Evidence
기존 solver를 재실행하거나 reference artifact를 수정하지 않고 다음 생성 ledger의 원본 MSVC Debug build와 214개 CTest를 통과한 현재 comparator가 다음 ledger를 생성했다.
FESA/reference 값을 새 규칙으로 재계산했다. Reference artifact는 수정하지 않았으며 B33은 regenerated 20-row element-endpoint identity를
직접 비교한다.
- `.harness/build/reference/cantilever-beam-b33/comparison.json` - `.harness/build/reference/cantilever-beam-b33/comparison.json`
- `.harness/build/reference/mitc4-shell-s4-comparison/comparison.json` - `.harness/build/reference/mitc4-shell-s4-comparison/comparison.json`
| case | row gate | maximum family relative RMS | RMS gate | verdict | | case | row gate | maximum family relative RMS | RMS gate | verdict |
| --- | --- | ---: | --- | --- | | --- | --- | ---: | --- | --- |
| B33 | 모든 blocking 행 통과 | 약 `1.17e-8` | 통과 | 통과 | | B33 direct endpoint | 모든 212 blocking 행 통과 | 약 `1.17e-8` | 통과 | 통과 |
| MITC4 U | 모든 blocking 행 통과 | 약 `1.03e-3` | 통과 | 통과 | | MITC4 U | 모든 blocking 행 통과 | 약 `1.03e-3` | 통과 | 통과 |
| MITC4 UR | 모든 warning-only 행 통과 | 약 `2.98e-3` | 통과 | warning 없음 | | MITC4 UR | 모든 warning-only 행 통과 | 약 `2.98e-3` | 통과 | warning 없음 |
@@ -1,12 +1,21 @@
Frame,Part Instance Name, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3 Frame,Part Instance Name, Element Label, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3
Increment 1: Step Time = 1.000,PART-1_1-1,1,0.000000000E+00,1.000000000E+07,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,1,1,0.000000000E+00,1.000000000E+07,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,1,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,2,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,2,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,3,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,3,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,4,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,4,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,5,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,10,0.000000000E+00,1.000000000E+06,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,5,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,11,0.000000000E+00,-1.560000000E-02,0.000000000E+00,0.000000000E+00 Increment 1: Step Time = 1.000,PART-1_1-1,6,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,6,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,7,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,7,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,8,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,8,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,9,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,9,10,0.000000000E+00,9.999998750E+05,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,10,10,0.000000000E+00,9.999998750E+05,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,10,11,0.000000000E+00,-1.562500000E-02,0.000000000E+00,0.000000000E+00
1 Frame Part Instance Name Element Label Node Label SF-SF1 SM-SM1 SM-SM2 SM-SM3
2 Increment 1: Step Time = 1.000 PART-1_1-1 1 1 0.000000000E+00 1.000000000E+07 0.000000000E+00 0.000000000E+00
3 Increment 1: Step Time = 1.000 PART-1_1-1 1 2 0.000000000E+00 9.000000000E+06 0.000000000E+00 0.000000000E+00
4 Increment 1: Step Time = 1.000 PART-1_1-1 2 3 2 0.000000000E+00 8.000000000E+06 9.000000000E+06 0.000000000E+00 0.000000000E+00
5 Increment 1: Step Time = 1.000 PART-1_1-1 2 4 3 0.000000000E+00 7.000000000E+06 8.000000000E+06 0.000000000E+00 0.000000000E+00
6 Increment 1: Step Time = 1.000 PART-1_1-1 3 5 3 0.000000000E+00 6.000000000E+06 8.000000000E+06 0.000000000E+00 0.000000000E+00
7 Increment 1: Step Time = 1.000 PART-1_1-1 3 6 4 0.000000000E+00 5.000000000E+06 7.000000000E+06 0.000000000E+00 0.000000000E+00
8 Increment 1: Step Time = 1.000 PART-1_1-1 4 7 4 0.000000000E+00 4.000000000E+06 7.000000000E+06 0.000000000E+00 0.000000000E+00
9 Increment 1: Step Time = 1.000 PART-1_1-1 4 8 5 0.000000000E+00 3.000000000E+06 6.000000000E+06 0.000000000E+00 0.000000000E+00
10 Increment 1: Step Time = 1.000 PART-1_1-1 5 9 5 0.000000000E+00 2.000000000E+06 6.000000000E+06 0.000000000E+00 0.000000000E+00
11 Increment 1: Step Time = 1.000 PART-1_1-1 5 10 6 0.000000000E+00 1.000000000E+06 5.000000000E+06 0.000000000E+00 0.000000000E+00
12 Increment 1: Step Time = 1.000 PART-1_1-1 6 11 6 0.000000000E+00 -1.560000000E-02 5.000000000E+06 0.000000000E+00 0.000000000E+00
13 Increment 1: Step Time = 1.000 PART-1_1-1 6 7 0.000000000E+00 4.000000000E+06 0.000000000E+00 0.000000000E+00
14 Increment 1: Step Time = 1.000 PART-1_1-1 7 7 0.000000000E+00 4.000000000E+06 0.000000000E+00 0.000000000E+00
15 Increment 1: Step Time = 1.000 PART-1_1-1 7 8 0.000000000E+00 3.000000000E+06 0.000000000E+00 0.000000000E+00
16 Increment 1: Step Time = 1.000 PART-1_1-1 8 8 0.000000000E+00 3.000000000E+06 0.000000000E+00 0.000000000E+00
17 Increment 1: Step Time = 1.000 PART-1_1-1 8 9 0.000000000E+00 2.000000000E+06 0.000000000E+00 0.000000000E+00
18 Increment 1: Step Time = 1.000 PART-1_1-1 9 9 0.000000000E+00 2.000000000E+06 0.000000000E+00 0.000000000E+00
19 Increment 1: Step Time = 1.000 PART-1_1-1 9 10 0.000000000E+00 9.999998750E+05 0.000000000E+00 0.000000000E+00
20 Increment 1: Step Time = 1.000 PART-1_1-1 10 10 0.000000000E+00 9.999998750E+05 0.000000000E+00 0.000000000E+00
21 Increment 1: Step Time = 1.000 PART-1_1-1 10 11 0.000000000E+00 -1.562500000E-02 0.000000000E+00 0.000000000E+00
+2
View File
@@ -76,6 +76,8 @@ target_link_libraries(
add_executable( add_executable(
fesa_reference_tests fesa_reference_tests
reference/reference_tolerance_policy.cpp
reference/reference_tolerance_policy_test.cpp
reference/reference_comparison.cpp reference/reference_comparison.cpp
reference/reference_comparison_test.cpp reference/reference_comparison_test.cpp
reference/b33_reference_comparison_test.cpp reference/b33_reference_comparison_test.cpp
@@ -26,8 +26,8 @@
namespace { namespace {
constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11"; constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11";
constexpr std::size_t kExpectedRowCount = 176U; constexpr std::size_t kExpectedRowCount = 212U;
constexpr std::size_t kExpectedMetricCount = 16U; constexpr std::size_t kExpectedMetricCount = 6U;
class Hdf5Handle { class Hdf5Handle {
public: public:
@@ -192,7 +192,10 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) {
report.rows.begin(), report.rows.end(), report.rows.begin(), report.rows.end(),
[](const fesa::test::RowDecision& row) { [](const fesa::test::RowDecision& row) {
return row.passed && std::isfinite(row.absolute_error) && return row.passed && std::isfinite(row.absolute_error) &&
std::isfinite(row.tolerance) && row.tolerance > 0.0 && std::isfinite(row.tolerance) && row.tolerance >= 0.0 &&
std::isfinite(row.reference_scale) &&
std::isfinite(row.near_zero_band) &&
std::isfinite(row.relative_error) &&
row.fesa.model_id == "cantilever-beam-b33" && row.fesa.model_id == "cantilever-beam-b33" &&
row.reference.model_id == "cantilever-beam-b33" && row.reference.model_id == "cantilever-beam-b33" &&
row.fesa.step_name == "Step-1" && row.fesa.step_name == "Step-1" &&
@@ -206,11 +209,11 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) {
std::all_of(report.metrics.begin(), report.metrics.end(), std::all_of(report.metrics.begin(), report.metrics.end(),
[](const fesa::test::ComponentMetrics& metric) { [](const fesa::test::ComponentMetrics& metric) {
return std::isfinite(metric.reference_scale) && return std::isfinite(metric.reference_scale) &&
std::isfinite(metric.near_zero_band) &&
std::isfinite(metric.maximum_absolute_error) && std::isfinite(metric.maximum_absolute_error) &&
std::isfinite(metric.maximum_normalized_error) && std::isfinite(metric.relative_rms) &&
std::isfinite(metric.rms_error) && metric.rms_passed && metric.passed &&
std::isfinite(metric.norm_error) && metric.row_count > 0U && !metric.components.empty();
metric.maximum_normalized_error <= 1.0;
})); }));
EXPECT_FALSE(report.stress_comparison_applicable); EXPECT_FALSE(report.stress_comparison_applicable);
@@ -234,6 +237,9 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) {
EXPECT_NE(json.find("\"stress_comparison_applicable\":false"), EXPECT_NE(json.find("\"stress_comparison_applicable\":false"),
std::string::npos); std::string::npos);
EXPECT_NE(json.find("\"physics_evidence\""), std::string::npos); EXPECT_NE(json.find("\"physics_evidence\""), std::string::npos);
EXPECT_NE(json.find("\"source_element_label\""), std::string::npos);
EXPECT_NE(json.find("\"family_identity\""), std::string::npos);
EXPECT_NE(json.find("\"row_count\""), std::string::npos);
std::vector<std::string> generated_names; std::vector<std::string> generated_names;
for (const auto& entry : for (const auto& entry :
+30 -7
View File
@@ -122,7 +122,7 @@ void ExpectCommonMetadata(const fesa::test::Mitc4ComparisonReport& report,
void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) { void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) {
ASSERT_EQ(report.rows.size(), kNodeCount * kComponentCount); ASSERT_EQ(report.rows.size(), kNodeCount * kComponentCount);
ASSERT_EQ(report.metrics.size(), kComponentCount); ASSERT_EQ(report.metrics.size(), 2U);
ASSERT_EQ(report.vector_metrics.size(), kNodeCount); ASSERT_EQ(report.vector_metrics.size(), kNodeCount);
EXPECT_TRUE(report.passed); EXPECT_TRUE(report.passed);
const std::size_t blocking_rows = static_cast<std::size_t>(std::count_if( const std::size_t blocking_rows = static_cast<std::size_t>(std::count_if(
@@ -135,12 +135,35 @@ void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) {
[](const fesa::test::Mitc4RowDecision& row) { [](const fesa::test::Mitc4RowDecision& row) {
return !row.blocking || row.within_tolerance; return !row.blocking || row.within_tolerance;
})); }));
EXPECT_EQ(report.warnings.size(), const auto rotation_metric =
static_cast<std::size_t>( std::find_if(report.metrics.begin(), report.metrics.end(),
std::count_if(report.rows.begin(), report.rows.end(), [](const fesa::test::Mitc4ComponentMetrics& metric) {
[](const fesa::test::Mitc4RowDecision& row) { return !metric.blocking;
return !row.blocking && !row.within_tolerance; });
}))); ASSERT_NE(rotation_metric, report.metrics.end());
const std::size_t row_warning_count = static_cast<std::size_t>(
std::count_if(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) {
return !row.blocking && !row.within_tolerance;
}));
ASSERT_EQ(report.warnings.size(),
row_warning_count + (rotation_metric->rms_passed ? 0U : 1U));
std::size_t warning_index = 0U;
for (std::size_t row_index = 0U; row_index < report.rows.size();
++row_index) {
const auto& row = report.rows[row_index];
if (!row.blocking && !row.within_tolerance) {
EXPECT_EQ(report.warnings[warning_index].code,
"rotation-reference-exceedance");
EXPECT_EQ(report.warnings[warning_index].row, row_index);
++warning_index;
}
}
if (!rotation_metric->rms_passed) {
EXPECT_EQ(report.warnings[warning_index].code,
"rotation-reference-rms-exceedance");
EXPECT_EQ(report.warnings[warning_index].row, rotation_metric->worst_row);
}
} }
// MITC4-E2E-S4-001 // MITC4-E2E-S4-001
+154 -79
View File
@@ -24,6 +24,8 @@
#include <utility> #include <utility>
#include <vector> #include <vector>
#include "reference_tolerance_policy.h"
namespace fesa::test { namespace fesa::test {
namespace { namespace {
@@ -32,7 +34,6 @@ constexpr const char* kDisplacementPath =
constexpr const char* kInternalFormulation = "FESA-MITC4"; constexpr const char* kInternalFormulation = "FESA-MITC4";
constexpr const char* kIntegrationRule = constexpr const char* kIntegrationRule =
"2x2x2-gauss; mitc4-edge-midpoint-shear"; "2x2x2-gauss; mitc4-edge-midpoint-shear";
constexpr double kFixedAbsoluteTolerance = 1.0e-5;
constexpr std::array<const char*, 6> kComponents{"U1", "U2", "U3", constexpr std::array<const char*, 6> kComponents{"U1", "U2", "U3",
"UR1", "UR2", "UR3"}; "UR1", "UR2", "UR3"};
const std::vector<std::string> expected_header{"Part Instance Name", const std::vector<std::string> expected_header{"Part Instance Name",
@@ -648,6 +649,18 @@ std::string FiniteText(const double value) {
return stream.str(); return stream.str();
} }
const char* BranchName(const ReferenceToleranceBranch branch) {
switch (branch) {
case ReferenceToleranceBranch::kNearZero:
return "near-zero";
case ReferenceToleranceBranch::kRelative:
return "relative";
case ReferenceToleranceBranch::kZeroScaleExact:
return "zero-scale-exact";
}
return "unknown";
}
std::string JsonEscape(const std::string& value) { std::string JsonEscape(const std::string& value) {
std::ostringstream stream; std::ostringstream stream;
for (const unsigned char character : value) { for (const unsigned char character : value) {
@@ -687,6 +700,18 @@ std::string JsonEscape(const std::string& value) {
return stream.str(); return stream.str();
} }
void WriteJsonStrings(std::ostream& stream,
const std::vector<std::string>& values) {
stream << '[';
for (std::size_t index = 0U; index < values.size(); ++index) {
if (index != 0U) {
stream << ',';
}
stream << '"' << JsonEscape(values[index]) << '"';
}
stream << ']';
}
} // namespace } // namespace
Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare( Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare(
@@ -713,14 +738,6 @@ Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare(
} }
} }
std::array<double, 6> scales{};
for (const auto& row : reference_rows) {
for (std::size_t component = 0U; component < scales.size(); ++component) {
scales[component] =
(std::max)(scales[component], std::abs(row.values[component]));
}
}
Mitc4ComparisonReport report{}; Mitc4ComparisonReport report{};
report.case_id = reference_case.case_id; report.case_id = reference_case.case_id;
report.source_element_type = std::move(hdf5.source_element_type); report.source_element_type = std::move(hdf5.source_element_type);
@@ -729,80 +746,126 @@ Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare(
report.passed = true; report.passed = true;
report.rows.reserve(hdf5.rows.size() * kComponents.size()); report.rows.reserve(hdf5.rows.size() * kComponents.size());
report.vector_metrics.reserve(hdf5.rows.size()); report.vector_metrics.reserve(hdf5.rows.size());
std::array<double, 6> error_norms{}; std::array<std::vector<std::size_t>, 2> family_rows;
std::array<double, 6> maximum_errors{};
std::array<double, 6> maximum_normalized{};
std::array<std::size_t, 6> worst_rows{};
double global_worst_normalized = -1.0;
for (const auto& hdf5_row : hdf5.rows) { for (const auto& hdf5_row : hdf5.rows) {
const auto reference = reference_by_identity.find(hdf5_row.identity); const auto reference = reference_by_identity.find(hdf5_row.identity);
if (reference == reference_by_identity.end()) { if (reference == reference_by_identity.end()) {
Fail("schema-mismatch", "A projected reference row is missing."); Fail("schema-mismatch", "A projected reference row is missing.");
} }
std::array<double, 6> errors{};
for (std::size_t component = 0U; component < kComponents.size(); for (std::size_t component = 0U; component < kComponents.size();
++component) { ++component) {
const double tolerance = kFixedAbsoluteTolerance;
const double absolute_error = std::abs(
hdf5_row.values[component] - reference->second->values[component]);
const double normalized_error = absolute_error / tolerance;
if (!std::isfinite(tolerance) || !(tolerance > 0.0) ||
!std::isfinite(absolute_error) ||
!std::isfinite(normalized_error)) {
Fail("schema-mismatch",
"A finite row produced a nonfinite comparison metric.");
}
const bool blocking = component < 3U; const bool blocking = component < 3U;
const bool within_tolerance = absolute_error <= tolerance;
const std::size_t row_index = report.rows.size(); const std::size_t row_index = report.rows.size();
report.rows.push_back( report.rows.push_back({reference_case.case_id,
{reference_case.case_id, hdf5_row.identity.instance_name, hdf5_row.identity.instance_name,
hdf5_row.identity.source_node_label, kComponents[component], hdf5_row.identity.source_node_label,
hdf5_row.values[component], reference->second->values[component], kComponents[component],
absolute_error, tolerance, normalized_error, blocking, hdf5_row.values[component],
within_tolerance}); reference->second->values[component],
errors[component] = absolute_error; 0.0,
error_norms[component] = 0.0,
std::hypot(error_norms[component], absolute_error); 0.0,
if (normalized_error > maximum_normalized[component]) { 0.0,
maximum_normalized[component] = normalized_error; 0.0,
maximum_errors[component] = absolute_error; false,
worst_rows[component] = row_index; {},
} blocking,
if (normalized_error > global_worst_normalized) { false});
global_worst_normalized = normalized_error; family_rows[blocking ? 0U : 1U].push_back(row_index);
report.worst_row = row_index;
}
if (blocking && !within_tolerance) {
report.passed = false;
} else if (!blocking && !within_tolerance) {
const std::string message =
"case=" + reference_case.case_id +
";instance=" + hdf5_row.identity.instance_name +
";node=" + std::to_string(hdf5_row.identity.source_node_label) +
";component=" + kComponents[component] +
";absolute_error=" + FiniteText(absolute_error) +
";tolerance=" + FiniteText(tolerance);
report.warnings.push_back(
{"rotation-reference-exceedance", row_index, message});
}
} }
report.vector_metrics.push_back(
{hdf5_row.identity.instance_name, hdf5_row.identity.source_node_label,
std::hypot(errors[0U], errors[1U], errors[2U]),
std::hypot(errors[3U], errors[4U], errors[5U])});
} }
report.metrics.reserve(kComponents.size()); std::vector<ReferenceToleranceFamily> families;
const double row_count = static_cast<double>(hdf5.rows.size()); families.reserve(2U);
for (std::size_t component = 0U; component < kComponents.size(); for (std::size_t family_index = 0U; family_index < family_rows.size();
++component) { ++family_index) {
ReferenceToleranceFamily family{};
family.identity = family_index == 0U ? "displacement-translation"
: "displacement-rotation";
family.components = family_index == 0U
? std::vector<std::string>{"U1", "U2", "U3"}
: std::vector<std::string>{"UR1", "UR2", "UR3"};
family.blocking = family_index == 0U;
family.rows.reserve(family_rows[family_index].size());
for (const std::size_t row_index : family_rows[family_index]) {
const auto& row = report.rows[row_index];
family.rows.push_back({row.fesa_value, row.reference_value});
}
families.push_back(std::move(family));
}
const auto evaluation = ReferenceTolerancePolicy::Evaluate(families);
if (!evaluation.valid || evaluation.families.size() != families.size()) {
Fail("schema-mismatch",
"A finite row produced a nonfinite comparison metric.");
}
report.metrics.reserve(evaluation.families.size());
double maximum_absolute_error = -1.0;
for (std::size_t family_index = 0U;
family_index < evaluation.families.size(); ++family_index) {
const auto& policy = evaluation.families[family_index];
for (std::size_t local = 0U; local < family_rows[family_index].size();
++local) {
const std::size_t row_index = family_rows[family_index][local];
auto& row = report.rows[row_index];
const auto& decision = policy.rows[local];
row.absolute_error = decision.absolute_error;
row.tolerance = decision.threshold;
row.normalized_error = decision.relative_error;
row.reference_scale = policy.reference_scale;
row.near_zero_band = policy.near_zero_band;
row.relative_error_applicable = decision.relative_error_applicable;
row.tolerance_branch = BranchName(decision.branch);
row.within_tolerance = decision.passed;
if (row.absolute_error > maximum_absolute_error) {
maximum_absolute_error = row.absolute_error;
report.worst_row = row_index;
}
if (!row.within_tolerance) {
if (row.blocking) {
report.passed = false;
} else {
const std::string message =
"case=" + reference_case.case_id +
";instance=" + row.instance_name +
";node=" + std::to_string(row.source_node_label) +
";component=" + row.component +
";absolute_error=" + FiniteText(row.absolute_error) +
";threshold=" + FiniteText(row.tolerance);
report.warnings.push_back(
{"rotation-reference-exceedance", row_index, message});
}
}
}
const std::size_t worst_row = family_rows[family_index][policy.worst_row];
report.metrics.push_back( report.metrics.push_back(
{kComponents[component], scales[component], kFixedAbsoluteTolerance, {policy.identity, policy.components, policy.blocking,
maximum_errors[component], maximum_normalized[component], policy.reference_scale, policy.near_zero_band,
error_norms[component] / std::sqrt(row_count), policy.near_zero_count, policy.maximum_absolute_error,
error_norms[component], worst_rows[component]}); policy.relative_rms, worst_row, policy.rms_passed, policy.passed,
policy.diagnostic_code, policy.row_count});
if (policy.blocking && !policy.passed) {
report.passed = false;
}
if (!policy.blocking && !policy.rms_passed) {
report.warnings.push_back(
{"rotation-reference-rms-exceedance", worst_row,
"case=" + reference_case.case_id + ";family=" + policy.identity +
";relative_rms=" + FiniteText(policy.relative_rms)});
}
}
for (std::size_t node = 0U; node < hdf5.rows.size(); ++node) {
const std::size_t row_offset = node * kComponents.size();
report.vector_metrics.push_back(
{report.rows[row_offset].instance_name,
report.rows[row_offset].source_node_label,
std::hypot(report.rows[row_offset].absolute_error,
report.rows[row_offset + 1U].absolute_error,
report.rows[row_offset + 2U].absolute_error),
std::hypot(report.rows[row_offset + 3U].absolute_error,
report.rows[row_offset + 4U].absolute_error,
report.rows[row_offset + 5U].absolute_error)});
} }
return Result<Mitc4ComparisonReport>::Success(std::move(report)); return Result<Mitc4ComparisonReport>::Success(std::move(report));
} catch (const ComparisonFailure& exception) { } catch (const ComparisonFailure& exception) {
@@ -845,8 +908,14 @@ Status Mitc4ReferenceComparison::WriteDeterministicJson(
<< "\",\"fesa_value\":" << row.fesa_value << "\",\"fesa_value\":" << row.fesa_value
<< ",\"reference_value\":" << row.reference_value << ",\"reference_value\":" << row.reference_value
<< ",\"absolute_error\":" << row.absolute_error << ",\"absolute_error\":" << row.absolute_error
<< ",\"tolerance\":" << row.tolerance << ",\"threshold\":" << row.tolerance
<< ",\"normalized_error\":" << row.normalized_error << ",\"relative_error\":" << row.normalized_error
<< ",\"reference_scale\":" << row.reference_scale
<< ",\"near_zero_band\":" << row.near_zero_band
<< ",\"relative_error_applicable\":"
<< (row.relative_error_applicable ? "true" : "false")
<< ",\"tolerance_branch\":\"" << JsonEscape(row.tolerance_branch)
<< "\""
<< ",\"blocking\":" << (row.blocking ? "true" : "false") << ",\"blocking\":" << (row.blocking ? "true" : "false")
<< ",\"within_tolerance\":" << ",\"within_tolerance\":"
<< (row.within_tolerance ? "true" : "false") << '}'; << (row.within_tolerance ? "true" : "false") << '}';
@@ -857,15 +926,21 @@ Status Mitc4ReferenceComparison::WriteDeterministicJson(
stream << ','; stream << ',';
} }
const auto& metric = report.metrics[index]; const auto& metric = report.metrics[index];
stream << "{\"component\":\"" << JsonEscape(metric.component) stream << "{\"family_identity\":\"" << JsonEscape(metric.family_identity)
<< "\",\"reference_scale\":" << metric.reference_scale << "\",\"components\":";
<< ",\"tolerance\":" << metric.tolerance WriteJsonStrings(stream, metric.components);
stream << ",\"blocking\":" << (metric.blocking ? "true" : "false")
<< ",\"reference_scale\":" << metric.reference_scale
<< ",\"near_zero_band\":" << metric.near_zero_band
<< ",\"near_zero_count\":" << metric.near_zero_count
<< ",\"row_count\":" << metric.row_count
<< ",\"maximum_absolute_error\":" << metric.maximum_absolute_error << ",\"maximum_absolute_error\":" << metric.maximum_absolute_error
<< ",\"maximum_normalized_error\":" << ",\"relative_rms\":" << metric.relative_rms
<< metric.maximum_normalized_error << ",\"worst_row\":" << metric.worst_row
<< ",\"rms_error\":" << metric.rms_error << ",\"rms_passed\":" << (metric.rms_passed ? "true" : "false")
<< ",\"vector_norm_error\":" << metric.vector_norm_error << ",\"passed\":" << (metric.passed ? "true" : "false")
<< ",\"worst_row\":" << metric.worst_row << '}'; << ",\"diagnostic_code\":\"" << JsonEscape(metric.diagnostic_code)
<< "\"}";
} }
stream << "],\"vector_metrics\":["; stream << "],\"vector_metrics\":[";
for (std::size_t index = 0U; index < report.vector_metrics.size(); for (std::size_t index = 0U; index < report.vector_metrics.size();
+14 -5
View File
@@ -29,19 +29,28 @@ struct Mitc4RowDecision {
double absolute_error; double absolute_error;
double tolerance; double tolerance;
double normalized_error; double normalized_error;
double reference_scale;
double near_zero_band;
bool relative_error_applicable;
std::string tolerance_branch;
bool blocking; bool blocking;
bool within_tolerance; bool within_tolerance;
}; };
struct Mitc4ComponentMetrics { struct Mitc4ComponentMetrics {
std::string component; std::string family_identity;
std::vector<std::string> components;
bool blocking;
double reference_scale; double reference_scale;
double tolerance; double near_zero_band;
std::size_t near_zero_count;
double maximum_absolute_error; double maximum_absolute_error;
double maximum_normalized_error; double relative_rms;
double rms_error;
double vector_norm_error;
std::size_t worst_row; std::size_t worst_row;
bool rms_passed;
bool passed;
std::string diagnostic_code;
std::size_t row_count{};
}; };
struct Mitc4VectorMetrics { struct Mitc4VectorMetrics {
@@ -458,11 +458,11 @@ const fesa::test::Mitc4RowDecision* FindRow(
const fesa::test::Mitc4ComponentMetrics* FindMetric( const fesa::test::Mitc4ComponentMetrics* FindMetric(
const fesa::test::Mitc4ComparisonReport& report, const fesa::test::Mitc4ComparisonReport& report,
const std::string& component) { const std::string& family_identity) {
const auto found = const auto found =
std::find_if(report.metrics.begin(), report.metrics.end(), std::find_if(report.metrics.begin(), report.metrics.end(),
[&](const fesa::test::Mitc4ComponentMetrics& metric) { [&](const fesa::test::Mitc4ComponentMetrics& metric) {
return metric.component == component; return metric.family_identity == family_identity;
}); });
return found == report.metrics.end() ? nullptr : &*found; return found == report.metrics.end() ? nullptr : &*found;
} }
@@ -496,6 +496,9 @@ TEST(Mitc4ReferenceComparison,
EXPECT_EQ(report.rows[0U].component, "U1"); EXPECT_EQ(report.rows[0U].component, "U1");
EXPECT_EQ(report.rows[5U].component, "UR3"); EXPECT_EQ(report.rows[5U].component, "UR3");
EXPECT_EQ(report.rows[6U].source_node_label, 2); EXPECT_EQ(report.rows[6U].source_node_label, 2);
ASSERT_EQ(report.metrics.size(), 2U);
EXPECT_EQ(report.metrics[0U].row_count, 6U);
EXPECT_EQ(report.metrics[1U].row_count, 6U);
EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(), EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) { [](const fesa::test::Mitc4RowDecision& row) {
return row.case_id == "shell-contract" && return row.case_id == "shell-contract" &&
@@ -588,7 +591,7 @@ TEST(Mitc4ReferenceComparison, RejectsInvalidInventoryBeforeNumericComparison) {
// MITC4-REF-003 // MITC4-REF-003
TEST(Mitc4ReferenceComparison, TEST(Mitc4ReferenceComparison,
AppliesFixedAbsoluteToleranceWithoutScaleClampOrRowDenominator) { SharesTranslationScaleAcrossUComponentsAndUsesNearZeroBranches) {
ContractFixture fixture{"tolerance"}; ContractFixture fixture{"tolerance"};
auto reference = DefaultRows(); auto reference = DefaultRows();
reference[0U].values[0U] = 0.0; reference[0U].values[0U] = 0.0;
@@ -597,10 +600,10 @@ TEST(Mitc4ReferenceComparison,
reference[1U].values[1U] = 0.0; reference[1U].values[1U] = 0.0;
WriteCsv(fixture.Csv(), reference); WriteCsv(fixture.Csv(), reference);
auto fesa_values = reference; auto fesa_values = reference;
fesa_values[0U].values[0U] = 0.9999e-5; fesa_values[0U].values[0U] = 0.999 * 8.0e-2;
fesa_values[1U].values[0U] += 1.0001e-5; fesa_values[1U].values[0U] += 1.001e-1;
fesa_values[0U].values[1U] = 0.9999e-5; fesa_values[0U].values[1U] = 0.999 * 8.0e-2;
fesa_values[1U].values[1U] = 1.0001e-5; fesa_values[1U].values[1U] = 1.001 * 8.0e-2;
WriteHdf5(fixture.Results(), fixture.Input(), fesa_values); WriteHdf5(fixture.Results(), fixture.Input(), fesa_values);
auto result = auto result =
@@ -616,25 +619,21 @@ TEST(Mitc4ReferenceComparison,
ASSERT_NE(u1_scaled, nullptr); ASSERT_NE(u1_scaled, nullptr);
ASSERT_NE(u2_near, nullptr); ASSERT_NE(u2_near, nullptr);
ASSERT_NE(u2_over, nullptr); ASSERT_NE(u2_over, nullptr);
EXPECT_DOUBLE_EQ(u1_zero->tolerance, 1.0e-5); EXPECT_DOUBLE_EQ(u1_zero->tolerance, 8.0e-2);
EXPECT_TRUE(u1_zero->within_tolerance); EXPECT_TRUE(u1_zero->within_tolerance);
EXPECT_NEAR(u1_zero->normalized_error, 0.9999, 1.0e-12); EXPECT_FALSE(u1_zero->relative_error_applicable);
EXPECT_FALSE(u1_scaled->within_tolerance); EXPECT_FALSE(u1_scaled->within_tolerance);
EXPECT_NEAR(u1_scaled->normalized_error, 1.0001, 2.0e-11); EXPECT_NEAR(u1_scaled->normalized_error, 0.05005, 1.0e-12);
EXPECT_DOUBLE_EQ(u2_near->tolerance, 1.0e-5); EXPECT_DOUBLE_EQ(u2_near->tolerance, 8.0e-2);
EXPECT_TRUE(u2_near->within_tolerance); EXPECT_TRUE(u2_near->within_tolerance);
EXPECT_NEAR(u2_near->normalized_error, 0.9999, 1.0e-12); EXPECT_FALSE(u2_near->relative_error_applicable);
EXPECT_FALSE(u2_over->within_tolerance); EXPECT_FALSE(u2_over->within_tolerance);
EXPECT_NEAR(u2_over->normalized_error, 1.0001, 1.0e-12); EXPECT_DOUBLE_EQ(u2_over->tolerance, 8.0e-2);
for (const auto& row : report.rows) { const auto* translation = FindMetric(report, "displacement-translation");
EXPECT_DOUBLE_EQ(row.tolerance, 1.0e-5); ASSERT_NE(translation, nullptr);
} EXPECT_DOUBLE_EQ(translation->reference_scale, 8.0);
for (const auto& metric : report.metrics) { EXPECT_EQ(translation->components,
EXPECT_DOUBLE_EQ(metric.tolerance, 1.0e-5); (std::vector<std::string>{"U1", "U2", "U3"}));
}
const auto* u1_metric = FindMetric(report, "U1");
ASSERT_NE(u1_metric, nullptr);
EXPECT_DOUBLE_EQ(u1_metric->reference_scale, 2.0);
} }
// MITC4-REF-004 // MITC4-REF-004
@@ -650,8 +649,9 @@ TEST(Mitc4ReferenceComparison,
ASSERT_TRUE(result.HasValue()); ASSERT_TRUE(result.HasValue());
const auto& report = result.Value(); const auto& report = result.Value();
EXPECT_TRUE(report.passed); EXPECT_TRUE(report.passed);
ASSERT_EQ(report.warnings.size(), 1U); ASSERT_EQ(report.warnings.size(), 2U);
EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance"); EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance");
EXPECT_EQ(report.warnings[1U].code, "rotation-reference-rms-exceedance");
const auto* row = FindRow(report, 1, "UR1"); const auto* row = FindRow(report, 1, "UR1");
ASSERT_NE(row, nullptr); ASSERT_NE(row, nullptr);
EXPECT_FALSE(row->blocking); EXPECT_FALSE(row->blocking);
@@ -663,6 +663,47 @@ TEST(Mitc4ReferenceComparison,
EXPECT_NE(report.warnings[0U].message.find("UR1"), std::string::npos); EXPECT_NE(report.warnings[0U].message.find("UR1"), std::string::npos);
} }
TEST(Mitc4ReferenceComparison,
RotationRmsExceedanceWarnsWhenEveryRotationRowPasses) {
ContractFixture fixture{"rotation-rms"};
auto fesa_values = DefaultRows();
fesa_values[1U].values[3U] = 0.1049;
fesa_values[1U].values[4U] = -0.2098;
fesa_values[1U].values[5U] = 0.3147;
WriteHdf5(fixture.Results(), fixture.Input(), fesa_values);
auto result =
fesa::test::Mitc4ReferenceComparison::Compare(fixture.ReferenceCase());
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
EXPECT_TRUE(report.passed);
ASSERT_EQ(report.warnings.size(), 1U);
EXPECT_EQ(report.warnings[0U].code, "rotation-reference-rms-exceedance");
EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) {
return row.within_tolerance;
}));
}
TEST(Mitc4ReferenceComparison,
RotationRowAndRmsExceedancesProduceSeparateWarnings) {
ContractFixture fixture{"rotation-row-and-rms"};
auto fesa_values = DefaultRows();
fesa_values[0U].values[3U] = 1.0;
WriteHdf5(fixture.Results(), fixture.Input(), fesa_values);
auto result =
fesa::test::Mitc4ReferenceComparison::Compare(fixture.ReferenceCase());
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
EXPECT_TRUE(report.passed);
ASSERT_EQ(report.warnings.size(), 2U);
EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance");
EXPECT_EQ(report.warnings[1U].code, "rotation-reference-rms-exceedance");
EXPECT_LT(report.warnings[0U].row, report.rows.size());
EXPECT_LT(report.warnings[1U].row, report.rows.size());
}
// MITC4-REF-005 // MITC4-REF-005
TEST(Mitc4ReferenceComparison, TEST(Mitc4ReferenceComparison,
ReportsMetricsVectorsWorstRowAndJsonDeterministically) { ReportsMetricsVectorsWorstRowAndJsonDeterministically) {
@@ -678,7 +719,7 @@ TEST(Mitc4ReferenceComparison,
ASSERT_TRUE(result.HasValue()); ASSERT_TRUE(result.HasValue());
const auto& report = result.Value(); const auto& report = result.Value();
ASSERT_TRUE(report.passed); ASSERT_TRUE(report.passed);
ASSERT_EQ(report.metrics.size(), 6U); ASSERT_EQ(report.metrics.size(), 2U);
ASSERT_EQ(report.vector_metrics.size(), 2U); ASSERT_EQ(report.vector_metrics.size(), 2U);
EXPECT_NEAR(report.vector_metrics[0U].displacement_norm_error, EXPECT_NEAR(report.vector_metrics[0U].displacement_norm_error,
std::sqrt(0.3125) * 1.0e-9, 1.0e-21); std::sqrt(0.3125) * 1.0e-9, 1.0e-21);
@@ -687,10 +728,10 @@ TEST(Mitc4ReferenceComparison,
EXPECT_EQ(report.rows[report.worst_row].component, "UR1"); EXPECT_EQ(report.rows[report.worst_row].component, "UR1");
for (const auto& metric : report.metrics) { for (const auto& metric : report.metrics) {
EXPECT_TRUE(std::isfinite(metric.reference_scale)); EXPECT_TRUE(std::isfinite(metric.reference_scale));
EXPECT_TRUE(std::isfinite(metric.near_zero_band));
EXPECT_TRUE(std::isfinite(metric.maximum_absolute_error)); EXPECT_TRUE(std::isfinite(metric.maximum_absolute_error));
EXPECT_TRUE(std::isfinite(metric.maximum_normalized_error)); EXPECT_TRUE(std::isfinite(metric.relative_rms));
EXPECT_TRUE(std::isfinite(metric.rms_error)); EXPECT_FALSE(metric.components.empty());
EXPECT_TRUE(std::isfinite(metric.vector_norm_error));
EXPECT_LT(metric.worst_row, report.rows.size()); EXPECT_LT(metric.worst_row, report.rows.size());
} }
@@ -704,8 +745,10 @@ TEST(Mitc4ReferenceComparison,
.IsOk()); .IsOk());
const std::string first = ReadBytes(json_a); const std::string first = ReadBytes(json_a);
EXPECT_EQ(first, ReadBytes(json_b)); EXPECT_EQ(first, ReadBytes(json_b));
for (const char* key : {"\"rows\"", "\"metrics\"", "\"vector_metrics\"", for (const char* key :
"\"warnings\"", "\"worst_row\"", "\"passed\""}) { {"\"rows\"", "\"metrics\"", "\"vector_metrics\"", "\"warnings\"",
"\"family_identity\"", "\"near_zero_band\"", "\"relative_rms\"",
"\"row_count\"", "\"worst_row\"", "\"passed\""}) {
EXPECT_NE(first.find(key), std::string::npos) << key; EXPECT_NE(first.find(key), std::string::npos) << key;
} }
} }
+255 -165
View File
@@ -29,7 +29,7 @@
#include "fesa/fem/dof_manager.h" #include "fesa/fem/dof_manager.h"
#include "fesa/io/abaqus/domain_mapper.h" #include "fesa/io/abaqus/domain_mapper.h"
#include "fesa/io/abaqus/input_reader.h" #include "fesa/io/abaqus/input_reader.h"
#include "fesa/results/result_recovery.h" #include "reference_tolerance_policy.h"
namespace fesa::test { namespace fesa::test {
namespace { namespace {
@@ -48,9 +48,6 @@ constexpr const char* kReactionPath = "/steps/Step-1/frames/0/nodal/reaction";
constexpr const char* kSectionPath = constexpr const char* kSectionPath =
"/steps/Step-1/frames/0/element/section_resultant"; "/steps/Step-1/frames/0/element/section_resultant";
constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11"; constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11";
constexpr double kKinematicFloor = 1.0e-9;
constexpr double kForceMomentFloor = 1.0e-3;
constexpr double kRelativeCoefficient = 1.0e-6;
class ComparisonFailure final : public std::runtime_error { class ComparisonFailure final : public std::runtime_error {
public: public:
@@ -166,6 +163,17 @@ struct ReferenceTable {
std::vector<WideReferenceRow> rows; std::vector<WideReferenceRow> rows;
}; };
struct SectionReferenceRow {
std::string instance_name;
std::int64_t source_element_label;
std::int64_t source_node_label;
std::vector<double> values;
};
struct SectionReferenceTable {
std::vector<SectionReferenceRow> rows;
};
ReferenceTable ReadReferenceCsv( ReferenceTable ReadReferenceCsv(
const std::filesystem::path& path, const std::filesystem::path& path,
const std::vector<std::string>& expected_header) { const std::vector<std::string>& expected_header) {
@@ -223,6 +231,70 @@ ReferenceTable ReadReferenceCsv(
return table; return table;
} }
SectionReferenceTable ReadSectionReferenceCsv(
const std::filesystem::path& path) {
std::ifstream stream{path};
if (!stream) {
Fail("needs-reference-artifacts", "An approved reference CSV is missing.");
}
std::string line;
if (!std::getline(stream, line)) {
Fail("schema-mismatch", "An approved reference CSV is empty.");
}
if (!line.empty() && line.back() == '\r') {
line.pop_back();
}
const std::vector<std::string> expected_header = {
"Frame", "Part Instance Name",
"Element Label", "Node Label",
"SF-SF1", "SM-SM1",
"SM-SM2", "SM-SM3"};
if (SplitCsvLine(line) != expected_header) {
Fail("schema-mismatch", "An approved section CSV header is not exact.");
}
SectionReferenceTable table;
while (std::getline(stream, line)) {
if (!line.empty() && line.back() == '\r') {
line.pop_back();
}
if (line.empty()) {
Fail("schema-mismatch", "Blank reference CSV rows are not allowed.");
}
const auto fields = SplitCsvLine(line);
if (fields.size() != expected_header.size() ||
CollapseWhitespace(fields[0U]) != kFrameText || fields[1U].empty()) {
Fail("schema-mismatch",
"A reference section row has invalid schema or frame identity.");
}
SectionReferenceRow row{};
row.instance_name = fields[1U];
row.source_element_label = ParsePositiveLabel(fields[2U]);
row.source_node_label = ParsePositiveLabel(fields[3U]);
row.values.reserve(fields.size() - 4U);
for (std::size_t field = 4U; field < fields.size(); ++field) {
row.values.push_back(ParseFiniteDouble(fields[field]));
}
const auto duplicate = std::find_if(
table.rows.begin(), table.rows.end(),
[&](const SectionReferenceRow& existing) {
return AsciiLower(existing.instance_name) ==
AsciiLower(row.instance_name) &&
existing.source_element_label == row.source_element_label &&
existing.source_node_label == row.source_node_label;
});
if (duplicate != table.rows.end()) {
Fail("schema-mismatch",
"A reference section row identity is duplicated.");
}
table.rows.push_back(std::move(row));
}
if (table.rows.empty()) {
Fail("schema-mismatch", "An approved reference CSV has no data rows.");
}
return table;
}
void RequireExactArtifactInventory( void RequireExactArtifactInventory(
const std::filesystem::path& legacy_directory) { const std::filesystem::path& legacy_directory) {
std::error_code error; std::error_code error;
@@ -894,76 +966,39 @@ std::vector<const WideReferenceRow*> OrderedRows(
return ordered; return ordered;
} }
double TableScale(const ReferenceTable& table, const std::size_t value_index) { std::vector<const SectionReferenceRow*> OrderedSectionRows(
double scale = 0.0; const SectionReferenceTable& table, const HdfProjection& hdf) {
for (const auto& row : table.rows) { const std::size_t endpoint_count = hdf.elements.size() * 2U;
if (value_index >= row.values.size()) { if (table.rows.size() != endpoint_count) {
Fail("schema-mismatch", "A reference row has the wrong component arity."); Fail("schema-mismatch", "The FESA/reference section row sets differ.");
}
scale = (std::max)(scale, std::abs(row.values[value_index]));
} }
return scale; std::vector<const SectionReferenceRow*> ordered;
} ordered.reserve(endpoint_count);
for (const auto& element : hdf.elements) {
std::vector<NodeStationResultRow> NormalizeStations(
const Domain& domain, const HdfProjection& hdf,
const ReferenceTable& section_table) {
auto model_result = AnalysisModel::Create(domain);
if (!model_result.HasValue()) {
Fail("schema-mismatch",
"The approved input cannot create an analysis view.");
}
const AnalysisModel model = std::move(model_result.Value());
const std::array<double, 4> tolerances = {
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 0U),
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 3U),
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 1U),
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 2U)};
std::vector<EndpointResultRow> endpoints;
endpoints.reserve(hdf.elements.size() * 2U);
for (std::size_t element = 0U; element < hdf.elements.size(); ++element) {
for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) { for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) {
const auto node = static_cast<std::size_t>( const std::size_t node_index =
hdf.elements[element].node_internal_ids[endpoint]); static_cast<std::size_t>(element.node_internal_ids[endpoint]);
std::array<double, 4> values{}; if (node_index >= hdf.nodes.size()) {
for (std::size_t component = 0U; component < values.size(); ++component) { Fail("schema-mismatch", "An HDF5 section endpoint references no node.");
values[component] =
hdf.section_resultants[(element * 2U + endpoint) * 4U + component];
} }
endpoints.push_back({static_cast<EntityIndex>(element), const auto& node = hdf.nodes[node_index];
static_cast<int>(endpoint), const auto found = std::find_if(
domain.Nodes()[node].source_id, table.rows.begin(), table.rows.end(),
{}, [&](const SectionReferenceRow& row) {
values}); return AsciiLower(row.instance_name) ==
AsciiLower(element.instance_name) &&
row.source_element_label == element.source_element_label &&
row.source_node_label == node.source_node_label;
});
if (found == table.rows.end() ||
found->instance_name != element.instance_name) {
Fail("schema-mismatch",
"A reference section endpoint identity does not match HDF5.");
}
ordered.push_back(&*found);
} }
} }
auto normalized = ResultRecovery::NormalizeSectionResultantsToNodeStations( return ordered;
model, endpoints, tolerances);
if (!normalized.HasValue()) {
const auto& diagnostics = normalized.GetStatus().Diagnostics();
const std::string code =
diagnostics.empty() ? std::string{} : diagnostics[0U].code;
if (code == "node-station-tolerance-failure") {
Fail("tolerance-failure",
"Interior endpoint section resultants disagree.");
}
Fail("schema-mismatch",
"A node station is not eligible for legacy projection.");
}
return std::move(normalized.Value());
}
const NodeStationResultRow& FindStation(
const std::vector<NodeStationResultRow>& stations, const HdfNode& node) {
const auto found = std::find_if(
stations.begin(), stations.end(), [&](const NodeStationResultRow& row) {
return row.node.instance_name == node.instance_name &&
row.node.source_label == node.source_node_label;
});
if (found == stations.end()) {
Fail("schema-mismatch", "A projected HDF5 node station is missing.");
}
return *found;
} }
CanonicalComparisonRow CanonicalRow(const HdfNode& node, CanonicalComparisonRow CanonicalRow(const HdfNode& node,
@@ -977,6 +1012,8 @@ CanonicalComparisonRow CanonicalRow(const HdfNode& node,
kFrameIndex, kFrameIndex,
node.instance_name, node.instance_name,
node.source_node_label, node.source_node_label,
0,
-1,
quantity, quantity,
std::move(component), std::move(component),
value, value,
@@ -1003,93 +1040,122 @@ void AppendNodalRows(ComparisonReport& report, const HdfProjection& hdf,
CanonicalRow(hdf.nodes[node], quantity, components[component], CanonicalRow(hdf.nodes[node], quantity, components[component],
reference[node]->values[component], units[component], reference[node]->values[component], units[component],
"global-cartesian", dataset_path); "global-cartesian", dataset_path);
report.rows.push_back( report.rows.push_back({std::move(fesa), std::move(abaqus)});
{std::move(fesa), std::move(abaqus), 0.0, 0.0, false});
} }
} }
} }
void AppendSectionRows(ComparisonReport& report, const HdfProjection& hdf, void AppendSectionRows(
const std::vector<const WideReferenceRow*>& reference, ComparisonReport& report, const HdfProjection& hdf,
const std::vector<NodeStationResultRow>& stations) { const std::vector<const SectionReferenceRow*>& reference) {
const std::array<std::string, 4> components = {"N", "T", "My", "Mz"}; const std::array<std::string, 4> components = {"N", "T", "My", "Mz"};
const std::array<std::string, 4> units = {"force", "force*length", const std::array<std::string, 4> units = {"force", "force*length",
"force*length", "force*length"}; "force*length", "force*length"};
const std::array<std::size_t, 4> reference_columns = {0U, 3U, 1U, 2U}; const std::array<std::size_t, 4> reference_columns = {0U, 3U, 1U, 2U};
for (std::size_t node = 0U; node < hdf.nodes.size(); ++node) { std::size_t row = 0U;
const auto& station = FindStation(stations, hdf.nodes[node]); for (std::size_t element = 0U; element < hdf.elements.size(); ++element) {
for (std::size_t component = 0U; component < components.size(); for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint, ++row) {
++component) { const auto& element_row = hdf.elements[element];
auto fesa = CanonicalRow( const std::size_t node_index =
hdf.nodes[node], ComparisonQuantity::kSectionResultant, static_cast<std::size_t>(element_row.node_internal_ids[endpoint]);
components[component], station.section_resultant[component], if (node_index >= hdf.nodes.size()) {
units[component], "beam-local", kSectionPath); Fail("schema-mismatch", "An HDF5 section endpoint references no node.");
auto abaqus = }
CanonicalRow(hdf.nodes[node], ComparisonQuantity::kSectionResultant, const auto& node = hdf.nodes[node_index];
components[component], const auto canonical = [&](const double value, std::string component,
reference[node]->values[reference_columns[component]], std::string unit) {
units[component], "beam-local", kSectionPath); return CanonicalComparisonRow{kModelId,
report.rows.push_back( kStepName,
{std::move(fesa), std::move(abaqus), 0.0, 0.0, false}); kFrameIndex,
element_row.instance_name,
node.source_node_label,
element_row.source_element_label,
static_cast<int>(endpoint),
ComparisonQuantity::kSectionResultant,
std::move(component),
value,
std::move(unit),
"beam-local",
kSectionPath};
};
for (std::size_t component = 0U; component < components.size();
++component) {
auto fesa = canonical(
hdf.section_resultants[(element * 2U + endpoint) * 4U + component],
components[component], units[component]);
auto abaqus =
canonical(reference[row]->values[reference_columns[component]],
components[component], units[component]);
report.rows.push_back({std::move(fesa), std::move(abaqus)});
}
} }
} }
} }
double AbsoluteFloor(const ComparisonQuantity quantity, const std::string&) { const char* BranchName(const ReferenceToleranceBranch branch) {
return quantity == ComparisonQuantity::kDisplacement ? kKinematicFloor switch (branch) {
: kForceMomentFloor; case ReferenceToleranceBranch::kNearZero:
return "near-zero";
case ReferenceToleranceBranch::kRelative:
return "relative";
case ReferenceToleranceBranch::kZeroScaleExact:
return "zero-scale-exact";
}
return "unknown";
} }
void EvaluateGroup(ComparisonReport& report, const ComparisonQuantity quantity, void EvaluateFamily(ComparisonReport& report, const ComparisonQuantity quantity,
const std::string& component) { const std::string& identity,
const std::vector<std::string>& components,
const bool blocking) {
std::vector<std::size_t> row_indices; std::vector<std::size_t> row_indices;
for (std::size_t index = 0U; index < report.rows.size(); ++index) { for (std::size_t index = 0U; index < report.rows.size(); ++index) {
if (report.rows[index].reference.quantity == quantity && if (report.rows[index].reference.quantity == quantity &&
report.rows[index].reference.component == component) { std::find(components.begin(), components.end(),
report.rows[index].reference.component) != components.end()) {
row_indices.push_back(index); row_indices.push_back(index);
} }
} }
if (row_indices.empty()) { if (row_indices.empty()) {
Fail("schema-mismatch", "A canonical comparison component has no rows."); Fail("schema-mismatch", "A canonical comparison family has no rows.");
} }
double reference_scale = 0.0; ReferenceToleranceFamily family{};
family.identity = identity;
family.components = components;
family.blocking = blocking;
family.rows.reserve(row_indices.size());
for (const std::size_t index : row_indices) { for (const std::size_t index : row_indices) {
reference_scale = (std::max)(reference_scale, family.rows.push_back(
std::abs(report.rows[index].reference.value)); {report.rows[index].fesa.value, report.rows[index].reference.value});
} }
const double tolerance = AbsoluteFloor(quantity, component) + const auto evaluation =
kRelativeCoefficient * reference_scale; ReferenceTolerancePolicy::Evaluate({std::move(family)});
double maximum_absolute = -1.0; if (!evaluation.valid || evaluation.families.size() != 1U) {
double maximum_normalized = 0.0; Fail("schema-mismatch", "A canonical comparison family is nonfinite.");
std::size_t worst_row = row_indices.front();
long double squared_error = 0.0L;
for (const std::size_t index : row_indices) {
auto& row = report.rows[index];
row.absolute_error = std::abs(row.fesa.value - row.reference.value);
if (!std::isfinite(row.absolute_error)) {
Fail("schema-mismatch", "A canonical row error is nonfinite.");
}
row.tolerance = tolerance;
row.passed = row.absolute_error <= tolerance;
report.passed = report.passed && row.passed;
const double normalized = row.absolute_error / tolerance;
if (row.absolute_error > maximum_absolute) {
maximum_absolute = row.absolute_error;
worst_row = index;
}
maximum_normalized = (std::max)(maximum_normalized, normalized);
const long double error = static_cast<long double>(row.absolute_error);
squared_error += error * error;
} }
const double norm_error = std::sqrt(static_cast<double>(squared_error)); const auto& policy = evaluation.families.front();
const double rms_error = std::sqrt(static_cast<double>( for (std::size_t local = 0U; local < row_indices.size(); ++local) {
squared_error / static_cast<long double>(row_indices.size()))); auto& row = report.rows[row_indices[local]];
if (!std::isfinite(norm_error) || !std::isfinite(rms_error)) { const auto& decision = policy.rows[local];
Fail("schema-mismatch", "A component aggregate error is nonfinite."); row.absolute_error = decision.absolute_error;
row.tolerance = decision.threshold;
row.reference_scale = policy.reference_scale;
row.near_zero_band = policy.near_zero_band;
row.relative_error = decision.relative_error;
row.relative_error_applicable = decision.relative_error_applicable;
row.tolerance_branch = BranchName(decision.branch);
row.passed = decision.passed;
}
const std::size_t worst_row = row_indices[policy.worst_row];
report.metrics.push_back({quantity, policy.identity, policy.components,
policy.reference_scale, policy.near_zero_band,
policy.near_zero_count,
policy.maximum_absolute_error, policy.relative_rms,
worst_row, policy.rms_passed, policy.passed,
policy.diagnostic_code, policy.row_count});
if (blocking && !policy.passed) {
report.passed = false;
} }
report.metrics.push_back({quantity, component, reference_scale,
maximum_absolute, maximum_normalized, rms_error,
norm_error, worst_row});
} }
PhysicsEvidence MakePhysicsEvidence(const Domain& domain, PhysicsEvidence MakePhysicsEvidence(const Domain& domain,
@@ -1214,7 +1280,8 @@ void WriteCanonicalRow(std::ostream& stream,
stream << ",\"frame_index\":" << row.frame_index << ",\"instance_name\":"; stream << ",\"frame_index\":" << row.frame_index << ",\"instance_name\":";
WriteJsonString(stream, row.instance_name); WriteJsonString(stream, row.instance_name);
stream << ",\"source_node_label\":" << row.source_node_label stream << ",\"source_node_label\":" << row.source_node_label
<< ",\"quantity\":"; << ",\"source_element_label\":" << row.source_element_label
<< ",\"endpoint_index\":" << row.endpoint_index << ",\"quantity\":";
WriteJsonString(stream, QuantityName(row.quantity)); WriteJsonString(stream, QuantityName(row.quantity));
stream << ",\"component\":"; stream << ",\"component\":";
WriteJsonString(stream, row.component); WriteJsonString(stream, row.component);
@@ -1231,6 +1298,18 @@ void WriteArray(std::ostream& stream, const std::array<double, 3>& values) {
stream << '[' << values[0U] << ',' << values[1U] << ',' << values[2U] << ']'; stream << '[' << values[0U] << ',' << values[1U] << ',' << values[2U] << ']';
} }
void WriteStrings(std::ostream& stream,
const std::vector<std::string>& values) {
stream << '[';
for (std::size_t index = 0U; index < values.size(); ++index) {
if (index != 0U) {
stream << ',';
}
WriteJsonString(stream, values[index]);
}
stream << ']';
}
bool FiniteReport(const ComparisonReport& report) { bool FiniteReport(const ComparisonReport& report) {
const auto finite_array = [](const std::array<double, 3>& values) { const auto finite_array = [](const std::array<double, 3>& values) {
return std::all_of(values.begin(), values.end(), return std::all_of(values.begin(), values.end(),
@@ -1245,17 +1324,19 @@ bool FiniteReport(const ComparisonReport& report) {
} }
for (const auto& row : report.rows) { for (const auto& row : report.rows) {
if (!std::isfinite(row.fesa.value) || !std::isfinite(row.reference.value) || if (!std::isfinite(row.fesa.value) || !std::isfinite(row.reference.value) ||
!std::isfinite(row.absolute_error) || !std::isfinite(row.tolerance)) { !std::isfinite(row.absolute_error) || !std::isfinite(row.tolerance) ||
!std::isfinite(row.reference_scale) ||
!std::isfinite(row.near_zero_band) ||
!std::isfinite(row.relative_error)) {
return false; return false;
} }
} }
return std::all_of(report.metrics.begin(), report.metrics.end(), return std::all_of(report.metrics.begin(), report.metrics.end(),
[](const ComponentMetrics& metric) { [](const ComponentMetrics& metric) {
return std::isfinite(metric.reference_scale) && return std::isfinite(metric.reference_scale) &&
std::isfinite(metric.near_zero_band) &&
std::isfinite(metric.maximum_absolute_error) && std::isfinite(metric.maximum_absolute_error) &&
std::isfinite(metric.maximum_normalized_error) && std::isfinite(metric.relative_rms);
std::isfinite(metric.rms_error) &&
std::isfinite(metric.norm_error);
}); });
} }
@@ -1276,18 +1357,12 @@ Result<ComparisonReport> ReferenceComparison::Compare(
ReadReferenceCsv(legacy_reference_directory / kReactionName, ReadReferenceCsv(legacy_reference_directory / kReactionName,
{"Frame", "Part Instance Name", "Node Label", "RF-RF1", {"Frame", "Part Instance Name", "Node Label", "RF-RF1",
"RF-RF2", "RF-RF3", "RM-RM1", "RM-RM2", "RM-RM3"}); "RF-RF2", "RF-RF3", "RM-RM1", "RM-RM2", "RM-RM3"});
const ReferenceTable section = const SectionReferenceTable section =
ReadReferenceCsv(legacy_reference_directory / kSectionName, ReadSectionReferenceCsv(legacy_reference_directory / kSectionName);
{"Frame", "Part Instance Name", "Node Label", "SF-SF1",
"SM-SM1", "SM-SM2", "SM-SM3"});
HdfProjection hdf = ReadHdfProjection(results_hdf5, input, domain); HdfProjection hdf = ReadHdfProjection(results_hdf5, input, domain);
const auto displacement_rows = OrderedRows(displacement, hdf.nodes); const auto displacement_rows = OrderedRows(displacement, hdf.nodes);
const auto reaction_rows = OrderedRows(reaction, hdf.nodes); const auto reaction_rows = OrderedRows(reaction, hdf.nodes);
const auto section_rows = OrderedRows(section, hdf.nodes); const auto section_rows = OrderedSectionRows(section, hdf);
const auto stations = NormalizeStations(domain, hdf, section);
if (stations.size() != hdf.nodes.size()) {
Fail("schema-mismatch", "The HDF5 node-station row set is incomplete.");
}
ComparisonReport report{}; ComparisonReport report{};
report.passed = true; report.passed = true;
@@ -1301,19 +1376,20 @@ Result<ComparisonReport> ReferenceComparison::Compare(
{"force", "force", "force", "force*length", "force*length", {"force", "force", "force", "force*length", "force*length",
"force*length"}, "force*length"},
hdf.reaction, kReactionPath); hdf.reaction, kReactionPath);
AppendSectionRows(report, hdf, section_rows, stations); AppendSectionRows(report, hdf, section_rows);
for (const std::string& component : EvaluateFamily(report, ComparisonQuantity::kDisplacement,
{"UX", "UY", "UZ", "URX", "URY", "URZ"}) { "displacement-translation", {"UX", "UY", "UZ"}, true);
EvaluateGroup(report, ComparisonQuantity::kDisplacement, component); EvaluateFamily(report, ComparisonQuantity::kDisplacement,
} "displacement-rotation", {"URX", "URY", "URZ"}, true);
for (const std::string& component : EvaluateFamily(report, ComparisonQuantity::kReaction, "reaction-force",
{"RF1", "RF2", "RF3", "RM1", "RM2", "RM3"}) { {"RF1", "RF2", "RF3"}, true);
EvaluateGroup(report, ComparisonQuantity::kReaction, component); EvaluateFamily(report, ComparisonQuantity::kReaction, "reaction-moment",
} {"RM1", "RM2", "RM3"}, true);
for (const std::string& component : {"N", "T", "My", "Mz"}) { EvaluateFamily(report, ComparisonQuantity::kSectionResultant,
EvaluateGroup(report, ComparisonQuantity::kSectionResultant, component); "section-force", {"N"}, true);
} EvaluateFamily(report, ComparisonQuantity::kSectionResultant,
"section-moment", {"T", "My", "Mz"}, true);
report.physics_evidence = MakePhysicsEvidence(domain, hdf); report.physics_evidence = MakePhysicsEvidence(domain, hdf);
report.stress_comparison_applicable = false; report.stress_comparison_applicable = false;
report.stress_comparison_reason = report.stress_comparison_reason =
@@ -1366,8 +1442,15 @@ Status ReferenceComparison::WriteDeterministicJson(
stream << ",\"reference\":"; stream << ",\"reference\":";
WriteCanonicalRow(stream, row.reference); WriteCanonicalRow(stream, row.reference);
stream << ",\"absolute_error\":" << row.absolute_error stream << ",\"absolute_error\":" << row.absolute_error
<< ",\"tolerance\":" << row.tolerance << ",\"threshold\":" << row.tolerance
<< ",\"passed\":" << (row.passed ? "true" : "false") << '}'; << ",\"reference_scale\":" << row.reference_scale
<< ",\"near_zero_band\":" << row.near_zero_band
<< ",\"relative_error\":" << row.relative_error
<< ",\"relative_error_applicable\":"
<< (row.relative_error_applicable ? "true" : "false")
<< ",\"tolerance_branch\":";
WriteJsonString(stream, row.tolerance_branch);
stream << ",\"passed\":" << (row.passed ? "true" : "false") << '}';
} }
stream << "],\"metrics\":["; stream << "],\"metrics\":[";
for (std::size_t index = 0U; index < report.metrics.size(); ++index) { for (std::size_t index = 0U; index < report.metrics.size(); ++index) {
@@ -1377,15 +1460,22 @@ Status ReferenceComparison::WriteDeterministicJson(
const auto& metric = report.metrics[index]; const auto& metric = report.metrics[index];
stream << "{\"quantity\":"; stream << "{\"quantity\":";
WriteJsonString(stream, QuantityName(metric.quantity)); WriteJsonString(stream, QuantityName(metric.quantity));
stream << ",\"component\":"; stream << ",\"family_identity\":";
WriteJsonString(stream, metric.component); WriteJsonString(stream, metric.family_identity);
stream << ",\"components\":";
WriteStrings(stream, metric.components);
stream << ",\"reference_scale\":" << metric.reference_scale stream << ",\"reference_scale\":" << metric.reference_scale
<< ",\"near_zero_band\":" << metric.near_zero_band
<< ",\"near_zero_count\":" << metric.near_zero_count
<< ",\"row_count\":" << metric.row_count
<< ",\"maximum_absolute_error\":" << metric.maximum_absolute_error << ",\"maximum_absolute_error\":" << metric.maximum_absolute_error
<< ",\"maximum_normalized_error\":" << ",\"relative_rms\":" << metric.relative_rms
<< metric.maximum_normalized_error << ",\"worst_row\":" << metric.worst_row
<< ",\"rms_error\":" << metric.rms_error << ",\"rms_passed\":" << (metric.rms_passed ? "true" : "false")
<< ",\"norm_error\":" << metric.norm_error << ",\"passed\":" << (metric.passed ? "true" : "false")
<< ",\"worst_row\":" << metric.worst_row << '}'; << ",\"diagnostic_code\":";
WriteJsonString(stream, metric.diagnostic_code);
stream << '}';
} }
stream << "],\"physics_evidence\":{\"free_residual_norm\":" stream << "],\"physics_evidence\":{\"free_residual_norm\":"
<< report.physics_evidence.free_residual_norm << ",\"applied_force\":"; << report.physics_evidence.free_residual_norm << ",\"applied_force\":";
+16 -4
View File
@@ -20,6 +20,8 @@ struct CanonicalComparisonRow {
std::size_t frame_index; std::size_t frame_index;
std::string instance_name; std::string instance_name;
std::int64_t source_node_label; std::int64_t source_node_label;
std::int64_t source_element_label;
int endpoint_index;
ComparisonQuantity quantity; ComparisonQuantity quantity;
std::string component; std::string component;
double value; double value;
@@ -33,18 +35,28 @@ struct RowDecision {
CanonicalComparisonRow reference; CanonicalComparisonRow reference;
double absolute_error; double absolute_error;
double tolerance; double tolerance;
double reference_scale;
double near_zero_band;
double relative_error;
bool relative_error_applicable;
std::string tolerance_branch;
bool passed; bool passed;
}; };
struct ComponentMetrics { struct ComponentMetrics {
ComparisonQuantity quantity; ComparisonQuantity quantity;
std::string component; std::string family_identity;
std::vector<std::string> components;
double reference_scale; double reference_scale;
double near_zero_band;
std::size_t near_zero_count;
double maximum_absolute_error; double maximum_absolute_error;
double maximum_normalized_error; double relative_rms;
double rms_error;
double norm_error;
std::size_t worst_row; std::size_t worst_row;
bool rms_passed;
bool passed;
std::string diagnostic_code;
std::size_t row_count{};
}; };
struct PhysicsEvidence { struct PhysicsEvidence {
+140 -68
View File
@@ -41,8 +41,8 @@ constexpr const char* kSectionName = "cantilever beam elemental forces.csv";
constexpr const char* kInstanceName = "PART-1_1-1"; constexpr const char* kInstanceName = "PART-1_1-1";
constexpr std::size_t kNodeCount = 11U; constexpr std::size_t kNodeCount = 11U;
constexpr std::size_t kElementCount = 10U; constexpr std::size_t kElementCount = 10U;
constexpr std::size_t kExpectedRowCount = 176U; constexpr std::size_t kExpectedRowCount = 212U;
constexpr std::size_t kExpectedMetricCount = 16U; constexpr std::size_t kExpectedMetricCount = 6U;
using NodalValues = std::array<std::array<double, 6>, kNodeCount>; using NodalValues = std::array<std::array<double, 6>, kNodeCount>;
using EndpointValues = using EndpointValues =
@@ -315,14 +315,29 @@ const fesa::test::RowDecision* FindRow(
const fesa::test::ComponentMetrics* FindMetric( const fesa::test::ComponentMetrics* FindMetric(
const fesa::test::ComparisonReport& report, const fesa::test::ComparisonReport& report,
const fesa::test::ComparisonQuantity quantity, const std::string& family_identity) {
const auto found =
std::find_if(report.metrics.begin(), report.metrics.end(),
[&](const fesa::test::ComponentMetrics& metric) {
return metric.family_identity == family_identity;
});
return found == report.metrics.end() ? nullptr : &*found;
}
const fesa::test::RowDecision* FindSectionRow(
const fesa::test::ComparisonReport& report,
const std::int64_t source_element_label, const int endpoint_index,
const std::string& component) { const std::string& component) {
const auto found = std::find_if( const auto found = std::find_if(
report.metrics.begin(), report.metrics.end(), report.rows.begin(), report.rows.end(),
[&](const fesa::test::ComponentMetrics& metric) { [&](const fesa::test::RowDecision& row) {
return metric.quantity == quantity && metric.component == component; return row.reference.quantity ==
fesa::test::ComparisonQuantity::kSectionResultant &&
row.reference.source_element_label == source_element_label &&
row.reference.endpoint_index == endpoint_index &&
row.reference.component == component;
}); });
return found == report.metrics.end() ? nullptr : &*found; return found == report.rows.end() ? nullptr : &*found;
} }
void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) { void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) {
@@ -381,13 +396,25 @@ void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) {
const std::array<std::string, 4> section_components = {"N", "T", "My", "Mz"}; const std::array<std::string, 4> section_components = {"N", "T", "My", "Mz"};
const std::array<std::string, 4> section_units = { const std::array<std::string, 4> section_units = {
"force", "force*length", "force*length", "force*length"}; "force", "force*length", "force*length", "force*length"};
for (std::size_t node = 0U; node < kNodeCount; ++node) { for (std::size_t element = 0U; element < kElementCount; ++element) {
for (std::size_t component = 0U; component < section_components.size(); for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) {
++component) { for (std::size_t component = 0U; component < section_components.size();
expect_row(fesa::test::ComparisonQuantity::kSectionResultant, node, ++component) {
section_components[component], section_units[component], ASSERT_LT(row_index, report.rows.size());
"beam-local", const auto& row = report.rows[row_index++];
"/steps/Step-1/frames/0/element/section_resultant"); for (const auto* side : {&row.fesa, &row.reference}) {
EXPECT_EQ(side->source_element_label,
static_cast<std::int64_t>(element + 1U));
EXPECT_EQ(side->endpoint_index, static_cast<int>(endpoint));
EXPECT_EQ(side->source_node_label,
static_cast<std::int64_t>(element + endpoint + 1U));
EXPECT_EQ(side->component, section_components[component]);
EXPECT_EQ(side->unit_dimension, section_units[component]);
EXPECT_EQ(side->coordinate_system, "beam-local");
EXPECT_EQ(side->hdf5_dataset_path,
"/steps/Step-1/frames/0/element/section_resultant");
}
}
} }
} }
EXPECT_EQ(row_index, report.rows.size()); EXPECT_EQ(row_index, report.rows.size());
@@ -475,14 +502,46 @@ TEST(ReferenceComparisonContract,
fixture.Results(), fixture.Legacy()), fixture.Results(), fixture.Legacy()),
"schema-mismatch"); "schema-mismatch");
} }
{
ContractFixture fixture{"section-missing-endpoint", mismatched_values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_EQ(lines.size(), kElementCount * 2U + 1U);
lines.pop_back();
WriteLines(fixture.Legacy() / kSectionName, lines);
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
{
ContractFixture fixture{"section-duplicate-endpoint", mismatched_values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_EQ(lines.size(), kElementCount * 2U + 1U);
lines.push_back(lines[1U]);
WriteLines(fixture.Legacy() / kSectionName, lines);
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
{
ContractFixture fixture{"section-mismatched-endpoint", mismatched_values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_GT(lines.size(), 2U);
ReplaceFirst(lines[1U], ",1,1,", ",1,3,");
WriteLines(fixture.Legacy() / kSectionName, lines);
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
} }
TEST(ReferenceComparisonContract, TEST(ReferenceComparisonContract,
AppliesAbaqusOnlyComponentScaleWithoutClampOrDrop) { AppliesSharedFamilyScaleAndNearZeroBranchesWithoutAnAbsoluteGate) {
auto values = ReferenceValues(); auto values = ReferenceValues();
values.displacement[1U][0U] = 0.999e-9; const double translation_scale = 1.90476272e-2;
values.displacement[2U][0U] = 1.001e-9; const double near_zero_band = 0.01 * translation_scale;
values.section_resultants[0U][0U][2U] = 1.0e7 + 9.0; values.displacement[1U][0U] = 0.999 * near_zero_band;
values.displacement[2U][0U] = 1.001 * near_zero_band;
values.section_resultants[0U][0U][2U] = 1.0e7 + 4.999e5;
values.section_resultants[9U][1U][2U] = 0.0; values.section_resultants[9U][1U][2U] = 0.0;
ContractFixture fixture{"tolerance", values}; ContractFixture fixture{"tolerance", values};
@@ -505,28 +564,28 @@ TEST(ReferenceComparisonContract,
ASSERT_NE(deliberate_failure, nullptr); ASSERT_NE(deliberate_failure, nullptr);
EXPECT_DOUBLE_EQ(zero->reference.value, 0.0); EXPECT_DOUBLE_EQ(zero->reference.value, 0.0);
EXPECT_DOUBLE_EQ(zero->fesa.value, 0.0); EXPECT_DOUBLE_EQ(zero->fesa.value, 0.0);
EXPECT_DOUBLE_EQ(zero->tolerance, 1.0e-9); EXPECT_DOUBLE_EQ(zero->tolerance, near_zero_band);
EXPECT_TRUE(zero->passed); EXPECT_TRUE(zero->passed);
EXPECT_TRUE(near_zero->passed); EXPECT_TRUE(near_zero->passed);
EXPECT_FALSE(deliberate_failure->passed); EXPECT_FALSE(deliberate_failure->passed);
EXPECT_EQ(near_zero->tolerance_branch, "near-zero");
EXPECT_FALSE(near_zero->relative_error_applicable);
const auto* my_metric = FindMetric( const auto* my_metric = FindMetric(report, "section-moment");
report, fesa::test::ComparisonQuantity::kSectionResultant, "My");
ASSERT_NE(my_metric, nullptr); ASSERT_NE(my_metric, nullptr);
EXPECT_DOUBLE_EQ(my_metric->reference_scale, 1.0e7); EXPECT_DOUBLE_EQ(my_metric->reference_scale, 1.0e7);
const auto* scaled = FindRow( EXPECT_EQ(my_metric->components, (std::vector<std::string>{"T", "My", "Mz"}));
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "My"); const auto* scaled = FindSectionRow(report, 1, 0, "My");
const auto* residue = FindRow( const auto* residue = FindSectionRow(report, 10, 1, "My");
report, fesa::test::ComparisonQuantity::kSectionResultant, 11, "My");
ASSERT_NE(scaled, nullptr); ASSERT_NE(scaled, nullptr);
ASSERT_NE(residue, nullptr); ASSERT_NE(residue, nullptr);
EXPECT_DOUBLE_EQ(scaled->tolerance, 10.001); EXPECT_DOUBLE_EQ(scaled->tolerance, 5.0e5);
EXPECT_DOUBLE_EQ(scaled->absolute_error, 9.0); EXPECT_DOUBLE_EQ(scaled->absolute_error, 4.999e5);
EXPECT_TRUE(scaled->passed); EXPECT_TRUE(scaled->passed);
EXPECT_DOUBLE_EQ(residue->reference.value, -1.56e-2); EXPECT_DOUBLE_EQ(residue->reference.value, -1.5625e-2);
EXPECT_DOUBLE_EQ(residue->fesa.value, 0.0); EXPECT_DOUBLE_EQ(residue->fesa.value, 0.0);
EXPECT_DOUBLE_EQ(residue->absolute_error, 1.56e-2); EXPECT_DOUBLE_EQ(residue->absolute_error, 1.5625e-2);
EXPECT_DOUBLE_EQ(residue->tolerance, 10.001); EXPECT_DOUBLE_EQ(residue->tolerance, 1.0e5);
EXPECT_TRUE(residue->passed); EXPECT_TRUE(residue->passed);
} }
@@ -548,19 +607,18 @@ TEST(ReferenceComparisonContract,
report.rows.begin(), report.rows.end(), report.rows.begin(), report.rows.end(),
[](const fesa::test::RowDecision& row) { return row.passed; })); [](const fesa::test::RowDecision& row) { return row.passed; }));
const auto* metric = const auto* metric = FindMetric(report, "displacement-translation");
FindMetric(report, fesa::test::ComparisonQuantity::kDisplacement, "UX");
const auto* worst = const auto* worst =
FindRow(report, fesa::test::ComparisonQuantity::kDisplacement, 2, "UX"); FindRow(report, fesa::test::ComparisonQuantity::kDisplacement, 2, "UX");
ASSERT_NE(metric, nullptr); ASSERT_NE(metric, nullptr);
ASSERT_NE(worst, nullptr); ASSERT_NE(worst, nullptr);
EXPECT_DOUBLE_EQ(metric->reference_scale, 0.0); EXPECT_EQ(metric->row_count, kNodeCount * 3U);
EXPECT_DOUBLE_EQ(metric->reference_scale, 1.90476272e-2);
EXPECT_DOUBLE_EQ(metric->maximum_absolute_error, 1.0e-9); EXPECT_DOUBLE_EQ(metric->maximum_absolute_error, 1.0e-9);
EXPECT_DOUBLE_EQ(metric->maximum_normalized_error, 1.0); EXPECT_NEAR(metric->relative_rms,
EXPECT_NEAR(metric->rms_error, std::sqrt(1.25 / static_cast<double>(kNodeCount * 3U)) * 1.0e-9 /
std::sqrt(1.25 / static_cast<double>(kNodeCount)) * 1.0e-9, metric->reference_scale,
1.0e-21); 1.0e-21);
EXPECT_NEAR(metric->norm_error, std::sqrt(1.25) * 1.0e-9, 1.0e-21);
EXPECT_EQ(metric->worst_row, EXPECT_EQ(metric->worst_row,
static_cast<std::size_t>(worst - report.rows.data())); static_cast<std::size_t>(worst - report.rows.data()));
@@ -589,7 +647,10 @@ TEST(ReferenceComparisonContract,
const std::string first = ReadBytes(json_a); const std::string first = ReadBytes(json_a);
EXPECT_EQ(first, ReadBytes(json_b)); EXPECT_EQ(first, ReadBytes(json_b));
for (const char* required : for (const char* required :
{"\"rows\"", "\"metrics\"", "\"stress_comparison_applicable\":false", {"\"rows\"", "\"metrics\"", "\"family_identity\"", "\"components\"",
"\"near_zero_band\"", "\"relative_rms\"", "\"row_count\"",
"\"tolerance_branch\"", "\"relative_error_applicable\"",
"\"stress_comparison_applicable\":false",
"\"stress_comparison_reason\"", "\"physics_evidence\"", "\"stress_comparison_reason\"", "\"physics_evidence\"",
"\"free_residual_norm\"", "\"applied_force\"", "\"reaction_force\"", "\"free_residual_norm\"", "\"applied_force\"", "\"reaction_force\"",
"\"applied_moment_about_origin\"", "\"reaction_moment_about_origin\"", "\"applied_moment_about_origin\"", "\"reaction_moment_about_origin\"",
@@ -598,11 +659,10 @@ TEST(ReferenceComparisonContract,
} }
} }
TEST(ReferenceComparisonContract, NormalizesEligibleStationsWithoutAveraging) { TEST(ReferenceComparisonContract,
PreservesSectionEndpointIdentityWithoutStationNormalization) {
auto values = ReferenceValues(); auto values = ReferenceValues();
values.section_resultants[0U][0U] = {2.0e-4, 3.0e-4, 1.0e7, 4.0e-4};
values.section_resultants[0U][1U][2U] = 9.0e6 - 5.0; values.section_resultants[0U][1U][2U] = 9.0e6 - 5.0;
values.section_resultants[1U][0U][2U] = 9.0e6 + 5.0;
ContractFixture fixture{"stations", values}; ContractFixture fixture{"stations", values};
auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(), auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(),
@@ -612,35 +672,47 @@ TEST(ReferenceComparisonContract, NormalizesEligibleStationsWithoutAveraging) {
ASSERT_TRUE(report.passed); ASSERT_TRUE(report.passed);
EXPECT_TRUE(report.physics_evidence.endpoint_consistency_passed); EXPECT_TRUE(report.physics_evidence.endpoint_consistency_passed);
const auto* n = FindRow( const auto* interior = FindSectionRow(report, 1, 1, "My");
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "N");
const auto* t = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "T");
const auto* my = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "My");
const auto* mz = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "Mz");
ASSERT_NE(n, nullptr);
ASSERT_NE(t, nullptr);
ASSERT_NE(my, nullptr);
ASSERT_NE(mz, nullptr);
EXPECT_DOUBLE_EQ(n->fesa.value, 2.0e-4);
EXPECT_DOUBLE_EQ(t->fesa.value, 3.0e-4);
EXPECT_DOUBLE_EQ(my->fesa.value, 1.0e7);
EXPECT_DOUBLE_EQ(mz->fesa.value, 4.0e-4);
const auto* interior = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 2, "My");
ASSERT_NE(interior, nullptr); ASSERT_NE(interior, nullptr);
EXPECT_DOUBLE_EQ(interior->fesa.value, 9.0e6 - 5.0); EXPECT_DOUBLE_EQ(interior->fesa.value, 9.0e6 - 5.0);
EXPECT_DOUBLE_EQ(interior->reference.value, 9.0e6); EXPECT_DOUBLE_EQ(interior->reference.value, 9.0e6);
EXPECT_DOUBLE_EQ(interior->absolute_error, 5.0); EXPECT_DOUBLE_EQ(interior->absolute_error, 5.0);
const auto* adjacent = FindSectionRow(report, 2, 0, "My");
auto mismatch_values = ReferenceValues(); ASSERT_NE(adjacent, nullptr);
mismatch_values.section_resultants[0U][1U][2U] = 9.0e6 - 6.0; EXPECT_DOUBLE_EQ(adjacent->fesa.value, 9.0e6);
mismatch_values.section_resultants[1U][0U][2U] = 9.0e6 + 6.0; EXPECT_DOUBLE_EQ(adjacent->reference.value, 9.0e6);
ContractFixture mismatch{"station-mismatch", mismatch_values}; }
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(mismatch.Results(),
mismatch.Legacy()), TEST(ReferenceComparisonContract,
"tolerance-failure"); MapsEachSectionCsvComponentToItsDirectEndpointResultant) {
auto values = ReferenceValues();
values.section_resultants[0U][0U] = {11.0, 22.0, 33.0, 44.0};
ContractFixture fixture{"section-component-mapping", values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_EQ(lines.size(), kElementCount * 2U + 1U);
lines[1U] = "Increment 1: Step Time = 1.000,PART-1_1-1,1,1,11,33,44,22";
WriteLines(fixture.Legacy() / kSectionName, lines);
auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(),
fixture.Legacy());
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
ASSERT_TRUE(report.passed);
const auto* n = FindSectionRow(report, 1, 0, "N");
const auto* t = FindSectionRow(report, 1, 0, "T");
const auto* my = FindSectionRow(report, 1, 0, "My");
const auto* mz = FindSectionRow(report, 1, 0, "Mz");
ASSERT_NE(n, nullptr);
ASSERT_NE(t, nullptr);
ASSERT_NE(my, nullptr);
ASSERT_NE(mz, nullptr);
EXPECT_DOUBLE_EQ(n->reference.value, 11.0);
EXPECT_DOUBLE_EQ(t->reference.value, 22.0);
EXPECT_DOUBLE_EQ(my->reference.value, 33.0);
EXPECT_DOUBLE_EQ(mz->reference.value, 44.0);
EXPECT_DOUBLE_EQ(n->fesa.value, 11.0);
EXPECT_DOUBLE_EQ(t->fesa.value, 22.0);
EXPECT_DOUBLE_EQ(my->fesa.value, 33.0);
EXPECT_DOUBLE_EQ(mz->fesa.value, 44.0);
} }
@@ -0,0 +1,115 @@
#include "reference_tolerance_policy.h"
#include <algorithm>
#include <cmath>
#include <limits>
namespace fesa::test {
namespace {
bool IsFiniteFamily(const ReferenceToleranceFamily& family) {
if (family.identity.empty() || family.components.empty() ||
family.rows.empty()) {
return false;
}
return std::all_of(family.rows.begin(), family.rows.end(),
[](const ReferenceToleranceRow& row) {
return std::isfinite(row.fesa_value) &&
std::isfinite(row.reference_value);
});
}
}
ReferenceToleranceEvaluation ReferenceTolerancePolicy::Evaluate(
const std::vector<ReferenceToleranceFamily>& families) {
ReferenceToleranceEvaluation evaluation{true, {}, true, {}};
evaluation.families.reserve(families.size());
for (const auto& family : families) {
if (!IsFiniteFamily(family)) {
return {false, "nonfinite-comparison-value", false, {}};
}
ReferenceToleranceFamilyDecision decision{};
decision.identity = family.identity;
decision.components = family.components;
decision.blocking = family.blocking;
decision.row_count = family.rows.size();
decision.rows.reserve(family.rows.size());
for (const auto& row : family.rows) {
decision.reference_scale =
(std::max)(decision.reference_scale, std::abs(row.reference_value));
}
decision.near_zero_band = kNearZeroRatio * decision.reference_scale;
double error_norm = 0.0;
for (std::size_t index = 0U; index < family.rows.size(); ++index) {
const auto& row = family.rows[index];
const double absolute_error =
std::abs(row.fesa_value - row.reference_value);
if (!std::isfinite(absolute_error)) {
return {false, "nonfinite-comparison-value", false, {}};
}
error_norm = std::hypot(error_norm, absolute_error);
if (!std::isfinite(error_norm)) {
return {false, "nonfinite-comparison-value", false, {}};
}
if (absolute_error > decision.maximum_absolute_error) {
decision.maximum_absolute_error = absolute_error;
decision.worst_row = index;
}
ReferenceToleranceRowDecision row_decision{};
row_decision.absolute_error = absolute_error;
if (decision.reference_scale == 0.0) {
++decision.near_zero_count;
row_decision.threshold = 0.0;
row_decision.branch = ReferenceToleranceBranch::kZeroScaleExact;
row_decision.passed = absolute_error == 0.0;
} else if (std::abs(row.reference_value) <= decision.near_zero_band) {
++decision.near_zero_count;
row_decision.threshold = decision.near_zero_band;
row_decision.branch = ReferenceToleranceBranch::kNearZero;
row_decision.passed = absolute_error <= row_decision.threshold;
} else {
row_decision.threshold =
kRelativeTolerance * std::abs(row.reference_value);
row_decision.relative_error =
absolute_error / std::abs(row.reference_value);
row_decision.relative_error_applicable = true;
row_decision.branch = ReferenceToleranceBranch::kRelative;
row_decision.passed = row_decision.relative_error <= kRelativeTolerance;
}
decision.rows.push_back(row_decision);
}
const bool rows_passed = std::all_of(
decision.rows.begin(), decision.rows.end(),
[](const ReferenceToleranceRowDecision& row) { return row.passed; });
if (decision.reference_scale == 0.0) {
decision.relative_rms = 0.0;
decision.rms_passed = rows_passed;
if (!rows_passed) {
decision.diagnostic_code = "zero-reference-scale-nonzero-error";
}
} else {
const double rms =
error_norm / std::sqrt(static_cast<double>(family.rows.size()));
decision.relative_rms = rms / decision.reference_scale;
if (!std::isfinite(decision.relative_rms)) {
return {false, "nonfinite-comparison-value", false, {}};
}
decision.rms_passed = decision.relative_rms <= kRelativeRmsTolerance;
}
decision.passed = rows_passed && decision.rms_passed;
if (decision.blocking && !decision.passed) {
evaluation.passed = false;
}
evaluation.families.push_back(std::move(decision));
}
return evaluation;
}
}
@@ -0,0 +1,73 @@
#ifndef FESA_TESTS_REFERENCE_REFERENCE_TOLERANCE_POLICY_H_
#define FESA_TESTS_REFERENCE_REFERENCE_TOLERANCE_POLICY_H_
#include <cstddef>
#include <string>
#include <vector>
namespace fesa::test {
enum class ReferenceToleranceBranch {
kNearZero,
kRelative,
kZeroScaleExact,
};
struct ReferenceToleranceRow {
double fesa_value;
double reference_value;
};
struct ReferenceToleranceFamily {
std::string identity;
std::vector<std::string> components;
bool blocking;
std::vector<ReferenceToleranceRow> rows;
};
struct ReferenceToleranceRowDecision {
double absolute_error;
double threshold;
double relative_error;
bool relative_error_applicable;
ReferenceToleranceBranch branch;
bool passed;
};
struct ReferenceToleranceFamilyDecision {
std::string identity;
std::vector<std::string> components;
bool blocking;
double reference_scale;
double near_zero_band;
std::size_t near_zero_count;
double relative_rms;
double maximum_absolute_error;
std::size_t worst_row;
bool rms_passed;
bool passed;
std::string diagnostic_code;
std::vector<ReferenceToleranceRowDecision> rows;
std::size_t row_count{};
};
struct ReferenceToleranceEvaluation {
bool valid;
std::string diagnostic_code;
bool passed;
std::vector<ReferenceToleranceFamilyDecision> families;
};
class ReferenceTolerancePolicy {
public:
static constexpr double kNearZeroRatio = 0.01;
static constexpr double kRelativeTolerance = 0.05;
static constexpr double kRelativeRmsTolerance = 0.01;
static ReferenceToleranceEvaluation Evaluate(
const std::vector<ReferenceToleranceFamily>& families);
};
}
#endif
@@ -0,0 +1,118 @@
#include "reference_tolerance_policy.h"
#include <gtest/gtest.h>
#include <algorithm>
#include <cmath>
#include <limits>
#include <string>
#include <utility>
#include <vector>
namespace {
using fesa::test::ReferenceToleranceBranch;
using fesa::test::ReferenceToleranceFamily;
using fesa::test::ReferenceTolerancePolicy;
using fesa::test::ReferenceToleranceRow;
ReferenceToleranceFamily MakeFamily(std::string identity,
std::vector<std::string> components,
bool blocking,
std::vector<ReferenceToleranceRow> rows) {
return {std::move(identity), std::move(components), blocking,
std::move(rows)};
}
TEST(ReferenceTolerancePolicy,
AppliesRelativeAndNearZeroThresholdsAtTheirInclusiveBoundaries) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate({MakeFamily(
"displacement-translation", {"UX", "UY", "UZ"}, true,
{{105.0, 100.0}, {105.001, 100.0}, {2.0, 1.0}, {2.001, 1.0}})});
ASSERT_TRUE(evaluation.valid);
ASSERT_EQ(evaluation.families.size(), 1U);
const auto& family = evaluation.families.front();
EXPECT_EQ(family.row_count, 4U);
EXPECT_DOUBLE_EQ(family.reference_scale, 100.0);
EXPECT_DOUBLE_EQ(family.near_zero_band, 1.0);
EXPECT_EQ(family.near_zero_count, 2U);
EXPECT_FALSE(family.passed);
ASSERT_EQ(family.rows.size(), 4U);
EXPECT_EQ(family.rows[0U].branch, ReferenceToleranceBranch::kRelative);
EXPECT_TRUE(family.rows[0U].relative_error_applicable);
EXPECT_DOUBLE_EQ(family.rows[0U].relative_error, 0.05);
EXPECT_DOUBLE_EQ(family.rows[0U].threshold, 5.0);
EXPECT_TRUE(family.rows[0U].passed);
EXPECT_FALSE(family.rows[1U].passed);
EXPECT_EQ(family.rows[2U].branch, ReferenceToleranceBranch::kNearZero);
EXPECT_FALSE(family.rows[2U].relative_error_applicable);
EXPECT_DOUBLE_EQ(family.rows[2U].threshold, 1.0);
EXPECT_TRUE(family.rows[2U].passed);
EXPECT_FALSE(family.rows[3U].passed);
}
TEST(ReferenceTolerancePolicy,
RejectsFamiliesWhoseScaleRelativeRmsExceedsOnePercent) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate({MakeFamily(
"reaction-force", {"RF1", "RF2", "RF3"}, true,
{{101.1, 100.0}, {101.1, 100.0}, {101.1, 100.0}, {101.1, 100.0}})});
ASSERT_TRUE(evaluation.valid);
const auto& family = evaluation.families.front();
EXPECT_EQ(family.row_count, 4U);
EXPECT_TRUE(std::all_of(family.rows.begin(), family.rows.end(),
[](const auto& row) { return row.passed; }));
EXPECT_NEAR(family.relative_rms, 0.011, 1.0e-15);
EXPECT_FALSE(family.rms_passed);
EXPECT_FALSE(family.passed);
EXPECT_FALSE(evaluation.passed);
}
TEST(ReferenceTolerancePolicy,
FailsClosedForZeroScaleNonzeroErrorWithoutNonfiniteMetrics) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate(
{MakeFamily("section-force", {"N"}, true, {{0.0, 0.0}, {1.0, 0.0}})});
ASSERT_TRUE(evaluation.valid);
const auto& family = evaluation.families.front();
EXPECT_EQ(family.row_count, 2U);
EXPECT_DOUBLE_EQ(family.reference_scale, 0.0);
EXPECT_DOUBLE_EQ(family.near_zero_band, 0.0);
EXPECT_EQ(family.near_zero_count, 2U);
EXPECT_EQ(family.diagnostic_code, "zero-reference-scale-nonzero-error");
EXPECT_EQ(family.rows[0U].branch, ReferenceToleranceBranch::kZeroScaleExact);
EXPECT_TRUE(family.rows[0U].passed);
EXPECT_FALSE(family.rows[1U].passed);
EXPECT_FALSE(family.rows[1U].relative_error_applicable);
EXPECT_TRUE(std::isfinite(family.relative_rms));
EXPECT_DOUBLE_EQ(family.relative_rms, 0.0);
EXPECT_FALSE(family.rms_passed);
EXPECT_FALSE(family.passed);
}
TEST(ReferenceTolerancePolicy,
KeepsWarningOnlyFamilyExceedanceOutOfBlockingVerdict) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate(
{MakeFamily("translation", {"U1", "U2", "U3"}, true, {{0.0, 0.0}}),
MakeFamily("rotation", {"UR1", "UR2", "UR3"}, false, {{1.0, 0.0}})});
ASSERT_TRUE(evaluation.valid);
ASSERT_EQ(evaluation.families.size(), 2U);
EXPECT_TRUE(evaluation.families[0U].passed);
EXPECT_FALSE(evaluation.families[1U].passed);
EXPECT_TRUE(evaluation.passed);
}
TEST(ReferenceTolerancePolicy,
RejectsNonfiniteInputsBeforeToleranceEvaluation) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate(
{MakeFamily("displacement-translation", {"UX", "UY", "UZ"}, true,
{{std::numeric_limits<double>::quiet_NaN(), 0.0}})});
EXPECT_FALSE(evaluation.valid);
EXPECT_EQ(evaluation.diagnostic_code, "nonfinite-comparison-value");
EXPECT_TRUE(evaluation.families.empty());
}
}