Compare commits

...

3 Commits

Author SHA1 Message Date
KOKO\Mimi a6fadcd8bb modify tolerance policy 2026-08-18 02:57:27 +09:00
KOKO\Mimi 624ea4872c modify tolerance policy 2026-08-18 02:29:26 +09:00
KOKO\Mimi 123a1b6ccc docs: design common reference tolerance policy 2026-08-17 02:57:58 +09:00
36 changed files with 1855 additions and 668 deletions
+14
View File
@@ -15,6 +15,8 @@
- 프로젝트의 배경, 목적, 사용자, 범위는 `docs/PRD.md`에서 확인한다.
- 전체 아키텍처와 모듈별 책임은 `docs/ARCHITECTURE.md`에서 확인한다.
- 주요 아키텍처 결정과 그 이유 및 트레이드오프는 `docs/ADR.md`에서 확인한다.
- 외부 reference tolerance 값, 판정 과정, 보고서와 변경 관리는
`docs/TOLERANCE.md`를 따른다.
- AI 에이전트는 설계나 구현 결정을 내리기 전에 PRD에서 제품 범위를, ARCHITECTURE에서 소유권과 데이터 흐름을, ADR에서 이미 결정된 트레이드오프를 먼저 확인한다. 기능별 요구조건, 연구, 정식화, 수치/reference 검토, I/O, 구현, 검증, 물리 및 release 의미는 `docs/<feature-id>/`의 승인 문서를 source of truth로 삼는다.
- Harness는 솔버 자체가 아니라 요구조건, TDD, phase 실행, 검증을 통제하는 개발 운영 인프라이다. 전체 실행 흐름은 `docs/HARNESS_WORKFLOW.md`, 설치와 설정은 `docs/HARNESS.md`를 따른다.
- FESA는 Abaqus와 독립적인 솔버다. 문서와 구현은 full Abaqus compatibility뿐 아니라
@@ -32,6 +34,18 @@
- 비교기는 필요한 source row identity와 component를 결정적으로 대응시키고 누락, 추가,
중복, nonfinite 값을 tolerance 전에 거부한다. Reference artifact는 rename, rewrite 또는
보정하지 않는다.
- 모든 외부 reference comparison은 `docs/TOLERANCE.md`의 공통 정책을 사용한다.
설계 근거는 `docs/superpowers/specs/2026-08-17-common-reference-tolerance-design.md`
ADR-022에 보존한다. 동일 logical quantity, unit dimension, coordinate system과 blocking behavior의
component family에서 Abaqus-only scale `S=max(abs(reference))`를 구한다. `abs(reference)`
`0.01*S` 이하인 행은 `abs(fesa-reference) <= 0.01*S`, 그 외 행은 상대오차 `<=0.05`
판정하고, family scale-relative RMS `RMS(error)/S <= 0.01`도 함께 통과해야 한다. 별도
absolute-error gate는 사용하지 않으며 zero-scale family는 FESA도 exact zero일 때만
통과한다.
- B33 section-resultant reference는 `(instance, element label, endpoint node label,
component)`로 HDF5 element endpoint와 직접 대응한다. Node-station collapse 또는 평균을
사용하지 않는다. MITC4 `U1/U2/U3`는 blocking, `UR1/UR2/UR3`는 warning-only 분류를
유지하면서 같은 공통 수치 정책을 사용한다.
- CSV는 FESA 공식 output이 아니며, FESA HDF5에서 추출한 deterministic CSV view는 비교 디버깅/검토용 보조 artifact로만 둔다.
## FESA 개발의 핵심 원칙
+40
View File
@@ -129,6 +129,9 @@ linear-static hook 사이에 조건문으로 새 physics를 삽입하지 않는
### ADR-014: B33 cantilever baseline은 component-scale 혼합 tolerance로 비교한다
**상태**: Artifact inventory와 component mapping의 역사적 결정은 유지하며 tolerance와
node-station projection 결정은 ADR-022로 대체됨.
**결정**: `reference/cantilever beam/`의 B33 input, displacement CSV, reaction CSV, elemental-force CSV를 V0 reference baseline으로 사용한다. Displacement와 reaction은 node identity로, section resultant는 CSV node station과 정규화된 FESA element endpoint로 비교한다. 같은 model, step/frame, quantity, component에 대해 `reference_scale = max(abs(reference rows))`를 계산하고 모든 row에 `absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation absolute floor는 `1e-9`, force/moment floor는 `1e-3`이다. Beam stress는 출력하되 Abaqus stress reference comparison은 N/A다.
**이유**: 자유단 moment처럼 이론적으로 0인 값에는 행별 상대오차가 정의되지 않으며 Abaqus 결과에 작은 수치 잔차가 남을 수 있다. Component scale을 사용하면 전체 물리량 크기에 비해 작은 잔차를 허용하면서 각 row를 결정적으로 판정할 수 있다.
@@ -232,6 +235,9 @@ false match와 결과 보정을 방지한다. 더 강한 provenance가 필요한
### ADR-020: MITC4 displacement reference는 고정 절대오차로 판정한다
**상태**: S4 artifact 선택과 U blocking/UR warning-only 분류는 유지하며 고정 절대오차
결정은 ADR-022로 대체됨.
**결정**: Full-integration FESA-MITC4의 sole S4 reference comparison은 matched global
`U1/U2/U3` row에 고정 절대오차 `1.0e-5`를 적용해 pass/fail을 판정한다.
`UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하지만 초과는 deterministic warning만
@@ -306,3 +312,37 @@ formulation, 연산·reduction 순서, sign, units, coordinates와 result identi
리팩터링보다 우선하며 HDF5 schema, reference artifact와 ADR-014/ADR-020 tolerance는
변경하지 않는다. MITC3, solid, dynamic과 plastic behavior는 별도 feature gate 전까지
구현된 것으로 간주하지 않는다.
### ADR-022: 외부 reference comparison은 공통 family-scale 정책을 사용한다
운영 상수, 검증 순서, report schema와 변경 관리는 `docs/TOLERANCE.md`를 따른다.
**결정**: B33, MITC4와 이후 기능의 외부 reference comparison은 동일한 무차원 수치
정책을 사용한다. 같은 model/case, step/frame, logical quantity, unit dimension, coordinate
system과 blocking behavior의 component family에 대해 reference-only scale
`S=max(abs(reference))`를 계산한다. `abs(reference)<=0.01*S`인 행은
`abs(fesa-reference)<=0.01*S`, 나머지 행은 `abs(fesa-reference)/abs(reference)<=0.05`
만족해야 한다. 모든 행과 함께 family scale-relative RMS `RMS(error)/S<=0.01`
통과해야 한다. 독립 absolute-error gate는 두지 않으며 `S=0`이면 모든 FESA 값도 exact
zero일 때만 통과한다.
B33 family는 translation, rotation, reaction force, reaction moment, section force와
section moment로 구분한다. 2026-08-18에 다시 생성된 elemental-force CSV의
`Element Label``Node Label`을 사용해 `(instance, element, endpoint node, component)`
HDF5 endpoint에 직접 대응하고 node-station collapse나 평균을 사용하지 않는다. MITC4는
translation `U1/U2/U3` family를 blocking으로, rotation `UR1/UR2/UR3` family를
warning-only로 유지한다.
**이유**: Feature별 absolute floor와 fixed absolute tolerance는 모델 단위와 크기에 따라
서로 다른 엄격도를 만들었다. 개별 component scale은 물리적으로 zero-like인 MITC4
`U1/U2` residue를 자체 scale로 만들어 잘못 실패시킨다. 동일 차원의 family scale,
near-zero 대체 분기와 relative RMS를 결합하면 zero row의 불안정한 raw relative error를
피하면서 모든 행과 전체 오차 수준을 함께 검사할 수 있다. B33의 element-endpoint identity는
reference와 HDF5가 제공하는 실제 source identity를 보존한다.
**트레이드오프**: Family 최대값이 작은 component의 허용폭을 결정하므로 행별 relative
gate와 family RMS gate를 모두 유지해야 한다. Reference family 전체가 zero이면 별도
절대 scale이 없으므로 exact-zero만 허용하는 엄격한 fail-closed 정책이 된다. 기존 B33과
MITC4 verification report 및 comparator schema는 새 정책에 맞춰 다시 생성해야 하지만
reference artifact, HDF5 schema, element formulation과 blocking/warning 분류는 변경하지
않는다.
+16 -12
View File
@@ -419,14 +419,19 @@ Schema requirements:
`[N,T,My,Mz]`, generalized strain/resultant는 두 Gauss point에 기록한다.
- General beam section stress는 section point의 axial `S11`만 복구한다. Section point가
없으면 centroid `(0,0)``source=fesa-default`로 기록한다.
- 승인된 `reference/cantilever beam/cantilever beam elemental forces.csv`node station
기준 `SF1/SM1/SM2/SM3`을 제공한다. FESA endpoint를 동일한 section-cut 부호로
정규화하고 interior node의 두 endpoint가 tolerance 안에서 일치하는지 먼저 확인한 뒤
`SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`로 비교한다.
- Reference tolerance는 같model, step/frame, quantity, component의 Abaqus rows에서
`reference_scale = max(abs(reference_value))`를 구하고 각 row에
`absolute_floor + 1e-6 * reference_scale`을 적용한다. SI displacement/rotation floor는
`1e-9`, force/moment floor는 `1e-3`이다.
- 승인된 `reference/cantilever beam/cantilever beam elemental forces.csv`각 B33
element의 두 endpoint에 `Element Label`, `Node Label``SF1/SM1/SM2/SM3`을 제공한다.
`(instance, element label, endpoint node label, component)` identity로 HDF5
`[element,endpoint,N/T/My/Mz]`에 직접 대응하고 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`,
`SM3 -> T`로 비교한다. External reference comparisonnode-station collapse나 평균을
사용하지 않는다.
- Reference tolerance는 같은 model/case, step/frame, logical quantity, unit dimension,
coordinate system과 blocking behavior의 component family에서 Abaqus-only scale
`S=max(abs(reference))`를 구한다. `abs(reference)<=0.01*S`인 행은 absolute error
`<=0.01*S`, 그 외 행은 relative error `<=0.05`로 판정하며 family scale-relative RMS
`RMS(error)/S<=0.01`도 통과해야 한다. 독립 absolute-error gate는 사용하지 않고
zero-scale family는 FESA도 exact zero일 때만 통과한다. 전체 판정 과정과 report/change
management contract는 `docs/TOLERANCE.md`를 따른다.
- Beam stress는 HDF5 schema와 unit/analytical test로 검증하지만 Abaqus reference
comparison은 N/A다.
@@ -469,9 +474,8 @@ stabilization만 둔다. Drilling calibration, artificial-energy policy와 별
dataset은 이 기능 범위가 아니다.
Full-integration FESA-MITC4의 reference comparison은 `reference/shell/` S4의 기존 input 및
displacement CSV만 사용한다. Global `U1/U2/U3`만 blocking이고 모든 matched row에 고정
절대오차 `1.0e-5`를 적용한다. `UR1/UR2/UR3`도 고정 절대오차 `1.0e-5`로 비교하되
warning-only evidence다. MITC4 판정에는 component scale을 사용하지 않으며 B33의 기존
혼합 tolerance는 변경하지 않는다. S4R은
displacement CSV만 사용한다. Global `U1/U2/U3` translation family만 blocking이고
`UR1/UR2/UR3` rotation family는 warning-only evidence다. 두 family 모두 ADR-022의 공통
family-scale row/RMS 정책을 사용한다. S4R은
같은 kernel을 선택하는 source mapping과 metadata를 unit/integration tests로 검증하며
`reference/shellR/` artifact는 acceptance comparison에 포함하지 않는다.
+7 -7
View File
@@ -113,7 +113,7 @@ Parser keyword, element kernel, solver backend 또는 output dataset 중 하나
| FESA-PRD-014 | 선형 정적 pipeline은 stiffness factorization과 load substitution을 분리해야 한다. | `Kff` factorization이 load vector assembly보다 먼저 수행되고, `rhs=Ff-Kfc*dc` substitution으로 full displacement를 복구한다. | orchestration test, solver-adapter test |
| FESA-PRD-015 | FESA는 V0 결과와 diagnostic을 안정된 외부 계약으로 출력해야 한다. | CLI가 `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5` exit code를 사용하고 diagnostic field가 계약과 일치하며 HDF5가 displacement, reaction, end force, section resultant, generalized strain/resultant, axial `S11`을 포함하고 실패 시 불완전한 최종 파일을 남기지 않는다. | CLI integration test, HDF5 schema/atomicity test |
| FESA-PRD-016 | Dense와 sparse math storage 및 backend 경계를 분리해야 한다. | `Vector`는 contiguous, `Matrix`는 row-major contiguous storage와 MKL CBLAS를 사용하고 `SparseMatrix`는 별도 0-based CSR 타입이며 MKL 타입이 public core API에 노출되지 않는다. | math unit test, dependency review |
| FESA-PRD-017 | B33 reference comparison은 component-scale 혼합 tolerance를 사용해야 한다. | 모든 matched row가 `abs_error <= absolute_floor + 1e-6 * reference_scale`을 만족하고 missing/extra/nonfinite row는 comparison 전에 실패한다. | reference comparison unit/integration test, verification report |
| FESA-PRD-017 | 모든 외부 reference comparison은 승인된 공통 family-scale tolerance를 사용해야 한다. | matched row가 near-zero 대체 조건 또는 상대오차 `0.05`를 통과하고 각 family의 scale-relative RMS가 `0.01` 이하이며 missing/extra/duplicate/nonfinite row는 comparison 전에 실패한다. | reference comparison unit/integration test, verification report |
| FESA-PRD-018 | 승인된 B33 reference artifact는 현재 경로의 read-only baseline으로 유지해야 한다. | `reference/cantilever beam/` 파일을 rename, rewrite 또는 보정하지 않고 기능이 선언한 exact path에서 읽는다. | artifact inventory, Git diff review |
| FESA-PRD-019 | B33 beam section과 local axis를 Abaqus 의미에 맞게 매핑해야 한다. | `n1 -> local y`, `t x n1 -> local z`, `Iy=I11`, `Iz=I22`, `I12=0`을 적용하고 nonpositive property, zero-length element, tangent-parallel guide vector를 구조화된 model diagnostic으로 거부한다. | section-mapping unit test, element geometry test |
@@ -121,12 +121,12 @@ Parser keyword, element kernel, solver backend 또는 output dataset 중 하나
- MSVC x64 Debug 환경에서 configure, build, CTest를 검증한다.
- reference test 결과는 deterministic해야 한다.
- HDF5 schema는 versioned contract로 관리한다.
- B33 reference tolerance는 model, step/frame, quantity, component별 Abaqus scale만 사용한다.
- 승인된 SI B33 bundle의 absolute floor는 displacement/rotation `1e-9`, force/moment
`1e-3`이며 relative coefficient는 `1e-6`이다.
- MITC4 U/UR comparison은 고정 절대오차 `1.0e-5`를 사용한다. `U1/U2/U3`
blocking이고 `UR1/UR2/UR3` 초과는 warning-only다. B33의 component-scale 혼합
tolerance는 별도 기존 계약으로 유지한다.
- 모든 외부 reference comparison은 동일 logical quantity와 unit dimension의 component
family별 Abaqus-only maximum scale을 사용한다. Near-zero ratio `0.01`, 일반 행
상대오차는 `0.05`, family scale-relative RMS는 `0.01`이며 독립 absolute-error gate는
사용하지 않는다.
- B33 section resultant는 source element label과 endpoint node label로 직접 대응한다.
MITC4 `U1/U2/U3`만 blocking이고 `UR1/UR2/UR3` 초과는 warning-only다.
- parser, solver, HDF5 writer는 실패 원인을 구조화된 diagnostic으로 보고한다.
- oneMKL, oneTBB, HDF5는 CMake에서 명시 탐지하고 실패 원인을 분류한다.
- 대규모 모델 성능 최적화보다 V0의 명확성, 테스트 가능성, 검증 traceability를 우선한다.
+2 -1
View File
@@ -148,7 +148,8 @@ gap은 Coordinator가 해당 owner 단계로 되돌린다.
Reference comparison은 source identity와 component로 행을 결정적으로 대응시키며 missing,
extra, duplicate 또는 nonfinite required row를 tolerance 전에 거부한다. CSV는 외부 reference이고
FESA의 authoritative output은 `results.h5`다.
FESA의 authoritative output은 `results.h5`다. 공통 tolerance 값, family 구성, row/RMS
판정, report evidence와 변경 관리는 `docs/TOLERANCE.md`를 따른다.
## 요구사항 단위 산출물 구조
+179
View File
@@ -0,0 +1,179 @@
# FESA Reference Tolerance Policy
## 문서 정보
- policy_id: `common-reference-tolerance`
- status: `approved-and-implemented`
- effective_date: `2026-08-18`
- decision_record: `docs/ADR.md`의 ADR-022
## 목적
이 문서는 FESA 결과와 외부 reference 결과를 비교할 때 사용하는 공통 tolerance 값과
검증 방법을 정의한다. 현재 기능뿐 아니라 앞으로 추가되는 요소, 재료, 해석, 하중 및
경계조건의 reference comparison에도 같은 규칙을 적용한다.
이 문서는 공통 수치 판정만 정의한다. 각 기능의 비교 대상, 단위, 좌표계, row identity,
component 구성과 최종 판정 영향은 해당 기능의 `requirements.md`, `reference-model.md`
`io.md`에서 정의한다.
## 적용 범위
이 정책은 승인된 외부 reference 값과 FESA 공식 결과인 `results.h5`의 값을 비교하는 데
사용한다. Parser schema, row identity, 물리 평형, 수렴성 및 정식화 검증에는 각각의 별도
계약을 적용한다. 이러한 검증 실패를 수치 tolerance로 완화해서는 안 된다.
## 공통 tolerance 값
| 항목 | 값 | 의미 |
| --- | ---: | --- |
| Near-zero 비율 | `0.01` | Reference family 최대값의 1% 이하를 near-zero로 분류 |
| 상대오차 tolerance | `0.05` | 일반 행의 상대오차를 5% 이하로 제한 |
| Relative RMS tolerance | `0.01` | Family 전체 RMS 오차를 reference scale의 1% 이하로 제한 |
독립적인 absolute-error tolerance는 사용하지 않는다. Absolute error는 near-zero 행을
판정하고 결과를 진단하기 위해서만 사용한다.
## Comparison family
수치 scale은 개별 행이나 component마다 만들지 않고 comparison family마다 계산한다.
하나의 family에는 다음 조건이 같은 값만 포함한다.
- 같은 model 또는 reference case
- 같은 step과 frame
- 같은 logical quantity
- 같은 단위 차원
- 같은 좌표계
- 같은 최종 판정 영향(`blocking` 또는 `warning-only`)
각 기능 문서는 family 이름, 포함 component와 위 항목을 명시해야 한다. 서로 다른 단위,
좌표계 또는 판정 영향을 가진 값은 같은 family에 포함할 수 없다.
## 검증 방법
### 1. 비교 입력 확정
기능 문서가 승인한 reference artifact와 FESA `results.h5`를 사용한다. Reference artifact는
비교를 통과시키기 위해 이름을 바꾸거나 값을 수정, 보정 또는 zero-clamp하지 않는다.
### 2. Row 대응 및 사전검사
Reference와 FESA 값을 기능 문서가 정의한 stable source identity와 component로 일대일
대응시킨다. 다음 오류는 tolerance 계산 전에 comparison을 실패시킨다.
- 필요한 파일, dataset 또는 component 누락
- missing, extra 또는 duplicate row
- source identity 불일치
- 비유한 값(`NaN`, `Inf`)
Tolerance는 schema 또는 identity 오류를 허용하는 수단이 아니다.
### 3. Reference scale 계산
Family의 reference 값 `r_i`만 사용해 scale `S`와 near-zero band `B`를 계산한다.
\[
S = \max_i |r_i|
\]
\[
B = 0.01S
\]
FESA 값은 scale 계산에 사용하지 않는다. 임의의 absolute floor나 `max(1, S)`도 추가하지
않는다.
### 4. 행별 오차 판정
FESA 값 `f_i`와 reference 값 `r_i`의 absolute error를 계산한다.
\[
e_i = |f_i-r_i|
\]
Reference 값이 near-zero band 안에 있으면 absolute error로 판정한다.
\[
|r_i| \le B \quad\Rightarrow\quad e_i \le B
\]
그 외 행은 상대오차로 판정한다.
\[
|r_i| > B \quad\Rightarrow\quad \frac{e_i}{|r_i|} \le 0.05
\]
경계값은 통과에 포함하며 모든 대응 행을 검사한다.
### 5. Family Relative RMS 판정
Family의 모든 absolute error로 RMS를 계산하고 reference scale로 정규화한다.
\[
\operatorname{relative\_rms} =
\frac{\sqrt{\frac{1}{n}\sum_i e_i^2}}{S}
\]
다음을 만족해야 RMS 판정을 통과한다.
\[
\operatorname{relative\_rms} \le 0.01
\]
Family가 통과하려면 모든 행과 Relative RMS가 모두 통과해야 한다.
### 6. Reference scale이 0인 경우
`S = 0`이면 family의 모든 reference 값이 정확히 0이다.
- 모든 FESA 값도 정확히 0이면 통과한다.
- 하나라도 0이 아니면 실패한다.
- 결과에는 비유한 metric 대신 `zero-reference-scale-nonzero-error`를 기록한다.
### 7. 최종 판정
- 사전검사 실패는 항상 전체 comparison을 실패시킨다.
- `blocking` family의 행 또는 RMS 실패는 전체 comparison을 실패시킨다.
- `warning-only` family의 실패는 warning을 기록하되 전체 blocking 판정은 변경하지 않는다.
- Warning은 행 실패와 RMS 실패를 구분해 결정적인 순서로 기록한다.
## 결과 기록
Comparison 결과는 원본 값에서 판정을 재현할 수 있어야 한다. 최소한 다음 정보를
기록한다.
- 사용한 input, reference artifact와 FESA 결과 identity
- 사전검사 결과
- Family identity, component, row 수와 reference scale
- 각 행의 FESA 값, reference 값, error, 적용 판정과 통과 여부
- Family Relative RMS와 통과 여부
- Blocking failure, warning과 전체 verdict
동일한 입력을 반복 비교하면 row, family, warning과 결과 출력 순서가 같아야 한다.
## 새 기능에 적용하는 방법
새 기능의 comparator를 구현하기 전에 기능 문서에서 다음 항목을 승인한다.
1. 사용할 input, reference artifact와 FESA HDF5 위치
2. 비교할 quantity와 component
3. 단위와 좌표계
4. Stable source row identity와 일대일 mapping
5. Comparison family 구성
6. `blocking` 또는 `warning-only` 판정 영향
구현 시에는 사전검사, tolerance 경계값, zero-scale, 행별 판정, Relative RMS와 결과 기록을
테스트한다. 이후 기능별 reference comparison을 다시 실행해 evidence를 남긴다.
## 변경 관리
Tolerance 값, 계산식 또는 family 구성 규칙을 변경하려면 다음 절차를 따른다.
1. 변경 이유와 영향을 검토하고 사용자 승인을 받는다.
2. 이 문서와 ADR을 갱신한다.
3. 경계값과 실패 동작을 테스트로 먼저 고정한다.
4. 영향받는 comparator와 기능 문서를 수정한다.
5. 전체 테스트와 영향받는 reference comparison을 다시 실행한다.
기존 comparison report의 과거 수치를 소급 수정하지 않는다. 변경된 정책으로 새 evidence를
생성하며 reference artifact 자체는 변경하지 않는다.
+19 -12
View File
@@ -1311,9 +1311,10 @@ $$
가 일치해야 한다. 따라서 이 V0 조건의 equilibrium end action은 요소 끝에 작용하는
outward action이고, endpoint section resultant는 동일 section을 양의 local $x$ 면으로
자른 값이다. Abaqus node-station force row에는 equilibrium end action이 아니라 section
resultant를 동일한 section-cut 부호로 정규화해 비교하며, interior node의 두 endpoint
승인 tolerance 안에서 먼저 일치해야 한다. 승인된 component mapping은
자른 값이다. Abaqus element-endpoint force row에는 equilibrium end action이 아니라
section resultant를 동일한 section-cut 부호로 정규화해 비교한다. Reference의 element
label과 endpoint node label을 HDF5 row에 직접 대응하며 node station collapse, 대표 endpoint
선택 또는 평균은 하지 않는다. 승인된 component mapping은
`SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`, `SM3 -> T`이고, CSV에 없는 transverse
`SF2`, `SF3`는 reference comparison 대상이 아니다.
@@ -1529,19 +1530,25 @@ $$
\sum\mathbf M_{\mathrm{reaction}}\right\|\le\epsilon_M.
$$
승인된 B33 reference row는 같은 model, step/frame, quantity, componentAbaqus
reference 값만 사용해
승인된 B33 reference row는 같은 model, step/frame, quantity, component로 family를 만들고
Abaqus reference 값만 사용해
$$
\operatorname{reference\_scale}=\max_i|r_i|,\qquad
\operatorname{row\_tolerance}=\operatorname{absolute\_floor}
+10^{-6}\operatorname{reference\_scale}
S=\max_i|r_i|
$$
적용한다. 승인된 SI bundle의 displacement/rotation floor는 $10^{-9}$,
force/moment floor는 $10^{-3}$이다. Reference 값을 zero-clamp하거나 row를 제거하지
않으며 missing, extra, duplicate, nonfinite, schema/identity mismatch는 수치 판정 전에
실패한다. 이 reference policy는 위 formulation/analytical tolerance를 대체하지 않는다.
계산한다. $|r_i|\le0.01S$인 near-zero row는 $|f_i-r_i|\le0.01S$로,
나머지 row는 $|f_i-r_i|/|r_i|\le0.05$로 판정한다. 또한
$$
\frac{\sqrt{n^{-1}\sum_i(f_i-r_i)^2}}{S}\le0.01
$$
을 family 전체에서 만족해야 한다. $S=0$이면 모든 FESA 값이 정확히 0일 때만 통과하고,
그렇지 않으면 zero-reference-scale-nonzero-error로 실패한다. 독립적인 절대오차 gate는
사용하지 않는다. Reference 값을 zero-clamp하거나 row를 제거하지 않으며 missing, extra,
duplicate, nonfinite, schema/identity mismatch는 수치 판정 전에 실패한다. 이 reference
policy는 위 formulation/analytical tolerance를 대체하지 않는다.
## 19. Numerical Risks
+47 -46
View File
@@ -12,7 +12,7 @@
- owner_agent: `io-definition-agent`
- date: `2026-08-09`
- authoritative_output: `results.h5`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b`
- reference_baseline: exact read-only artifacts under `reference/cantilever beam/`
이 문서는 승인된 V0의 semantic I/O contract만 정의한다. Parser, HDF5 writer,
comparison tooling의 C++ API나 구현 구조는 정의하지 않으며 Abaqus full compatibility를
@@ -336,7 +336,7 @@ CSV는 `SF1/SM1/SM2/SM3`만 제공하고 section-cut result와 비교하므로 `
0으로 만들거나 reference row로 합성하지 않는다. HDF5의 transverse end action은
unit/analytical test와 physics sanity 대상이다.
### Frame, instance, and node-station normalization
### Frame, instance, and element-endpoint normalization
- Legacy `Frame` value `Increment 1: Step Time = 1.000`은 canonical `(Step-1, frame 0)`으로
변환한다. 승인 bundle의 다른 increment/time string은 `schema-mismatch`다.
@@ -344,29 +344,27 @@ unit/analytical test와 physics sanity 대상이다.
`instance_name`과 case-insensitive lookup 후 raw identity 일치 여부를 확인한다.
- `Node Label`은 instance 안의 preserved source node label로 resolve한다. Displacement와
reaction key는 `(model_id,Step-1,0,instance_name,source_node_label,quantity,component)`다.
- Elemental-force CSV는 element label이 없으므로 HDF5 endpoint section resultants를
source node station으로 project한다. Boundary station은 유일한 incident endpoint를 쓴다.
- Interior station collapse는 정확히 두 incident B33 endpoints, 동일 section/local-axis
orientation, 일관된 chain connectivity, 해당 node의 concentrated force/moment가 없는
경우에만 허용한다. 두 positive-face section-cut 값이 아래 승인 component tolerance
안에서 먼저 일치해야 한다.
- Interior 값이 일치하면 stable internal element ID가 작은 endpoint를 deterministic
representative 선택한다. 두 값을 평균하지 않는다. 불일치는 `tolerance-failure`다.
- Reversed connectivity, local-axis discontinuity, section jump, branch 또는 loaded interior
station은 element label 없는 legacy schema로 collapse할 수 없다. 이 approved bundle
밖에서는 element-aware reference row가 필요하며 legacy projection은 `schema-mismatch`
중단한다.
- Elemental-force CSV`Element Label`은 instance 안의 preserved source element label로,
`Node Label`은 해당 B33 connectivity의 정확한 endpoint source node로 resolve한다.
- Section-resultant key는
`(model_id,Step-1,0,instance_name,source_element_label,source_node_label,quantity,component)`다.
한 element에는 connectivity와 일치하는 두 endpoint 행이 정확히 존재해야 한다.
- Missing, extra, duplicate 또는 element-connectivity-mismatched endpoint row는
`schema-mismatch`로 tolerance 전에 실패한다. Source node station collapse,
deterministic representative 선택 또는 endpoint 평균은 사용하지 않는다.
Projected canonical comparison row는 다음 fields를 가진다.
```text
model_id, step_name, frame_index, instance_name, source_node_label,
quantity, component, value, unit_dimension, coordinate_system, hdf5_dataset_path
model_id, step_name, frame_index, instance_name,
[source_element_label], source_node_label, quantity, component, value,
unit_dimension, coordinate_system, hdf5_dataset_path
```
Stable ordering은 quantity inventory order, instance declaration order, stable source node
order, component order다. Approved model ID는 `cantilever-beam-b33`; nodal quantities는
global Cartesian, section resultants는 beam local이다.
`source_element_label`은 section-resultant row에 필수이고 nodal row에는 적용하지 않는다.
Stable ordering은 quantity inventory order, instance declaration order, stable source
node/element-endpoint order, component order다. Approved model ID는 `cantilever-beam-b33`;
nodal quantities는 global Cartesian, section resultants는 beam local이다.
### Row-set precheck and tolerance
@@ -376,27 +374,32 @@ identity-mismatched row가 하나라도 있으면 tolerance 계산 전에 실패
mandatory인 `SF2/SF3` 대응 end action, generalized results와 `S11`은 계약상 비교 대상이
아니므로 extra reference row가 아니다.
Matched rows는 같은 `model_id`, step/frame, quantity, component로 group한다.
Matched rows는 같은 model/case, step/frame, logical quantity, unit dimension, coordinate
system과 blocking behavior의 component family로 group한다. B33 family는 translation
`UX/UY/UZ`, rotation `URX/URY/URZ`, reaction force `RF1/RF2/RF3`, reaction moment
`RM1/RM2/RM3`, section force `N`, section moment `T/My/Mz`다.
```text
reference_scale = max(abs(reference_value_i))
row_tolerance = absolute_floor + 1e-6 * reference_scale
row_pass = abs(fesa_value_i - reference_value_i) <= row_tolerance
S = max(abs(reference_value_i))
error_i = abs(fesa_value_i-reference_value_i)
near_zero_band = 0.01*S
if abs(reference_value_i) <= near_zero_band:
row_pass = error_i <= near_zero_band
else:
row_pass = error_i/abs(reference_value_i) <= 0.05
relative_rms = sqrt(mean(error_i^2))/S
family_pass = all(row_pass) and relative_rms <= 0.01
```
즉 exact policy는 `absolute_floor + 1e-6 * reference_scale`이다. `reference_scale`
read-only Abaqus values만 사용하고 FESA 값으로 조정하지 않는다. Scale이 zero면 relative
term은 zero다. Reference value나 작은 residue를 zero-clamp하지 않고 모든 row를 판정한다.
| approved SI component class | absolute floor |
| --- | ---: |
| displacement and rotation | `1e-9` |
| force and moment | `1e-3` |
Interior endpoint consistency도 해당 CSV quantity/component의 same Abaqus-only scale과
floor를 사용한다. Verification report는 모든 row pass/fail과 quantity별 max absolute
error, component-scale normalized error, RMS error, norm error, worst row/component를
기록한다. 이 reference tolerance는 analytical/formulation tolerance를 대체하지 않는다.
`S`는 read-only Abaqus values만 사용하고 FESA 값으로 조정하지 않는다. 독립적인
absolute-error gate와 zero clamp는 사용하지 않는다. `S=0`이면 모든 FESA 값도 exact
zero일 때 relative RMS를 0으로 기록하고 통과하며, 하나라도 nonzero이면 NaN/Inf 대신
`zero-reference-scale-nonzero-error`로 실패한다. Verification report는 모든 row의 적용
branch와 pass/fail, family identity/components, scale, near-zero band/count, max absolute
error, scale-relative RMS와 worst row/component를 기록한다. 이 reference tolerance는
analytical/formulation tolerance를 대체하지 않는다.
## CLI and Diagnostics Contract
@@ -445,17 +448,15 @@ release approval을 이 status가 의미하지 않는다.
### Resolved numerical-review handoff
`NR-O03-STATION-NORMALIZATION`approved legacy bundle에 대해 unloaded, consistently
oriented two-endpoint interior station만 collapse하고, tolerance check 후 smaller stable
element ID를 선택하는 규칙으로 구체화했다. Reversed/branched/loaded/jumped station은
element-aware reference가 없는 한 비교하지 않는다.
`NR-O03-STATION-NORMALIZATION`2026-08-18 regenerated elemental-force CSV의
`Element Label`로 해소됐다. Comparator는 `(instance, element label, endpoint node label,
component)`를 직접 대응하며 station collapse, 대표 endpoint 선택 또는 평균을 하지 않는다.
### Reference Model Agent
- Exact legacy inventory, generator `Abaqus/CAE Learning Edition 2024`, source commit
`2b34d0b`, external SI provenance와 stress comparison N/A를 계약에 고정한다.
- 추가 reference model은 canonical filenames와 metadata를 사용하며 이 legacy file을
변경하지 않는다.
- Exact legacy inventory, current row schema와 stress comparison N/A를 계약에 고정한다.
- 추가 reference case의 exact paths와 수치 비교 계약은 해당 feature requirement가 정하며,
canonical naming이나 optional metadata를 readiness 조건으로 추가하지 않는다.
### Implementation Planning Agent
@@ -467,7 +468,7 @@ element-aware reference가 없는 한 비교하지 않는다.
### Reference Verification Agent
- Artifact precheck 뒤 HDF5-to-legacy projection, node-station eligibility, row-set equality,
component-scale comparison 순서를 유지한다.
- Artifact precheck 뒤 HDF5-to-CSV source identity projection, direct element-endpoint row-set
equality, common family-scale comparison 순서를 유지한다.
- Missing/extra/nonfinite row를 무시하거나 `SF2/SF3`/stress reference row를 합성하지 않는다.
@@ -191,8 +191,13 @@ Confirmed defects, risks, and open issues are separated.
2. `NR-O02-DETERMINISTIC-REDUCTION`: stable COO sort and duplicate-summation rules are project policy and must be made explicit before NR-T11.
3. `NR-O03-STATION-NORMALIZATION`: reversed connectivity/local-axis orientation and legitimate jumps at loaded interior nodes need an explicit downstream row-normalization/eligibility rule. The legacy baseline may use its documented stable orientation and unloaded interior stations, but mismatch must never be averaged. NR-T07 covers element signs.
2026-08-18 amendment: `NR-O03-STATION-NORMALIZATION` is resolved and superseded for the
approved case. The regenerated elemental-force CSV supplies `Element Label`, so comparison uses
direct `(instance, element label, endpoint node label, component)` identity and never collapses,
selects or averages adjacent endpoints.
No open issue requires formulation revision. NR-O01/NR-O02 are implementation-planning
handoffs; NR-O03 belongs to I/O and reference-model contracts.
handoffs; the resolved NR-O03 identity is fixed by the I/O and reference-model contracts.
## Required Revisions
@@ -206,7 +211,7 @@ handoffs; NR-O03 belongs to I/O and reference-model contracts.
### Reference Model Agent
- Make NR-O03 orientation and unloaded-interior assumptions explicit without modifying the approved legacy artifacts.
- Enforce the resolved NR-O03 direct element-endpoint identity without modifying reference artifacts.
## Downstream Handoff
@@ -10,11 +10,45 @@
- source_io_definition: `docs/linear-static-3d-euler-beam/io.md`
- source_implementation_plan: `docs/linear-static-3d-euler-beam/implementation-plan.md`
- source_implementation_report: `docs/linear-static-3d-euler-beam/implementation-report.md`
- status: `pass-for-physics-evaluation`
- historical_status: `pass-for-physics-evaluation`
- current_status: `pass-for-physics-evaluation`
- superseded_on: `2026-08-18`
- revalidated_on: `2026-08-18`
- owner_agent: `reference-verification-agent`
- date: `2026-08-09`
- review_fix_date: `2026-08-10`
The 2026-08-09 evidence below is preserved as a historical record of the former 11-station,
component-scale comparison. ADR-022 and the regenerated 20-row element-endpoint CSV supersede
that identity and tolerance. The following revalidation is the current reference-gate evidence.
## 2026-08-18 Common-policy Revalidation
`cmake --build .harness/build --config Debug` passed, followed by
`ctest --test-dir .harness/build -C Debug --output-on-failure`: 214/214 tests passed.
The B33 reference test generated
`.harness/build/reference/cantilever-beam-b33/comparison.json` from the exact read-only
reference paths.
- exact canonical rows: 212 = 66 displacement + 66 reaction + 80 direct element-endpoint
section-resultant rows
- exact families: 6; all row gates and family RMS gates passed
- direct section identity: 20 endpoint rows, 10 B33 elements x 2 connectivity-matched endpoints
- warnings, identity/schema/nonfinite failures: 0
- overall verdict: `passed=true`
| family | rows | scale | near-zero rows | max absolute error | scale-relative RMS |
| --- | ---: | ---: | ---: | ---: | ---: |
| translation `UX/UY/UZ` | 33 | `1.90476272e-2` | 23 | `5.333229170789711e-10` | `8.459623217000381e-9` |
| rotation `URX/URY/URZ` | 33 | `2.85714399e-3` | 23 | `1.000013943180944e-10` | `1.1723557271997219e-8` |
| reaction force `RF1/RF2/RF3` | 33 | `1.0e6` | 32 | `8.195638656616211e-7` | `2.0922817757699256e-13` |
| reaction moment `RM1/RM2/RM3` | 33 | `1.0e7` | 32 | `5.0514936447143555e-6` | `8.811160558471221e-14` |
| section force `N` | 20 | `0` | 20 | `0` | `0` |
| section moment `T/My/Mz` | 60 | `1.0e7` | 41 | `1.2499958951957524e-1` | `2.2910664449287597e-9` |
All values above come from the generated JSON ledger. The historical report body starts below
and must not be read as the current identity/tolerance evidence.
The prerequisite build/test report has status
`pass-for-reference-verification`. This report applies only the approved Abaqus
B33 reference tolerance. It does not approve physics sanity or release readiness.
@@ -12,9 +12,10 @@
- status: `ready-for-implementation-planning`
- owner_agent: `reference-model-agent`
- date: `2026-08-09`
- amended_on: `2026-08-18`
- approved_reference_model: `cantilever-beam-b33`
- approved_reference_schema: `abaqus-cae-report-csv-v0`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b`
- approved_reference_identity: direct source-node and element-endpoint rows
- reference_baseline: exact current files under `reference/cantilever beam/`
이 문서는 구현 전에 필요한 code verification, analytical solution verification 및
approved B33 reference comparison의 모델·artifact 계약을 정의한다. 이 status는 모델과
@@ -45,7 +46,7 @@ test fixtures이며 reference artifact bundle로 가장하지 않는다.
논리 모델 `cantilever-beam-b33`의 exact read-only legacy bundle만 사용하여 FESA
`results.h5`의 displacement, reaction 및 endpoint section resultant를 Abaqus/CAE report
CSV row와 비교한다. Artifact precheck와 exact row-set matching이 먼저 통과해야 하며,
수치 비교는 component-scale mixed tolerance를 사용한다. Axial `S11` output은 필수지만
수치 비교는 ADR-022의 common family-scale tolerance를 사용한다. Axial `S11` output은 필수지만
Abaqus beam stress comparison은 명시적 N/A다.
### Excluded validation scope
@@ -136,8 +137,8 @@ single-step deck and changes only the named condition.
- boundary_conditions: source node 1, DOFs 1 through 6 fixed
- load: source node 11, global DOF 3, magnitude `-1e6` N
- model_id: `cantilever-beam-b33`
- logical_schema: `abaqus-cae-report-csv-v0`
- source_commit: `2b34d0b`
- historical_schema_record: `abaqus-cae-report-csv-v0` for the 2026-08-09 inventory only
- historical_source_commit: `2b34d0b` for the 2026-08-09 inventory only
- generator: `Abaqus/CAE Learning Edition 2024`
- units: SI
- nodal_coordinate_system: global Cartesian
@@ -145,7 +146,7 @@ single-step deck and changes only the named condition.
- step_name: `Step-1`
- increment: `1`
- step_time: `1.0`
- artifact_status: all four exact paths present; structural precheck observed; FESA comparison not run
- artifact_status: all four exact paths present; 20-row element-endpoint comparison passed on 2026-08-18
- stress: N/A for Abaqus reference comparison; mandatory FESA `S11` remains covered by unit/analytical and HDF5 schema tests
The input and CSV numeric reference values are not recalculated, repaired, rounded, clamped or
@@ -193,93 +194,44 @@ For this approved legacy bundle:
- `README.md`: N/A
- stress CSV: N/A because beam stress reference comparison is outside the approved V0 scope
The approved design and this contract record model ID, provenance, generator, source commit,
units, coordinate systems, step/frame identity, logical CSV schema, exact inventory, tolerance
policy and the stress N/A reason. The optional metadata file's absence and the approved legacy
README/stress exclusions therefore do not change the record to `needs-reference-artifacts`.
This contract records the exact required paths, row identities, comparison quantities, tolerance
policy and the stress N/A reason. Historical schema/provenance fields are retained as dated
inventory only and are not readiness gates. The optional metadata file's absence and the approved
legacy README/stress exclusions do not change the record to `needs-reference-artifacts`.
### Future reference bundles
### Later reference cases and optional metadata
Every later reference model shall use this structure unless its approved requirement explicitly
marks a quantity N/A:
```text
reference/
<model-id>/
model.inp
metadata.json # optional
<model-id>_displacements.csv
<model-id>_reactions.csv
<model-id>_internalforces.csv
<model-id>_stresses.csv
README.md
```
CSV names are canonical `<model-id>_*.csv` names. `README.md` is mandatory for later bundles;
`metadata.json` is optional. A quantity CSV may be omitted only when the upstream acceptance
contract explicitly records N/A and gives its verification replacement. Missing required files
or required Reference Model Contract provenance keep that model at `needs-reference-artifacts`.
## Reference Metadata Contract
This document is the required source of truth for the following metadata. A later bundle may
optionally duplicate it in `metadata.json` using at least this schema:
```json
{
"feature_id": "linear-static-3d-euler-beam",
"model_id": "<model-id>",
"artifact_status": "needs-reference-artifacts | ready-for-verification",
"input_file": "model.inp",
"abaqus_version": "<exact generator/version>",
"generation_owner": "<person or approved procedure>",
"generation_date": "<YYYY-MM-DD>",
"source_commit": "<commit>",
"units": "<consistent unit system>",
"coordinate_system": "<nodal and element result systems>",
"analysis_type": "single linear static",
"element_types": ["B33"],
"step_name": "Step-1",
"increment": 1,
"step_time": 1.0,
"output_requests": ["U", "RF", "SF"],
"reference_csv_schema_version": "<approved schema>",
"reference_csv_files": ["<canonical filenames>"],
"tolerance_policy": "<approved quantity/component policy>",
"limitations": ["<known limitations and explicit N/A quantities>"]
}
```
No agent may invent unknown provenance fields or mark a bundle ready merely because filenames
exist. An absent `metadata.json` is allowed. If the file exists, inventory it read-only and report
any disagreement with this contract or stored artifacts as an upstream contract/provenance issue.
Each later feature requirement names its exact input and comparison CSV paths, required row
identity/components and N/A quantities. Canonical filenames, `README.md`, `metadata.json`,
generator/version, provenance, duplicated unit/schema fields and a portfolio-wide directory layout
are not readiness gates unless that feature explicitly makes one part of its numerical comparison
contract. No agent may invent unknown provenance. If optional metadata exists, inspect it read-only
and report disagreement without rewriting the reference artifacts.
## Abaqus Reference CSV Requirements
Header comparison trims whitespace around each comma-separated field but does not rename fields.
For every file, `Frame` must normalize exactly from
`Increment 1: Step Time = 1.000` to `(Step-1, frame 0)`, `Part Instance Name` must resolve to the
preserved instance identity, `Node Label` must be a unique source-node station, and all projected
preserved instance identity, every declared source identity must be unique, and all projected
numeric values must be finite.
| exact legacy path | expected trimmed header | unique row key | observed inventory |
| --- | --- | --- | --- |
| `reference/cantilever beam/cantilever beam displacements.csv` | `Frame, Part Instance Name, Node Label, U-U1, U-U2, U-U3, UR-UR1, UR-UR2, UR-UR3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed |
| `reference/cantilever beam/cantilever beam reactions.csv` | `Frame, Part Instance Name, Node Label, RF-RF1, RF-RF2, RF-RF3, RM-RM1, RM-RM2, RM-RM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed |
| `reference/cantilever beam/cantilever beam elemental forces.csv` | `Frame, Part Instance Name, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3` | `(Frame, Part Instance Name, Node Label)` | 11 rows; header/key/finite/arity checks observed |
| `reference/cantilever beam/cantilever beam elemental forces.csv` | `Frame, Part Instance Name, Element Label, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3` | `(Frame, Part Instance Name, Element Label, Node Label)` | 20 rows; 10 B33 elements x 2 connectivity-matched endpoints; header/key/finite/arity checks observed |
The wide-row key becomes unique canonical component rows after adding `quantity` and `component`.
Missing, extra, duplicate, nonfinite, header/schema or identity mismatch stops comparison as
`needs-reference-artifacts` or `schema-mismatch`. No bad or near-zero row may be silently dropped.
For the elemental-force CSV, source node station is not an element-end identity. A boundary
station uses its only incident endpoint. An interior station may collapse exactly two endpoints
only when chain connectivity, section and local axes are consistent and the station has no
concentrated force/moment. The two positive-local-x section-cut values must first agree within the
approved component tolerance. If they agree, choose the endpoint with smaller stable internal
element ID; never average. Reversed orientation, branch, section jump, local-axis discontinuity or
loaded interior station requires an element-aware future schema and is a `schema-mismatch` under
this legacy schema.
For the elemental-force CSV, `(Part Instance Name, Element Label, Node Label)` is the source
element-endpoint identity. Each B33 element must have exactly two rows whose node labels match its
ordered input connectivity. Every row maps directly to the corresponding HDF5
`[element,endpoint,N/T/My/Mz]` row. Missing, extra, duplicate or connectivity-mismatched endpoint
identity fails as `schema-mismatch` before tolerance. The comparator does not collapse endpoints to
a node station, choose a representative or average values.
## Coverage Matrix
@@ -288,20 +240,20 @@ the canonical V0 step identity.
| verification quantity | requirement ids | model_id | FESA HDF5 dataset | legacy CSV and components | row identity/location | tolerance | verification method | status |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| nodal displacement/rotation | 003, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/displacement` | `reference/cantilever beam/cantilever beam displacements.csv`: `U-U1/U-U2/U-U3 -> UX/UY/UZ`, `UR-UR1/UR-UR2/UR-UR3 -> URX/URY/URZ` | preserved instance + source node; global nodal | `1e-9 + 1e-6*reference_scale` per displacement/rotation component | HDF5-to-read-only CSV after artifact/row-set precheck | ready for implementation planning; comparison not run |
| nodal reaction force | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RF-RF1/RF-RF2/RF-RF3 -> RF1/RF2/RF3` | preserved instance + source node; global nodal | `1e-3 + 1e-6*reference_scale` per force component | HDF5-to-read-only CSV plus global equilibrium | ready for implementation planning; comparison not run |
| nodal reaction moment | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RM-RM1/RM-RM2/RM-RM3 -> RM1/RM2/RM3` | preserved instance + source node; global nodal | `1e-3 + 1e-6*reference_scale` per moment component | HDF5-to-read-only CSV plus moment equilibrium | ready for implementation planning; comparison not run |
| section axial force | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SF-SF1 -> N` | positive-local-x endpoint projected to eligible source node station | `1e-3 + 1e-6*reference_scale` for `N` | endpoint consistency, deterministic station selection, HDF5-to-CSV | ready for implementation planning; comparison not run |
| section moments/torsion | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SM-SM1 -> My`, `SM-SM2 -> Mz`, `SM-SM3 -> T` | positive-local-x endpoint projected to eligible source node station; beam local | `1e-3 + 1e-6*reference_scale` separately for `My`, `Mz`, `T` | endpoint consistency, deterministic station selection, HDF5-to-CSV | ready for implementation planning; comparison not run |
| nodal displacement/rotation | 003, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/displacement` | `reference/cantilever beam/cantilever beam displacements.csv`: `U-U1/U-U2/U-U3 -> UX/UY/UZ`, `UR-UR1/UR-UR2/UR-UR3 -> URX/URY/URZ` | preserved instance + source node; global nodal | common translation/rotation family policy | HDF5-to-read-only CSV after artifact/row-set precheck | passed 2026-08-18 |
| nodal reaction force | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RF-RF1/RF-RF2/RF-RF3 -> RF1/RF2/RF3` | preserved instance + source node; global nodal | common reaction-force family policy | HDF5-to-read-only CSV plus global equilibrium | passed 2026-08-18 |
| nodal reaction moment | 007, 027, 029-031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/nodal/reaction` | `reference/cantilever beam/cantilever beam reactions.csv`: `RM-RM1/RM-RM2/RM-RM3 -> RM1/RM2/RM3` | preserved instance + source node; global nodal | common reaction-moment family policy | HDF5-to-read-only CSV plus moment equilibrium | passed 2026-08-18 |
| section axial force | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SF-SF1 -> N` | direct instance + source element + endpoint node; beam local | common section-force family policy | direct HDF5-to-endpoint CSV comparison | passed 2026-08-18 |
| section moments/torsion | 031, 036-042 | `cantilever-beam-b33` | `/steps/Step-1/frames/0/element/section_resultant` | `reference/cantilever beam/cantilever beam elemental forces.csv`: `SM-SM1 -> My`, `SM-SM2 -> Mz`, `SM-SM3 -> T` | direct instance + source element + endpoint node; beam local | common section-moment family policy | direct HDF5-to-endpoint CSV comparison | passed 2026-08-18 |
| equilibrium end action | 031, 035, 043 | analytical models and physics portfolio | `/steps/Step-1/frames/0/element/end_force_local` | Abaqus CSV N/A for direct outward-action comparison | element endpoint `xi=-1,+1`; local outward action `[FX,FY,FZ,MX,MY,MZ]` | analytical normalized `1e-12`; residual `1e-10` | unit/analytical end-sign tests and later physics sanity | planned |
| generalized strain/resultant | 029, 031, 035 | code and analytical models | `/steps/Step-1/frames/0/element/generalized_strain` and `/steps/Step-1/frames/0/element/generalized_resultant` | Abaqus CSV N/A | two Gauss points; beam local | matrix/formulation normalized `1e-12`, analytical relative `1e-9` | formulation/unit/HDF5 schema tests | planned |
| axial stress | 029, 032, 035 | axial/local-z analytical models | `/steps/Step-1/frames/0/element/stress_s11` | stress CSV N/A; Abaqus beam stress reference comparison N/A | element, Gauss point, input section point or `fesa-default` centroid | analytical relative `1e-9`; exact unit/row schema | unit/analytical recovery and HDF5 schema tests | planned; reference N/A |
For every matched reference group,
`reference_scale=max(abs(Abaqus reference rows))` for the same model, step/frame, quantity and
component, and `row_tolerance=absolute_floor+1e-6*reference_scale`. Abaqus values alone set the
scale. A zero scale uses only the floor. Every row decision and max absolute, component-scale
normalized, RMS, norm and worst-row/component metrics must be reported.
For every matched family, `S=max(abs(Abaqus reference rows))` uses only read-only reference values.
Rows with `abs(reference)<=0.01*S` use `error<=0.01*S`; other rows use relative error `<=0.05`;
the family also requires `RMS(error)/S<=0.01`. A zero-scale family requires exact-zero FESA values.
Every row branch/decision and family scale, near-zero band/count, max absolute error,
scale-relative RMS, worst row/component and zero-scale diagnostic must be reported.
### Complete must-requirement coverage
@@ -346,19 +298,20 @@ it does not waive the requirement.
| `FESA-REQ-LS3DEB-033` | `smoke-b33-cli`, diagnostic negative cases | CLI exit-code/field/order integration tests; CSV N/A |
| `FESA-REQ-LS3DEB-034` | every implementation model/test | per-step RED/GREEN/VERIFY and full MSVC/CTest evidence; model CSV N/A |
| `FESA-REQ-LS3DEB-035` | `NR-T01` through `NR-T11` and analytical inventory | exact numerical criteria in model records |
| `FESA-REQ-LS3DEB-036` | `cantilever-beam-b33` and comparison-policy unit fixtures | Abaqus-only component scale and exact formula tests |
| `FESA-REQ-LS3DEB-037` | same comparison fixtures, including zero-scale groups | exact SI floors and zero-scale tests |
| `FESA-REQ-LS3DEB-036` | `cantilever-beam-b33` and comparison-policy unit fixtures | exact family membership and Abaqus-only scale tests |
| `FESA-REQ-LS3DEB-037` | same comparison fixtures, including boundary and zero-scale families | near-zero, relative-row, scale-relative RMS and zero-scale tests |
| `FESA-REQ-LS3DEB-038` | malformed reference cases in `neg-b33-input-contract` | pre-tolerance fail-fast and no-clamp/no-drop tests |
| `FESA-REQ-LS3DEB-039` | verification-report schema fixture | every row decision and aggregate/worst metrics test |
| `FESA-REQ-LS3DEB-039` | verification-report schema fixture | every row branch/decision and family aggregate/worst metrics test |
| `FESA-REQ-LS3DEB-040` | exact legacy Artifact Bundle Contract | inventory plus `git diff --exit-code -- reference/` process check |
| `FESA-REQ-LS3DEB-041` | approved artifact precheck | four files, B33, exact headers, unique keys, finite values |
| `FESA-REQ-LS3DEB-042` | three comparison quantities and station normalization | exact component mapping, endpoint consistency and no-average tests |
| `FESA-REQ-LS3DEB-042` | three comparison quantities and direct endpoint projection | exact component mapping plus missing/extra/duplicate/connectivity-mismatch endpoint tests |
| `FESA-REQ-LS3DEB-043` | gate audit; later physics portfolio | reference execution N/A at this step; enforce build/test -> comparison -> physics -> release order |
| `FESA-REQ-LS3DEB-044` | process/Git diff audit and limitations review | no reference execution/mutation and no out-of-scope support claim |
## Artifact Acceptance Checklist
Read-only inventory inspection on `2026-08-09` established the following pre-implementation facts:
The following is historical 2026-08-09 pre-implementation inventory evidence. Its 11-row
elemental-force observation is superseded by the 2026-08-18 regenerated 20-row endpoint file:
- all four exact legacy paths exist;
- the input declares `TYPE=B33`;
@@ -370,6 +323,11 @@ Read-only inventory inspection on `2026-08-09` established the following pre-imp
- absent `metadata.json` is allowed by project-wide policy; legacy `README.md` and stress CSV are accepted N/A exceptions;
- no reference value was recalculated and no comparison was performed.
Read-only inspection on `2026-08-18` confirms that the elemental-force CSV has 20 rows,
exactly two connectivity-matched endpoint rows for each of the 10 B33 elements. Displacement and
reaction CSVs remain 11-row source-node tables. The fresh comparison passed on 2026-08-18;
current metrics are recorded in `reference-comparison.md`.
Before an actual comparison, tooling must repeat all artifact checks, verify the exact Frame and
instance identities, compare the complete projected row sets, and stop on any failure. The Step AC
must also show no working-tree diff under `reference/`. Passing this checklist is not a reference
@@ -383,8 +341,8 @@ comparison pass.
model.
- `NR-O01` (official oneMKL PARDISO contract) and `NR-O02` (deterministic duplicate-reduction
algorithm) remain implementation-planning inputs, not reference artifact defects.
- Future reversed, branched, loaded-interior or section-jump reference models require an
element-aware canonical CSV schema; the legacy node-station schema must not be generalized.
- Future reversed, branched, loaded-interior or section-jump reference models use the same
element-endpoint identity and require their own approved reference coverage.
### Implementation Planning Agent
@@ -403,8 +361,8 @@ artifacts for code/analytical fixtures.
### Reference Verification Agent
Run `ARTIFACT CHECK -> HDF5 ROW PROJECTION -> EXACT ROW-SET CHECK -> ENDPOINT CONSISTENCY ->
COMPONENT-SCALE COMPARE -> REPORT`. Use only the exact legacy files and the HDF5 paths/component
Run `ARTIFACT CHECK -> HDF5 ROW PROJECTION -> EXACT ROW-SET CHECK -> DIRECT ENDPOINT MATCH ->
COMMON FAMILY-SCALE COMPARE -> REPORT`. Use only the exact legacy files and the HDF5 paths/component
mappings in the Coverage Matrix. Do not synthesize `SF2/SF3` or stress rows, clamp values, omit
rows, average interior endpoints, or change the approved tolerance.
+7 -1
View File
@@ -19,11 +19,17 @@
- reference_model_id: `cantilever-beam-b33`
- reference_schema: `abaqus-cae-report-csv-v0`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b`
- status: `ready-for-release`
- historical_status: `ready-for-release`
- current_status: `superseded-pending-revalidation`
- superseded_on: `2026-08-18`
- owner_agent: `release-agent`
- date: `2026-08-10`
- final_review_source_head: `b7a1258ce0f36a85b888e23470cf9d936a7595cd`
The readiness evidence below is historical. The 2026-08-18 build/reference revalidation passed,
but fresh physics and release audits are still required before a current `ready-for-release`
verdict may be issued.
This is an internal feature-readiness verdict. It authorizes no publish, deploy, package, tag,
push, external release, or reference-artifact operation.
@@ -10,7 +10,7 @@
- approved_design: `docs/superpowers/specs/2026-08-08-linear-static-3d-euler-beam-design.md`
- approval_basis: design `status: approved`, user approval on `2026-08-08`, and amendment on `2026-08-09`
- source_formulation: `docs/linear-static-3d-euler-beam/formulation.md`
- reference_baseline: `reference/cantilever beam/` at source commit `2b34d0b`
- reference_baseline: exact read-only artifacts under `reference/cantilever beam/`
## Purpose
@@ -113,7 +113,7 @@ compatibility, 새로운 수학 계약 또는 새로운 구현 정책을 추가
- nodal_displacement: required, global six components, HDF5-to-Abaqus CSV comparison
- reaction: required, global six components plus global force/moment equilibrium
- equilibrium_end_action: required, local endpoint six components; unit/analytical and physics tests
- section_resultant: required, endpoint `[N,T,My,Mz]`, node-station-normalized reference comparison
- section_resultant: required, endpoint `[N,T,My,Mz]`, element-endpoint reference comparison
- generalized_strain_and_resultant: required at two Gauss points; formulation and schema tests
- stress: axial `S11` required; Abaqus reference comparison N/A
- residual: required, free-DOF and normalized global equilibrium checks
@@ -122,21 +122,22 @@ compatibility, 새로운 수학 계약 또는 새로운 구현 정책을 추가
## Tolerance Policy
- **FESA-REQ-LS3DEB-036** — The approved B33 reference comparison shall group rows by the same model, step/frame, quantity, and component, compute `reference_scale` only from read-only Abaqus values, and apply `absolute_floor + 1e-6 * reference_scale` to every matched row.
- **FESA-REQ-LS3DEB-037**For the approved SI bundle, displacement and rotation shall use `absolute_floor=1e-9`, force and moment shall use `absolute_floor=1e-3`, and a zero component scale shall use the applicable absolute floor alone.
- **FESA-REQ-LS3DEB-036** — The approved B33 reference comparison shall group rows into translation, rotation, reaction-force, reaction-moment, section-force and section-moment families and compute each family scale `S=max(abs(reference))` only from read-only Abaqus values.
- **FESA-REQ-LS3DEB-037**A matched row with `abs(reference)<=0.01*S` shall pass when `abs(fesa-reference)<=0.01*S`; every other row shall pass when its relative error is at most `0.05`; each family shall also satisfy `RMS(error)/S<=0.01`. No independent absolute-error gate is used, and a zero-scale family passes only when every FESA value is exactly zero.
- **FESA-REQ-LS3DEB-038** — Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation.
- **FESA-REQ-LS3DEB-039** — The verification report shall record every row decision and maximum absolute error, component-scale normalized error, RMS error, norm error, and worst row/component for each compared quantity.
- **FESA-REQ-LS3DEB-039** — The verification report shall record every row decision and comparison branch, family identity/components, reference scale, near-zero band/count, maximum absolute error, scale-relative RMS, worst row/component and zero-scale diagnostic for each compared family.
## Reference Artifact Requirements
The approved logical model is `cantilever-beam-b33`, schema is
`abaqus-cae-report-csv-v0`, source commit is `2b34d0b`, generator is
The approved logical model is `cantilever-beam-b33`. The input header records
`Abaqus/CAE Learning Edition 2024`, and the coordinate/output contract is global Cartesian
nodal output plus beam-local section-force output at `Step-1`, increment 1, step time 1.0.
The historical `abaqus-cae-report-csv-v0` and source commit `2b34d0b` inventory is not a
readiness requirement for the regenerated elemental-force CSV.
- **FESA-REQ-LS3DEB-040** — The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle.
- **FESA-REQ-LS3DEB-041** — Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start.
- **FESA-REQ-LS3DEB-042** — Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and node-station-normalized section resultant by `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; adjacent interior endpoints shall first agree within approved tolerance and shall not be averaged to hide a mismatch.
- **FESA-REQ-LS3DEB-042** — Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and section resultants by direct `(instance, element label, endpoint node label, component)` identity using `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; missing, extra, duplicate or connectivity-mismatched endpoint rows shall fail before tolerance and shall not be collapsed or averaged.
- **FESA-REQ-LS3DEB-043** — Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations.
- **FESA-REQ-LS3DEB-044** — FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior.
@@ -180,13 +181,13 @@ nodal output plus beam-local section-force output at `Step-1`, increment 1, step
| FESA-REQ-LS3DEB-033 | The CLI shall support `fesa.exe <model.inp> --output <results.h5>`, default output to the current directory's `results.h5`, use exit codes `0=success`, `2=usage`, `3=input`, `4=model`, `5=solver`, `6=HDF5`, and emit `severity`, `code`, `file`, `line`, `keyword`, `entity_identity`, and `message` diagnostics to stderr in deterministic order. | output | Stabilize automation and failure classification. | Approved design §10 | must | CLI integration and diagnostic ordering tests | Default/explicit output works and every failure class returns its exact code and complete ordered fields. | Exact codes, fields, and order | io-definition-agent; implementation-planning-agent | approved |
| FESA-REQ-LS3DEB-034 | Every production C++ behavior shall be developed in one step as GoogleTest `RED -> GREEN -> VERIFY`, with a related C++ test file, focused CTest evidence, full MSVC x64 Debug build/CTest evidence, at least one discovered test, and no new warning under the FESA target's `/W4 /WX` policy. | verification | Enforce project TDD and warning policy. | Approved design §§11.1, 11.4; ADR-012 | must | Implementation report; build/CTest logs | The related test fails first, then focused/full tests pass, discovery finds tests, and FESA emits no warning. | Zero test failures and new warnings | implementation-planning-agent; implementation-agent; build-test-executor-agent | approved |
| FESA-REQ-LS3DEB-035 | Numerical tests shall satisfy normalized `1e-12` for stiffness symmetry and two-point-Gauss/closed-form agreement, normalized `1e-10` for rigid-mode and linear-system residual, and relative `1e-9` for analytical solutions, while checking six rigid modes, rank 6, positive deformation energy, transformation orthogonality/energy invariance, prescribed-displacement recovery, and axial/torsion/two-plane bending benchmarks. | verification | Detect sign, integration, rank, and transform defects. | Approved design §§11.2, 11.3; formulation §18 | must | Unit, analytical, and orchestration tests | Every listed invariant and analytical case passes at its stated threshold. | `1e-12` matrix; `1e-10` residual; `1e-9` analytical | formulation-agent; numerical-review-agent; implementation-planning-agent | approved |
| FESA-REQ-LS3DEB-036 | The approved B33 reference comparison shall group rows by the same model, step/frame, quantity, and component, compute `reference_scale` only from read-only Abaqus values, and apply `absolute_floor + 1e-6 * reference_scale` to every matched row. | tolerance | Give zero and nonzero rows one deterministic rule. | Approved design §11.3; ADR-014 | must | Comparison unit/integration test; report review | Every group uses the Abaqus-only maximum absolute scale and every matched row uses the exact formula. | Relative coefficient `1e-6` | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-037 | For the approved SI bundle, displacement and rotation shall use `absolute_floor=1e-9`, force and moment shall use `absolute_floor=1e-3`, and a zero component scale shall use the applicable absolute floor alone. | tolerance | Preserve dimensional meaning near zero. | Approved design §11.3; ADR-014 | must | Comparison tests with zero/near-zero rows | Each quantity uses its exact SI floor and zero-scale groups use no relative contribution. | SI `1e-9` displacement/rotation; `1e-3` force/moment | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-036 | The approved B33 reference comparison shall group rows into translation, rotation, reaction-force, reaction-moment, section-force and section-moment families and compute `S=max(abs(reference))` from read-only Abaqus values. | tolerance | Give zero-like components a dimensionally compatible reference scale. | Common tolerance design; ADR-022 | must | Comparison unit/integration test; report review | Every family uses the exact declared components and Abaqus-only maximum scale. | Family scale | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-037 | Near-zero rows shall use the `0.01*S` fallback, other rows relative error `0.05`, and every family scale-relative RMS `0.01`; zero-scale families require exact-zero FESA values and no independent absolute gate is used. | tolerance | Stabilize zero-like rows while retaining row and aggregate checks. | Common tolerance design; ADR-022 | must | Boundary, zero-scale and RMS comparison tests | Every row and family uses the exact common constants and branch rules without NaN/Inf. | `0.01`, `0.05`, `0.01` | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-038 | Reference values shall not be zero-clamped and rows shall not be dropped; missing, extra, duplicate, nonfinite, schema-mismatched, or identity-mismatched rows shall fail before tolerance evaluation. | tolerance | Prevent false passes through omission or clamping. | Approved design §§11.3, 12; ADR-014 | must | Negative comparison tests | Every listed invalid case fails before numeric comparison and zero values remain unchanged. | No ignored invalid rows | reference-verification-agent | approved |
| FESA-REQ-LS3DEB-039 | The verification report shall record every row decision and maximum absolute error, component-scale normalized error, RMS error, norm error, and worst row/component for each compared quantity. | tolerance | Make the pass/fail decision auditable. | Approved design §11.3; ADR-014 | must | Verification report schema/review | Per-row decisions and all required aggregate/worst metrics are present for every quantity. | Report completeness | reference-verification-agent; release-agent | approved |
| FESA-REQ-LS3DEB-039 | The verification report shall record every row branch/decision, family identity/components, scale, near-zero band/count, maximum absolute error, scale-relative RMS, worst row/component and zero-scale diagnostic. | tolerance | Make the pass/fail decision auditable. | Common tolerance design; ADR-022 | must | Verification report schema/review | Per-row decisions and all required family metrics are present without nonfinite report values. | Report completeness | reference-verification-agent; release-agent | approved |
| FESA-REQ-LS3DEB-040 | The V0 reference baseline shall use the exact read-only files `reference/cantilever beam/cantilever beam.inp`, `reference/cantilever beam/cantilever beam displacements.csv`, `reference/cantilever beam/cantilever beam reactions.csv`, and `reference/cantilever beam/cantilever beam elemental forces.csv` without rename, rewrite, correction, or restoration; absent `metadata.json` is allowed by project-wide policy and `README.md` is N/A for this approved legacy bundle. | reference | Protect the approved correctness baseline. | Approved design §12; ADR-010, ADR-014 | must | Artifact inventory; Git diff review | Exact filenames exist and no reference file is added, removed, renamed, or content-modified. | Exact path/content identity | reference-model-agent; reference-verification-agent; release-agent | approved |
| FESA-REQ-LS3DEB-041 | Before comparison, artifact validation shall confirm all four files, `TYPE=B33`, expected CAE report headers, unique row keys, and finite values; failure shall be classified as `needs-reference-artifacts` or `schema-mismatch` and comparison shall not start. | reference | Detect stale B31 or malformed evidence. | Approved design §§6.3, 12 | must | Artifact-check integration test | All checks pass before comparison and every failure uses an approved classification. | Exact inventory and schema | reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-042 | Reference verification shall compare displacement by source-node identity (`U1/U2/U3/UR1/UR2/UR3`), reaction by source-node identity (`RF1/RF2/RF3/RM1/RM2/RM3`), and node-station-normalized section resultant by `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; adjacent interior endpoints shall first agree within approved tolerance and shall not be averaged to hide a mismatch. | reference | Compare equivalent quantities despite legacy station rows. | Approved design §§8.2, 12 | must | Reference comparison test/report | All components match by source identity and interior endpoints pass before deterministic representative selection. | Requirements 036 and 037 policy | io-definition-agent; reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-042 | Reference verification shall compare displacement and reaction by source-node identity and section resultants by direct `(instance, element label, endpoint node label, component)` identity using `SF1->N`, `SM1->My`, `SM2->Mz`, `SM3->T`; endpoint rows shall not be collapsed or averaged. | reference | Preserve the source identity supplied by the element-endpoint CSV and HDF5. | Common tolerance design; ADR-022 | must | Reference comparison test/report | All 20 B33 endpoint rows map one-to-one and malformed endpoint identity fails before tolerance. | Requirements 036 and 037 policy | io-definition-agent; reference-model-agent; reference-verification-agent | approved |
| FESA-REQ-LS3DEB-043 | Reference comparison shall run only after build/test passes; physics sanity shall run only after reference comparison passes and shall check global force/moment equilibrium, reaction sign, displacement direction, symmetry, element section-force consistency, and normalized residual; release readiness shall require all prior gate evidence and known limitations. | governance | Keep numerical similarity distinct from physical/release approval. | Approved design §§11, 12, 13 | must | Gate evidence audit | Each downstream report cites the preceding pass and physics evidence covers all six checks before release review. | Relevant upstream tolerances | coordinator-agent; physics-evaluation-agent; release-agent | approved |
| FESA-REQ-LS3DEB-044 | FESA agents and Harness shall not execute Abaqus, Nastran, or another reference solver and shall not create, modify, rename, or restore reference artifacts; release documentation shall not claim support for any Out Of Scope behavior. | governance | Prevent baseline contamination and scope inflation. | Approved design §§2.2, 12; ADR-010 | must | Process audit; Git diff; release documentation review | No reference execution or artifact mutation occurs and every exclusion is recorded without a support claim. | N/A | coordinator-agent; reference-model-agent; release-agent | approved |
@@ -1540,11 +1540,11 @@ or drilling-energy warning is part of this check.
### 17.5 Reference-comparison boundary
Abaqus comparison uses only the declared full-integration S4 case and blocks only
on matched global `U1/U2/U3` rows under the fixed absolute criterion
`abs(fesa-reference) <= 1.0e-5`. `UR1/UR2/UR3` uses the same fixed absolute value but
an exceedance emits only a deterministic nonblocking warning. A reported reference
scale is diagnostic only and does not enter the MITC4 decision or normalization.
Abaqus comparison uses only the declared full-integration S4 case. Matched global
`U1/U2/U3` families are blocking and `UR1/UR2/UR3` families are warning-only. Both use
the common family-scale row rule (`S=max(abs(reference))`, near-zero ratio `0.01`, relative
tolerance `0.05`) and scale-relative RMS tolerance `0.01`. `S=0` requires exact-zero FESA
values and there is no independent absolute-error gate.
FESA `S4` and `S4R` inputs must produce the same internal numerical rows for identical
supported models while preserving distinct source metadata. This common-path property
is verified without consuming an S4R Abaqus artifact; Abaqus S4R is not an acceptance
@@ -1631,7 +1631,7 @@ research brief remain the project source of truth.
| `024-029` | deterministic element buffers, partitioned linear lifecycle, full-residual reaction | planning |
| `039-048` | nodal/global and shell/local recovery inventory, units, identities, physical shell energy | I/O schema |
| `049-057` | normalized invariants, patches, fixed drilling, declared S4 reference and equilibrium | Numerical Review/reference/physics |
| `058-064` | fixed absolute `1.0e-5`; U blocking and UR warning-only | reference verification |
| `058-064` | common family-scale row/RMS policy; U blocking and UR warning-only | reference verification |
| `065-072` | exact existing S4 paths, S4R reference non-consumption, immutability and displacement-only boundary | reference model |
### 20.1 Numerical Review revision traceability
@@ -1685,7 +1685,8 @@ items or an expanded reference portfolio as missing evidence.
without claiming element equivalence.
- Do not consume `reference/shellR/` in acceptance comparison; preserve S4R support
through source-mapping/common-kernel/metadata tests.
- Use the fixed absolute MITC4 tolerance `1.0e-5`; do not alter the separate B33 tolerance or add administrative metadata or portfolio gates.
- Use the project-wide common family-scale row/RMS tolerance; do not add an independent
absolute-error gate, administrative metadata or portfolio gates.
- Do not create, repair, rename, or run reference artifacts during this formulation
gate.
+26 -17
View File
@@ -36,8 +36,8 @@ The user approved the following I/O-specific decisions on `2026-08-12`:
retaining the existing common metadata, nodal-result, diagnostic, CLI, and
failure-atomicity conventions.
The approved requirements fix the drilling rule and the MITC4 fixed absolute
displacement tolerance `1.0e-5`.
The approved requirements fix the drilling rule and the project-wide common
family-scale reference tolerance.
Drilling calibration/output, `NR-O03`, `NR-O04`, bundle-administration metadata, and
an expanded reference portfolio are outside this contract.
@@ -515,22 +515,31 @@ Before tolerance evaluation:
Only `U1/U2/U3` affect pass/fail. `UR1/UR2/UR3` are always compared and reported but
can emit only an approved deterministic nonblocking warning.
For every matched displacement row:
Matched U rows form one blocking translation family and matched UR rows form one warning-only
rotation family. For each family:
```text
row_tolerance = 1.0e-5
row_pass = abs(fesa_value-reference_value) <= row_tolerance
S = max(abs(reference_value_i))
error_i = abs(fesa_value_i-reference_value_i)
near_zero_band = 0.01*S
if abs(reference_value_i) <= near_zero_band:
row_pass = error_i <= near_zero_band
else:
row_pass = error_i/abs(reference_value_i) <= 0.05
relative_rms = sqrt(mean(error_i^2))/S
family_pass = all(row_pass) and relative_rms <= 0.01
```
The `1.0e-5` value is in the user-consistent length unit for U and dimensionless for
UR. No reference or result value is zero-clamped and neither component scale nor a
row-specific denominator changes the fixed value. A reference scale may remain in the
report as diagnostic information only. U exceedance fails; UR exceedance emits a
deterministic warning only. The separate B33 mixed tolerance is unchanged.
Reference values alone define `S`; values are not zero-clamped or omitted and there is no
independent absolute-error gate. A zero-scale family passes only if every FESA value is exactly
zero, otherwise it fails without emitting NaN/Inf. U row or RMS exceedance fails; the same UR
exceedance emits a deterministic warning only.
The report records every U/UR row, blocking/nonblocking decision, absolute error,
fixed-tolerance-normalized error, RMS error, displacement/rotation vector-norm
error, worst source row/component, and every UR warning.
The report records every U/UR row, blocking/nonblocking decision, family scale,
near-zero branch, absolute and applicable row-relative error, scale-relative RMS,
worst source row/component, and every UR warning.
## 8. CLI and Diagnostics Contract
@@ -615,7 +624,7 @@ physics review, or release status follows from approval of this document alone.
| `031-038` | source-independent MITC4 identity, fixed drilling stabilization and full-integration policy; no drilling output | Implementation Planning tests |
| `039-048` | additive HDF5 v0 paths, mandatory quantities, location identity, atomic output | Reference Model and Implementation Planning |
| `049-057` | diagnostic/schema hooks and required verification-metric/physical-energy evidence | Numerical Review and planning |
| `058-064` | normalized U/UR rows, fixed absolute `1.0e-5`, blocking/warning behavior, report inventory | Reference Verification |
| `058-064` | common-policy U/UR families, blocking/warning behavior, report inventory | Reference Verification |
| `065-072` | exact current S4 paths, S4R reference non-consumption, immutability and displacement-only gate | Reference Model |
## 11. Open Issues and Downstream Handoff
@@ -623,8 +632,8 @@ physics review, or release status follows from approval of this document alone.
### 11.1 Numerical Review boundary
No I/O-owned calibration value remains open. Numerical Review shall verify the exact
fixed drilling rule, basic geometry predicates, required HDF5 inventory, and fixed
absolute MITC4 tolerance mapping. Drilling calibration/energy output, `NR-O03`, `NR-O04`, bundle
fixed drilling rule, basic geometry predicates, required HDF5 inventory, and common
family-scale row/RMS tolerance mapping. Drilling calibration/energy output, `NR-O03`, `NR-O04`, bundle
administration and reference-portfolio expansion are removed scope.
### 11.2 Reference Model Agent
@@ -632,7 +641,7 @@ administration and reference-portfolio expansion are removed scope.
- Write `docs/linear-static-mitc4-shell/reference-model.md` using
this exact keyword/HDF5/reference-row contract.
- Record only the two exact existing input/displacement pairs, comparison components,
HDF5 projection, source-row identity, fixed absolute MITC4 tolerance and immutability rule.
HDF5 projection, source-row identity, common family-scale tolerance and immutability rule.
- Treat reaction/stress artifacts as nonblocking review evidence and do not create
location-equivalence claims absent from this contract.
@@ -75,7 +75,7 @@ confirmed mathematical defect는 없다.
| `NR-O02` drilling-energy ratio | `removed from scope` | Drilling energy는 내부 quadratic identity일 뿐 physical energy나 mandatory output이 아니며 ratio/warning threshold도 요구하지 않는다. |
| `NR-O03` smooth-director calibration | `removed from scope` | Pairwise positive incident-normal orientation, finite/nonzero averaging 및 duplicate-node fold modeling이 승인된 exact predicate다. 별도 angle calibration은 gate가 아니다. |
| `NR-O04` distortion/warp calibration | `removed from scope` | Basic topology, finite/nonzero surface measure 및 required-point `J>0`가 승인된 predicate다. Quality sweep이나 cutoff는 gate가 아니다. |
| `NR-O05` U/UR tolerance | `resolved` | 모든 관련 문서가 sole S4 reference에 고정 절대오차 `1.0e-5`, U blocking, UR warning-only를 동일하게 정의한다. Reference scale은 판정에 사용하지 않으며 S4R은 reference gate가 아닌 common-path evidence다. |
| `NR-O05` U/UR tolerance | `resolved` | 모든 관련 문서가 sole S4 reference에 공통 family-scale row/RMS 규칙, U blocking, UR warning-only를 동일하게 정의한다. S4R은 reference gate가 아닌 common-path evidence다. |
이전의 `needs-reference-model` 판정에 포함됐던 canonical naming, README,
`metadata.json`, provenance, expanded portfolio 및 아직 없는 comparison result는 현재
@@ -223,9 +223,10 @@ confirmed mathematical defect는 없다.
not emitted, and `S13/S23` point stress is not synthesized. Different natural or
section locations are never averaged.
- Reference comparison first rejects missing, extra, duplicate, nonfinite or
identity-mismatched rows. Every U/UR row then uses fixed absolute tolerance
`1.0e-5`; no reference-scale decision term, zero clamp or row denominator is
introduced. U1/U2/U3 is blocking and UR1/UR2/UR3 is warning-only.
identity-mismatched rows. Every U/UR family then uses the common reference-only scale,
near-zero ratio `0.01`, row relative tolerance `0.05` and scale-relative RMS tolerance
`0.01`. `S=0` requires exact-zero FESA values; there is no zero clamp or independent
absolute-error gate. U1/U2/U3 is blocking and UR1/UR2/UR3 is warning-only.
- Source S4 and S4R select the same FESA MITC4 kernel/quadrature/recovery path while
preserving source type. This is an input mapping, not an Abaqus formulation,
integration, stabilization or recovery equivalence claim.
@@ -11,10 +11,39 @@
`docs/linear-static-mitc4-shell/reference-model.md`
- source_io_definition: `docs/linear-static-mitc4-shell/io.md`
- source_requirements: `docs/linear-static-mitc4-shell/requirements.md`
- status: `pass-for-physics-evaluation`
- historical_status: `pass-for-physics-evaluation`
- current_status: `pass-for-physics-evaluation`
- superseded_on: `2026-08-18`
- revalidated_on: `2026-08-18`
- owner_agent: `reference-verification-agent`
- date: `2026-08-13`
The fixed-absolute-tolerance evidence below is preserved as a historical record. ADR-022
supersedes that decision with the common family-scale row/RMS policy. The following revalidation
is the current reference-gate evidence.
## 2026-08-18 Common-policy Revalidation
`cmake --build .harness/build --config Debug` passed, followed by
`ctest --test-dir .harness/build -C Debug --output-on-failure`: 214/214 tests passed.
The S4 reference test generated
`.harness/build/reference/mitc4-shell-s4-comparison/comparison.json` from the declared
read-only input/CSV pair.
- exact canonical rows: 294
- blocking translation family: 147 rows, all row/RMS gates passed
- warning-only rotation family: 147 rows, all row/RMS gates passed; warnings 0
- identity/schema/nonfinite failures: 0
- overall verdict: `passed=true`
| family | rows | scale | near-zero rows | max absolute error | scale-relative RMS |
| --- | ---: | ---: | ---: | ---: | ---: |
| translation `U1/U2/U3` | 147 | `2.37408203e-5` | 122 | `1.903785349151439e-7` | `1.0332756650397166e-3` |
| rotation `UR1/UR2/UR3` | 147 | `7.60725743e-6` | 107 | `6.882854962740428e-8` | `2.9771372973060382e-3` |
All values above come from the generated JSON ledger. The historical report body starts below
and must not be read as the current identity/tolerance evidence.
The prerequisite build/test report has status `pass-for-reference-verification`.
This report verifies only the approved full-integration S4 displacement case. It
does not compare S4R artifacts, claim Abaqus formulation equivalence, approve
@@ -8,6 +8,7 @@
- status: `approved-for-implementation-planning`
- owner_agent: `reference-model-agent`
- date: `2026-08-13`
- comparison_revalidated_on: `2026-08-18`
- artifact_policy: `read-only-existing-files`
- authoritative_fesa_output: `results.h5`
@@ -80,26 +81,31 @@ zero-clamped and mismatched rows are not omitted, averaged, or synthesized.
## 5. Tolerance and decision rule
For every matched row:
Matched `U1/U2/U3` rows form one blocking translation family and matched
`UR1/UR2/UR3` rows form one warning-only rotation family. For each family:
```text
tolerance = 1.0e-5
absolute_error_i = abs(fesa_value_i - abaqus_value_i)
S = max(abs(abaqus_value_i))
absolute_error_i = abs(fesa_value_i-abaqus_value_i)
near_zero_band = 0.01*S
near-zero row: absolute_error_i <= near_zero_band
other row: absolute_error_i/abs(abaqus_value_i) <= 0.05
relative_rms = sqrt(mean(absolute_error_i^2))/S <= 0.01
```
The fixed `1.0e-5` value is expressed in the model's user-consistent length unit for
`U1/U2/U3` and is dimensionless for `UR1/UR2/UR3`. Neither a component reference scale
nor a row-specific denominator changes the value. Reference scale may be reported as
non-decision diagnostic information. The separate B33 mixed tolerance is unchanged.
The scale uses read-only Abaqus values only. No independent absolute-error gate, zero clamp or
row omission is permitted. A zero-scale family passes only when all FESA values are exactly zero;
otherwise it reports `zero-reference-scale-nonzero-error` without NaN/Inf. The U family controls
the verdict and the UR family uses the same numeric rule for deterministic warning-only evidence.
- Every matched `U1/U2/U3` row must satisfy `absolute_error_i <= tolerance`.
Any U exceedance fails that case and the feature reference comparison.
- `UR1/UR2/UR3` uses the same fixed value. Every exceedance produces a deterministic
warning containing the case, source row, component, error, and tolerance, but does
not change pass/fail.
- Every matched `U1/U2/U3` row and the translation family RMS must satisfy the common
row/RMS rule. Any U row or family-RMS exceedance fails the feature reference comparison.
- `UR1/UR2/UR3` uses the same family-scale row/RMS rule. Every row or family-RMS
exceedance produces deterministic warning evidence but does not change pass/fail.
The comparison report records every U/UR row decision, maximum absolute error,
fixed-tolerance-normalized error, RMS error, vector-norm error, worst source
The comparison report records every U/UR row decision, family scale, near-zero branch,
maximum absolute error, row relative error, scale-relative RMS, worst source
row/component, and every UR warning.
## 6. Coverage and handoff
+7 -1
View File
@@ -18,11 +18,17 @@
- source_physics_evaluation_report: `docs/linear-static-mitc4-shell/physics-evaluation.md`
- audited_source_head: `820ba30c717b3d0e113775608e20dfd5fbc05d53`
- audited_branch: `feat-linear-static-mitc4-shell`
- status: `ready-for-release`
- historical_status: `ready-for-release`
- current_status: `superseded-pending-revalidation`
- superseded_on: `2026-08-18`
- owner_agent: `release-agent`
- date: `2026-08-13`
- release_boundary: internal FESA feature release readiness only; no publish, deploy, package, tag, commit, or external release was performed
The readiness evidence below is historical. The 2026-08-18 common-policy reference revalidation
passed, but fresh physics and release audits are still required before a current
`ready-for-release` verdict may be issued.
## Release Scope
| item | included | excluded | notes |
@@ -184,13 +184,13 @@ Formulation, Numerical Review, I/O, Reference Model, Implementation Planning 및
## Tolerance Policy
- **FESA-REQ-LSMITC4-058** — Abaqus reference pass/fail shall apply only to matched global `U1/U2/U3` rows using the fixed absolute tolerance `abs(fesa-reference) <= 1.0e-5` for every row.
- **FESA-REQ-LSMITC4-059**MITC4 row tolerance and tolerance-normalized error shall not depend on `reference_scale`, a row-specific denominator, zero clamp or component magnitude; a reference scale may be reported only as non-decision diagnostic information.
- **FESA-REQ-LSMITC4-060**The fixed `1.0e-5` U tolerance is expressed in the model's user-consistent length unit and is independent of the approved B33 component-scale mixed tolerance.
- **FESA-REQ-LSMITC4-061** — Global `UR1/UR2/UR3` rows shall use the same fixed absolute value `1.0e-5` and shall be fully reported; an exceedance emits a deterministic nonblocking warning and never changes pass/fail.
- **FESA-REQ-LSMITC4-062** The fixed `1.0e-5` UR tolerance is dimensionless. No separate UR large-error or drilling-energy threshold is required.
- **FESA-REQ-LSMITC4-058** — Abaqus reference pass/fail shall apply only to matched global `U1/U2/U3` rows grouped as one blocking translation family with Abaqus-only scale `S=max(abs(reference))`.
- **FESA-REQ-LSMITC4-059**A U row with `abs(reference)<=0.01*S` shall pass when `abs(fesa-reference)<=0.01*S`; every other U row shall pass when its relative error is at most `0.05`; the U family shall also satisfy `RMS(error)/S<=0.01`.
- **FESA-REQ-LSMITC4-060**MITC4 shall use the project-wide family-scale policy without an independent absolute-error gate, zero clamp or row omission; a zero-scale family passes only when every FESA value is exactly zero.
- **FESA-REQ-LSMITC4-061** — Global `UR1/UR2/UR3` rows shall form one warning-only rotation family using the same near-zero, relative-row and scale-relative RMS constants; an exceedance emits a deterministic nonblocking warning and never changes pass/fail.
- **FESA-REQ-LSMITC4-062** — No separate UR large-error, fixed absolute or drilling-energy threshold is required; every row decision and family metric shall remain finite and auditable.
- **FESA-REQ-LSMITC4-063** — Missing, extra, duplicate, nonfinite, schema-mismatched or source-identity-mismatched rows shall fail artifact/schema validation before numeric tolerance evaluation for both U and UR inventories.
- **FESA-REQ-LSMITC4-064** — The comparison report shall record each U/UR row decision, maximum absolute error, fixed-tolerance-normalized error, RMS error, vector-norm error and worst source row/component; nonblocking UR warnings shall not be omitted from an otherwise passing report.
- **FESA-REQ-LSMITC4-064** — The comparison report shall record each U/UR row branch/decision, family identity/components, scale, near-zero band/count, maximum absolute error, scale-relative RMS, vector-norm diagnostic, worst source row/component and zero-scale diagnostic; nonblocking UR warnings shall not be omitted from an otherwise passing report.
## Reference Artifact Requirements
@@ -226,8 +226,8 @@ without gaps or overlap.
| `031-038` | 5-DOF physics embedded in 6-DOF with fixed drilling stabilization | numerical boundary | User approval; MITC literature and thesis 6-DOF discussion | must | Formulation review, invariant and rank tests | Exact `10^-3` positive rotational-diagonal rule; physical outputs exclude drilling | Fixed by Requirements 033-036 | Formulation; Numerical Review; Implementation Planning | approved |
| `039-048` | Mandatory HDF5 output and failure atomicity | output | User approval; ADR-005/016/018 | must | Recovery, schema, identity, nonfinite and atomicity tests | Every quantity/location/unit/identity exists; failure commits no partial success | Exact component/location inventory; I/O Definition owns schema | Formulation; I/O Definition; Implementation Planning | approved |
| `049-057` | TDD, invariants, patch, declared reference and physics | verification | User approval; shell formulation evidence; project process | must | CTest evidence, analytical/patch tests, the S4 reference case and physics review | Required tests pass; removed calibration/portfolio checks are not reintroduced | `1e-12` symmetry/frame; `1e-10` rigid/residual | Numerical Review; Implementation Planning | approved |
| `058-060` | Translational displacement pass/fail tolerance | tolerance | User approval; ADR-020 | must | Comparator unit/integration tests and report review | Every matched U row uses fixed absolute `1.0e-5` without scale, clamp or omission | Fixed by Requirements 058-060 | Reference Verification | approved |
| `061-062` | Rotational warning-only comparison | tolerance/warning | User approval; ADR-020 | must | Comparator/diagnostic tests and report review | UR never changes pass/fail; fixed absolute `1.0e-5` exceedance emits a deterministic warning | Fixed by Requirements 061-062 | Reference Verification | approved |
| `058-060` | Translational displacement pass/fail tolerance | tolerance | User approval; ADR-022 | must | Comparator unit/integration tests and report review | Every matched U row and U-family RMS use the common family-scale policy without clamp or omission | `0.01`, `0.05`, `0.01` | Reference Verification | approved |
| `061-062` | Rotational warning-only comparison | tolerance/warning | User approval; ADR-022 | must | Comparator/diagnostic tests and report review | UR never changes pass/fail; common-policy exceedance emits a deterministic warning | `0.01`, `0.05`, `0.01` | Reference Verification | approved |
| `063-064` | Row/schema failure and report completeness | reference verification | User approval; ADR-005/014/018 | must | Negative comparator and report-schema tests | Invalid inventory fails before numeric comparison; all U/UR metrics remain visible | No ignored invalid rows | I/O Definition; Reference Verification | approved |
| `065-068` | Exact S4 reference-case inventory, S4R exclusion and row validity | reference | User declaration; ADR-019 | must | Read-only S4 inventory, source-row/component precheck and S4R non-consumption test | Two declared S4 paths exist; required rows are unique, finite and deterministically mapped; S4R artifacts are not required or consumed | Requirements `058-063` | Reference Model; Reference Verification | approved |
| `069-071` | S4 reference coverage and displacement-only comparison | reference | User approval | must | HDF5-to-CSV comparison | Declared S4 case only; U blocks and UR only warns; S4R mapping remains independently tested | Requirements `058-064` | Reference Verification; Physics Evaluation | approved |
@@ -270,7 +270,7 @@ and tangent derivation may remain in the formulation document.
- Record only the exact existing S4 input/displacement CSV paths from Requirement 065 as acceptance artifacts and keep every existing reference artifact read-only.
- Do not consume the S4R bundle in reference verification; route S4R source support to parser/common-kernel/HDF5 tests from Requirement 066.
- Define only the HDF5-to-CSV source-node/component projection and the approved fixed absolute MITC4 tolerance `1.0e-5`; do not add bundle administration or portfolio gates.
- Define only the HDF5-to-CSV source-node/component projection and the approved common family-scale tolerance; do not add bundle administration or portfolio gates.
### Implementation Planning Agent
@@ -7,11 +7,15 @@
- status: `approved`
- approved_by: user
- approved_on: `2026-08-08`
- amended_on: `2026-08-09`
- amended_on: `2026-08-18`
- source_formulation: `docs/linear-static-3d-euler-beam/formulation.md`
- reference_baseline: `reference/cantilever beam/` from source commit `2b34d0b`
- reference_baseline: exact read-only artifacts under `reference/cantilever beam/`
- implementation_environment: C++17, MSVC, CMake, CTest, GoogleTest, Intel oneMKL, Intel oneTBB, HDF5
2026-08-18 amendment: the former node-station projection and mixed absolute-floor tolerance are
superseded by the direct element-endpoint identity and common family-scale row/RMS policy in
`2026-08-17-common-reference-tolerance-design.md` and ADR-022.
## 1. 목적
이 설계는 Abaqus `.inp` keyword subset을 읽어 2절점 3차원
@@ -288,8 +292,8 @@ node label 또는 node set이며 assembly-level set의 `INSTANCE` parameter를
Abaqus B31은 transverse shear deformation을 포함하는 Timoshenko beam이고 B33은
2절점 cubic EulerBernoulli beam이다. FESA V0는 `TYPE=B33`만 Euler 요소로 매핑한다.
`TYPE=B31``unsupported-element-formulation` 오류로 거부한다.
`reference/cantilever beam/cantilever beam.inp`source commit `2b34d0b`에서
`TYPE=B33`으로 생성된 승인 reference input이다. Reference artifact check는 비교 전에
`reference/cantilever beam/cantilever beam.inp``TYPE=B33`으로 생성된 승인 reference
input이다. Reference artifact check는 비교 전에
이 element type을 다시 확인하며 B31로 되돌아간 입력이나 결과를 허용하지 않는다.
이 결정은 [[Abaqus Structural Element Families]], [[Beam and Frame Finite Elements]],
@@ -370,19 +374,20 @@ Abaqus internal-force CSV는 equilibrium end action이 아니라 section resulta
| `SM1` | `MY` |
| `SM2` | `MZ` |
승인된 B33 reference의 `cantilever beam elemental forces.csv`element label 없이
`Frame`, `Part Instance Name`, `Node Label`, `SF1`, `SM1`, `SM2`, `SM3` 기록한다.
승인된 B33 reference의 `cantilever beam elemental forces.csv``Frame`,
`Part Instance Name`, `Element Label`, `Node Label`, `SF1`, `SM1`, `SM2`, `SM3`
기록한다.
따라서 이 파일의 comparison은 다음 규칙을 사용한다.
- Reference row key는 `(Step-1, frame 0, instance name, source node label, component)`다.
- Reference row key는 `(Step-1, frame 0, instance name, source element label,
endpoint source node label, component)`다.
단일 step invariant에 따라 CSV의 `Increment 1: Step Time = 1.000`
`(Step-1, frame 0)`으로 정규화한다.
- FESA endpoint section resultant를 동일한 section-cut 부호로 변환한 뒤 source node
station으로 정규화한다. Interior node의 두 인접 element endpoint 값은 먼저 승인된
tolerance 안에서 서로 일치해야 한다. 불일치는 평균으로 숨기지 않고
`tolerance-failure`로 보고한다.
- Interior endpoint가 일치하면 stable internal element ID가 작은 endpoint를 대표 row로
선택한다. Boundary node는 하나의 endpoint를 사용한다.
- 각 B33 element에는 input connectivity의 두 source node label과 일치하는 endpoint row가
정확히 두 개 있어야 하며, 각 row를 해당 HDF5 element endpoint에 직접 대응한다.
Missing, extra, duplicate 또는 connectivity mismatch는 tolerance 전에 실패한다.
- Comparator는 같은 source node의 인접 endpoint를 node station으로 축약하거나 대표값을
선택하거나 평균하지 않는다.
- 이 bundle에서 비교하는 mapping은 `SF1 -> N`, `SM1 -> My`, `SM2 -> Mz`,
`SM3 -> T`다. CSV에 없는 `SF2``SF3`는 reference comparison 대상이 아니며,
FESA equilibrium end action의 transverse force는 unit/analytical test와 physics sanity로
@@ -496,8 +501,8 @@ GoogleTest target에는 FESA warning policy를 강제하지 않는다.
- axial, torsion, y/z bending cantilever analytical cases
- HDF5 schema, identity, component, metadata, atomic finalization
- CLI `.inp -> results.h5` integration
- 승인된 B33 CSV header/Frame 정규화, node-station matching, interior endpoint 일치 검사
- zero 및 near-zero reference row를 포함한 component-scale 혼합 허용오차 검사
- 승인된 B33 CSV header/Frame 정규화와 element-endpoint direct identity 검사
- zero 및 near-zero reference row를 포함한 공통 family-scale 상대오차/RMS 검사
### 11.3 수치 tolerance
@@ -506,37 +511,33 @@ GoogleTest target에는 FESA warning policy를 강제하지 않는다.
| matrix symmetry and Gauss/closed-form comparison | normalized `1e-12` |
| rigid-mode and linear-system residual | normalized `1e-10` |
| analytical solution tests | relative `1e-9` |
| Abaqus B33 reference comparison | component-scale relative `1e-6` |
| SI displacement and rotation absolute floor | `1e-9` |
| SI force and moment absolute floor | `1e-3` |
| Abaqus reference row relative error | `0.05` |
| near-zero threshold ratio | reference family scale의 `0.01` |
| reference family scale-relative RMS | `0.01` |
Reference row 판정은 zero-reference에서도 의미가 있도록 component-scale 혼합
허용오차를 사용한다. 같은 model, step/frame, quantity, component의 Abaqus reference
rows에 대해 다음 값을 계산한다.
Reference row 판정은 같은 model, step/frame, quantity, component로 구성한 reference
family별 공통 규칙을 사용한다. Reference 값으로만 family scale을 계산한다.
```text
reference_scale = max(abs(reference_value_i))
row_tolerance = absolute_floor + 1e-6 * reference_scale
row_pass = abs(fesa_value_i - reference_value_i) <= row_tolerance
S = max(abs(reference_value_i))
near_zero_i = abs(reference_value_i) <= 0.01 * S
row_pass_i = abs(fesa_value_i - reference_value_i) / abs(reference_value_i) <= 0.05
if not near_zero_i
row_pass_i = abs(fesa_value_i - reference_value_i) <= 0.01 * S
if near_zero_i
relative_rms = sqrt(mean((fesa_value_i - reference_value_i)^2)) / S
family_pass = all(row_pass_i) and relative_rms <= 0.01
```
- Scale은 displacement, rotation, force, moment 및 각 component를 섞지 않는다.
- Family는 displacement, rotation, force, moment 및 각 component를 섞지 않는다.
- Scale은 read-only Abaqus reference 값만 사용하며 FESA 결과로 조정하지 않는다.
- Scale이 0이면 absolute floor만 적용한다.
- `S == 0`이면 모든 FESA 값이 정확히 0일 때만 통과한다. 하나라도 0이 아니면
`zero-reference-scale-nonzero-error`로 실패한다.
- 모든 row를 개별 판정하며 reference 값을 zero-clamp하거나 row를 제거하지 않는다.
- Missing/extra row, nonfinite value, schema 또는 identity mismatch는 tolerance 계산 전에
실패한다.
- Report는 max absolute error, component-scale normalized error, RMS error, norm error,
worst row/component를 기록한다.
현재 B33 `SM1`의 reference scale은 `1.0e7`이고 moment row tolerance는
`1.0e-3 + 1.0e-6 * 1.0e7 = 10.001`이다. 자유단 reference residue `-1.56e-2`
FESA의 이론적 zero 사이 오차 `1.56e-2`는 이 기준을 통과하지만, 예를 들어 `100`
자유단 moment 오차는 실패한다.
승인된 cantilever bundle은 SI absolute floor를 사용한다. 이후 SI가 아닌 reference를
추가하면 quantity별 absolute floor를 해당 단위로 변환하고 변환 근거를 metadata와
verification report에 기록해야 한다.
- Report는 family scale, max absolute error, max row relative error, scale-relative RMS,
worst row/component와 판정 reason을 기록한다. 독립적인 absolute-error gate는 없다.
### 11.4 공통 build/test command
@@ -561,9 +562,10 @@ CTest discovery 결과는 한 개 이상의 test를 포함해야 한다.
## 12. Reference 및 release gate
`reference/cantilever beam/`을 V0의 승인된 Abaqus B33 reference baseline으로 사용한다.
논리 model ID는 `cantilever-beam-b33`, reference schema ID는
`abaqus-cae-report-csv-v0`이며 source baseline은 commit `2b34d0b`이다. Input header가
기록한 generator는 `Abaqus/CAE Learning Edition 2024`다. 모델은 SI 단위계, global
논리 model ID는 `cantilever-beam-b33`이다. 기존 `abaqus-cae-report-csv-v0`과 source
commit `2b34d0b` 기록은 2026-08-09 당시 inventory의 역사적 provenance이며, 2026-08-18
regenerated elemental-force CSV의 현재 readiness gate가 아니다. Input header가 기록한
generator는 `Abaqus/CAE Learning Edition 2024`다. 모델은 SI 단위계, global
Cartesian nodal output, beam local section-force output, `Step-1`, increment 1,
step time 1.0 계약으로 해석한다.
@@ -588,8 +590,8 @@ Reference verification은 build/test gate 통과 후 승인 input으로 FESA `re
- nodal displacement: CSV `U1/U2/U3/UR1/UR2/UR3`과 HDF5 displacement
- nodal reaction: CSV `RF1/RF2/RF3/RM1/RM2/RM3`과 HDF5 reaction
- section resultant: CSV `SF1/SM1/SM2/SM3`node station으로 정규화한 HDF5
section resultant
- section resultant: CSV `SF1/SM1/SM2/SM3`element label 및 endpoint node label로
직접 대응한 HDF5 section resultant
- stress: 명시적 N/A
Artifact check는 네 파일의 존재, B33 element type, expected header, 유일한 row key,
@@ -3,11 +3,16 @@
## Status
- date: `2026-08-12`
- amended_on: `2026-08-18`
- status: `approved`
- scope: FESA project policy, agents, skills, and `linear-static-mitc4-shell` upstream contracts
- implementation_code: out of scope
- reference_artifact_mutation: prohibited
2026-08-18 amendment: the former MITC4 fixed absolute tolerance is superseded by the common
family-scale row/RMS policy in `2026-08-17-common-reference-tolerance-design.md` and ADR-022.
The independent-solver, minimal-reference-case and drilling decisions remain unchanged.
## Decision
FESA is an independent finite element solver. It accepts an approved subset of the
@@ -84,19 +89,23 @@ Only matched global `U1/U2/U3` rows are blocking. `UR1/UR2/UR3` rows are compare
reported as warning-only evidence. Reactions and stresses may be inspected but are not
reference pass/fail quantities.
For every matched U/UR row:
The project-wide common policy now evaluates each matched U/UR component family:
```text
tolerance = 1.0e-5
S = max(abs(reference_value_i))
near_zero_i = abs(reference_value_i) <= 0.01 * S
ordinary_row_pass_i = abs(error_i) / abs(reference_value_i) <= 0.05
near_zero_row_pass_i = abs(error_i) <= 0.01 * S
relative_rms = sqrt(mean(error_i^2)) / S <= 0.01
```
The fixed value is expressed in the model's user-consistent length unit for translation
and as a dimensionless rotation value for warning-only UR. Reference scale may remain
diagnostic report data but does not enter the tolerance or normalized-error denominator.
Values are not zero-clamped. Missing, duplicate, unmatched, or nonfinite required rows
fail before numeric tolerance evaluation. U tolerance failure fails reference
verification; UR tolerance exceedance emits a deterministic warning only. The B33 mixed
tolerance remains unchanged.
The scale is computed from read-only reference values in the same model, step/frame,
quantity and component family. If `S == 0`, only exact-zero FESA values pass; otherwise
the reason is `zero-reference-scale-nonzero-error`. There is no independent absolute-error
gate. Values are not zero-clamped. Missing, duplicate, unmatched, or nonfinite required
rows fail before numeric tolerance evaluation. U tolerance failure fails reference
verification; UR tolerance exceedance emits a deterministic warning only. B33 uses the
same row and family-RMS thresholds.
No additional locking, convergence, distorted, curved, director-angle, or invalid
geometry reference portfolio is required for this MITC4 implementation acceptance.
@@ -147,7 +156,7 @@ For MITC4:
- fixed drilling stabilization closes `NR-O01` and removes `NR-O02` by scope;
- `NR-O03` and `NR-O04` are removed from the approved verification scope;
- the fixed absolute MITC4 tolerance `1.0e-5` closes the blocking part of `NR-O05`;
- the common family-scale row/RMS policy closes the blocking part of `NR-O05`;
- missing heavyweight reference metadata is `not-required-by-policy`;
- confirmed formulation defects remain blocking.
@@ -0,0 +1,227 @@
# Common Reference Tolerance Policy Design
## Status
- date: `2026-08-17`
- status: `implemented-and-verified`
- verified_on: `2026-08-18`
- scope: 모든 FESA 외부 reference comparison
- production_solver_behavior: 변경 없음
- reference_artifact_mutation: 금지
현재 운영 상수, 검증 순서, report schema와 변경 관리는 `docs/TOLERANCE.md`를 source of
truth로 사용한다. 이 문서는 승인 결정의 설계 근거와 대안 검토를 보존한다.
## 목적
B33, MITC4 및 이후 추가되는 기능이 서로 다른 수치 tolerance 식을 만들지 않도록 공통
reference comparison 규칙을 정의한다. 이 정책은 Abaqus CSV와 FESA HDF5에서 결정적으로
대응된 행을 비교하는 외부 reference gate에만 적용한다. Element formulation, analytical
solution, matrix symmetry, residual 및 physics sanity tolerance는 이 정책의 범위가 아니다.
## 결정
모든 reference comparison은 다음 세 개의 무차원 상수를 사용한다.
```text
near_zero_ratio = 0.01
relative_tolerance = 0.05
relative_rms_tolerance = 0.01
```
독립적인 absolute-error gate는 두지 않는다. Absolute error는 reference 값이 0에 가까워
행별 상대오차를 안정적으로 계산할 수 없는 경우의 대체 판정과 진단에만 사용한다.
## Comparison Family와 Scale
Tolerance scale은 개별 component가 아니라 동일한 물리 차원의 component family마다
계산한다. 하나의 family는 최소한 다음 identity가 모두 같은 행으로 제한한다.
- model/reference case
- step 및 frame
- logical quantity
- unit dimension
- coordinate system
- blocking 또는 warning-only behavior
서로 다른 quantity나 단위 차원의 값은 scale을 공유하지 않는다. Family scale은 대응된
reference 값만 사용해 계산한다.
\[
S_g = \max_{i \in g}|R_i|
\]
여기서 \(g\)는 comparison family, \(R_i\)는 원본 reference 값이다. Reference 값은 scale
계산 전후에 clamp, rewrite 또는 생략하지 않는다.
현재 기능의 family mapping은 다음과 같다.
| feature | logical quantity / dimension | components | behavior |
| --- | --- | --- | --- |
| B33 | displacement / length | `UX`, `UY`, `UZ` | blocking |
| B33 | displacement / rotation | `URX`, `URY`, `URZ` | blocking |
| B33 | reaction / force | `RF1`, `RF2`, `RF3` | blocking |
| B33 | reaction / moment | `RM1`, `RM2`, `RM3` | blocking |
| B33 | section resultant / force | `N` | blocking |
| B33 | section resultant / moment | `T`, `My`, `Mz` | blocking |
| MITC4 | displacement / length | `U1`, `U2`, `U3` | blocking |
| MITC4 | displacement / rotation | `UR1`, `UR2`, `UR3` | warning-only |
새 quantity는 해당 기능의 requirements와 reference-model 계약에서 logical quantity,
dimension, components 및 behavior를 명시해 family에 배치한다. 물리 차원이 같다는 이유만으로
서로 다른 logical quantity를 자동으로 합치지 않는다.
### B33 element-endpoint identity amendment
2026-08-18에 사용자가 다시 생성한
`reference/cantilever beam/cantilever beam elemental forces.csv``Element Label`
`Node Label`을 모두 포함하고 B33 요소마다 두 endpoint 행을 제공한다. Section-resultant
comparison은 다음 key로 각 CSV 행을 HDF5 `[element, endpoint, component]` 행에 직접
대응시킨다.
```text
(model_id, step/frame, instance_name, source_element_label,
source_node_label, quantity, component)
```
Component mapping은 `SF-SF1 -> N`, `SM-SM1 -> My`, `SM-SM2 -> Mz`,
`SM-SM3 -> T`다. Comparator는 이 CSV를 source-node station으로 collapse하거나 인접
endpoint를 평균하지 않는다. Missing, extra, duplicate 또는 connectivity-mismatched
element-endpoint identity는 tolerance 전에 실패한다. Production
`ResultRecovery::NormalizeSectionResultantsToNodeStations` 계약과 그 단위 테스트는 이 외부
reference identity 변경의 범위가 아니다.
## 행별 판정
각 대응 행에 대해 다음 absolute error를 계산한다.
\[
E_i = |F_i-R_i|
\]
Family scale이 양수이면 near-zero band는 다음과 같다.
\[
Z_g = 0.01S_g
\]
행은 다음 두 분기 중 정확히 하나로 판정한다.
1. \(|R_i| \le Z_g\)이면 near-zero 행이다. 이 행은 \(E_i \le Z_g\)일 때 통과한다.
2. \(|R_i| > Z_g\)이면 일반 상대오차 행이다. 이 행은
\(E_i/|R_i| \le 0.05\)일 때 통과한다.
따라서 absolute error는 모든 행에 적용되는 별도 acceptance gate가 아니다. Near-zero
분기는 0 또는 0에 가까운 reference 행에서 정의되지 않거나 과도하게 증폭되는 raw relative
error를 대체한다.
### Zero-scale family
\(S_g=0\)이면 해당 family의 모든 reference 값이 정확히 0이다. 사용자가 독립 absolute
floor를 제외했으므로 이 경우에는 다음 fail-closed 규칙을 사용한다.
- 모든 FESA 값도 정확히 0이면 모든 행과 family relative RMS가 통과한다.
- 하나라도 0이 아닌 FESA 값이 있으면 해당 행과 family가 실패한다.
Comparator는 이 경우 infinity 또는 NaN을 report에 기록하지 않고
`zero-reference-scale-nonzero-error` 진단을 기록한다.
## Scale-relative RMS
Family 전체의 평균적인 오차 수준은 scale-relative RMS로 판정한다.
\[
\operatorname{relative\_rms}_g =
\frac{\sqrt{\frac{1}{N_g}\sum_{i \in g} E_i^2}}{S_g}
\]
\[
\operatorname{relative\_rms}_g \le 0.01
\]
이 값은 raw row-relative error의 RMS가 아니다. Error RMS를 같은 family의 reference 최대
scale로 정규화한 무차원 값이다. 행별 gate가 국부적인 큰 오차를 검출하고, relative RMS
gate가 family 전체의 평균 오차를 검출한다. 두 gate를 모두 통과해야 한다.
## 판정 순서
1. 필요한 artifact와 schema를 확인한다.
2. Source identity 및 component를 결정적으로 대응시킨다.
3. Missing, extra, duplicate, identity-mismatched 또는 nonfinite required row를 tolerance 전에
실패시킨다.
4. Comparison family와 reference-only scale을 계산한다.
5. 모든 행에 near-zero 또는 일반 상대오차 판정을 적용한다.
6. Family scale-relative RMS를 계산한다.
7. Blocking family는 모든 행과 RMS가 통과해야 reference gate를 통과한다.
8. Warning-only family의 동일한 실패는 deterministic warning을 생성하지만 전체 reference
verdict를 실패시키지 않는다.
Tolerance 정책은 feature별 blocking/warning-only quantity 선택을 변경하지 않는다.
## Report Contract
Comparator report는 최소한 다음 항목을 보존한다.
- family identity와 component 목록
- reference scale와 near-zero band
- 전체 행 수와 near-zero 행 수
- 각 행의 원본 FESA/reference 값, absolute error, 적용된 분기 및 판정
- 일반 상대오차 행의 relative error
- family scale-relative RMS와 판정
- maximum absolute error와 worst row를 진단 정보로 기록하되 독립 gate로 사용하지 않음
- blocking 실패와 warning-only exceedance의 구분
## 현재 B33/MITC4 Evidence
MSVC Debug build와 214개 CTest를 통과한 현재 comparator가 다음 ledger를 생성했다.
Reference artifact는 수정하지 않았으며 B33은 regenerated 20-row element-endpoint identity를
직접 비교한다.
- `.harness/build/reference/cantilever-beam-b33/comparison.json`
- `.harness/build/reference/mitc4-shell-s4-comparison/comparison.json`
| case | row gate | maximum family relative RMS | RMS gate | verdict |
| --- | --- | ---: | --- | --- |
| B33 direct endpoint | 모든 212 blocking 행 통과 | 약 `1.17e-8` | 통과 | 통과 |
| MITC4 U | 모든 blocking 행 통과 | 약 `1.03e-3` | 통과 | 통과 |
| MITC4 UR | 모든 warning-only 행 통과 | 약 `2.98e-3` | 통과 | warning 없음 |
개별 component scale을 사용하면 MITC4 `U1/U2`의 최대 reference가 약 `3.12e-23`이어서
near-zero 행 판정과 relative RMS가 실패한다. Translation family의 `U1/U2/U3`가 공통
scale을 사용하면 물리적으로 zero-like인 in-plane residue를 실제 translational response와
같은 차원에서 판정할 수 있다.
## 검토한 대안
### Component별 scale
규칙은 단순하지만 component 전체가 zero-like이면 수치 residue 자체가 scale이 된다. 현재
MITC4 `U1/U2`가 실패하므로 채택하지 않는다.
### Reference RMS를 분모로 한 relative L2 norm
`sqrt(sum(error^2)/sum(reference^2))`는 일반적인 상대 norm이지만 zero-dominant component나
family에서 분모가 불안정하거나 정의되지 않는다. Near-zero 행 규칙과 동일한 scale 의미를
공유하지 않으므로 공통 정책으로 채택하지 않는다.
### Near-zero 보정 row-relative error의 RMS
각 행의 분모를 `max(abs(reference), near_zero_band)`로 바꿔 RMS를 계산할 수 있다. 그러나
행별 near-zero 판정을 다시 집계해 같은 오차를 중복 평가하고 해석이 복잡해지므로 채택하지
않는다.
## 구현 및 계약 정렬 범위
이 문서의 승인은 목표 tolerance 정책을 고정하지만 기존 comparator와 기능 문서를 즉시
변경하지 않는다. 후속 implementation plan은 TDD로 다음을 함께 정렬해야 한다.
- 공통 comparison policy와 metric 구현
- B33 및 MITC4 comparator의 공통 정책 사용
- positive, boundary, zero-scale, near-zero, nonfinite 및 row-identity 테스트
- B33와 MITC4 requirements, numerical review, reference model, I/O 및 reference comparison
문서의 tolerance 부분
- `AGENTS.md``docs/ADR.md`의 기존 B33/MITC4 tolerance 결정을 새 공통 정책으로 대체
기존 reference input과 CSV는 생성, rename, rewrite 또는 보정하지 않는다. Production solver
formulation, HDF5 output schema, component mapping, blocking/warning-only 분류 및 physics gate도
이 tolerance 변경으로 수정하지 않는다.
@@ -1,12 +1,21 @@
Frame,Part Instance Name, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3
Increment 1: Step Time = 1.000,PART-1_1-1,1,0.000000000E+00,1.000000000E+07,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,10,0.000000000E+00,1.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,11,0.000000000E+00,-1.560000000E-02,0.000000000E+00,0.000000000E+00
Frame,Part Instance Name, Element Label, Node Label, SF-SF1, SM-SM1, SM-SM2, SM-SM3
Increment 1: Step Time = 1.000,PART-1_1-1,1,1,0.000000000E+00,1.000000000E+07,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,1,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,2,2,0.000000000E+00,9.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,2,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,3,3,0.000000000E+00,8.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,3,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,4,4,0.000000000E+00,7.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,4,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,5,5,0.000000000E+00,6.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,5,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,6,6,0.000000000E+00,5.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,6,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,7,7,0.000000000E+00,4.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,7,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,8,8,0.000000000E+00,3.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,8,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,9,9,0.000000000E+00,2.000000000E+06,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,9,10,0.000000000E+00,9.999998750E+05,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,10,10,0.000000000E+00,9.999998750E+05,0.000000000E+00,0.000000000E+00
Increment 1: Step Time = 1.000,PART-1_1-1,10,11,0.000000000E+00,-1.562500000E-02,0.000000000E+00,0.000000000E+00
1 Frame Part Instance Name Element Label Node Label SF-SF1 SM-SM1 SM-SM2 SM-SM3
2 Increment 1: Step Time = 1.000 PART-1_1-1 1 1 0.000000000E+00 1.000000000E+07 0.000000000E+00 0.000000000E+00
3 Increment 1: Step Time = 1.000 PART-1_1-1 1 2 0.000000000E+00 9.000000000E+06 0.000000000E+00 0.000000000E+00
4 Increment 1: Step Time = 1.000 PART-1_1-1 2 3 2 0.000000000E+00 8.000000000E+06 9.000000000E+06 0.000000000E+00 0.000000000E+00
5 Increment 1: Step Time = 1.000 PART-1_1-1 2 4 3 0.000000000E+00 7.000000000E+06 8.000000000E+06 0.000000000E+00 0.000000000E+00
6 Increment 1: Step Time = 1.000 PART-1_1-1 3 5 3 0.000000000E+00 6.000000000E+06 8.000000000E+06 0.000000000E+00 0.000000000E+00
7 Increment 1: Step Time = 1.000 PART-1_1-1 3 6 4 0.000000000E+00 5.000000000E+06 7.000000000E+06 0.000000000E+00 0.000000000E+00
8 Increment 1: Step Time = 1.000 PART-1_1-1 4 7 4 0.000000000E+00 4.000000000E+06 7.000000000E+06 0.000000000E+00 0.000000000E+00
9 Increment 1: Step Time = 1.000 PART-1_1-1 4 8 5 0.000000000E+00 3.000000000E+06 6.000000000E+06 0.000000000E+00 0.000000000E+00
10 Increment 1: Step Time = 1.000 PART-1_1-1 5 9 5 0.000000000E+00 2.000000000E+06 6.000000000E+06 0.000000000E+00 0.000000000E+00
11 Increment 1: Step Time = 1.000 PART-1_1-1 5 10 6 0.000000000E+00 1.000000000E+06 5.000000000E+06 0.000000000E+00 0.000000000E+00
12 Increment 1: Step Time = 1.000 PART-1_1-1 6 11 6 0.000000000E+00 -1.560000000E-02 5.000000000E+06 0.000000000E+00 0.000000000E+00
13 Increment 1: Step Time = 1.000 PART-1_1-1 6 7 0.000000000E+00 4.000000000E+06 0.000000000E+00 0.000000000E+00
14 Increment 1: Step Time = 1.000 PART-1_1-1 7 7 0.000000000E+00 4.000000000E+06 0.000000000E+00 0.000000000E+00
15 Increment 1: Step Time = 1.000 PART-1_1-1 7 8 0.000000000E+00 3.000000000E+06 0.000000000E+00 0.000000000E+00
16 Increment 1: Step Time = 1.000 PART-1_1-1 8 8 0.000000000E+00 3.000000000E+06 0.000000000E+00 0.000000000E+00
17 Increment 1: Step Time = 1.000 PART-1_1-1 8 9 0.000000000E+00 2.000000000E+06 0.000000000E+00 0.000000000E+00
18 Increment 1: Step Time = 1.000 PART-1_1-1 9 9 0.000000000E+00 2.000000000E+06 0.000000000E+00 0.000000000E+00
19 Increment 1: Step Time = 1.000 PART-1_1-1 9 10 0.000000000E+00 9.999998750E+05 0.000000000E+00 0.000000000E+00
20 Increment 1: Step Time = 1.000 PART-1_1-1 10 10 0.000000000E+00 9.999998750E+05 0.000000000E+00 0.000000000E+00
21 Increment 1: Step Time = 1.000 PART-1_1-1 10 11 0.000000000E+00 -1.562500000E-02 0.000000000E+00 0.000000000E+00
+2
View File
@@ -76,6 +76,8 @@ target_link_libraries(
add_executable(
fesa_reference_tests
reference/reference_tolerance_policy.cpp
reference/reference_tolerance_policy_test.cpp
reference/reference_comparison.cpp
reference/reference_comparison_test.cpp
reference/b33_reference_comparison_test.cpp
@@ -26,8 +26,8 @@
namespace {
constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11";
constexpr std::size_t kExpectedRowCount = 176U;
constexpr std::size_t kExpectedMetricCount = 16U;
constexpr std::size_t kExpectedRowCount = 212U;
constexpr std::size_t kExpectedMetricCount = 6U;
class Hdf5Handle {
public:
@@ -192,7 +192,10 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) {
report.rows.begin(), report.rows.end(),
[](const fesa::test::RowDecision& row) {
return row.passed && std::isfinite(row.absolute_error) &&
std::isfinite(row.tolerance) && row.tolerance > 0.0 &&
std::isfinite(row.tolerance) && row.tolerance >= 0.0 &&
std::isfinite(row.reference_scale) &&
std::isfinite(row.near_zero_band) &&
std::isfinite(row.relative_error) &&
row.fesa.model_id == "cantilever-beam-b33" &&
row.reference.model_id == "cantilever-beam-b33" &&
row.fesa.step_name == "Step-1" &&
@@ -206,11 +209,11 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) {
std::all_of(report.metrics.begin(), report.metrics.end(),
[](const fesa::test::ComponentMetrics& metric) {
return std::isfinite(metric.reference_scale) &&
std::isfinite(metric.near_zero_band) &&
std::isfinite(metric.maximum_absolute_error) &&
std::isfinite(metric.maximum_normalized_error) &&
std::isfinite(metric.rms_error) &&
std::isfinite(metric.norm_error) &&
metric.maximum_normalized_error <= 1.0;
std::isfinite(metric.relative_rms) &&
metric.rms_passed && metric.passed &&
metric.row_count > 0U && !metric.components.empty();
}));
EXPECT_FALSE(report.stress_comparison_applicable);
@@ -234,6 +237,9 @@ TEST(B33ReferenceComparison, GeneratesAuthoritativeHdf5AndComparisonEvidence) {
EXPECT_NE(json.find("\"stress_comparison_applicable\":false"),
std::string::npos);
EXPECT_NE(json.find("\"physics_evidence\""), std::string::npos);
EXPECT_NE(json.find("\"source_element_label\""), std::string::npos);
EXPECT_NE(json.find("\"family_identity\""), std::string::npos);
EXPECT_NE(json.find("\"row_count\""), std::string::npos);
std::vector<std::string> generated_names;
for (const auto& entry :
+30 -7
View File
@@ -122,7 +122,7 @@ void ExpectCommonMetadata(const fesa::test::Mitc4ComparisonReport& report,
void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) {
ASSERT_EQ(report.rows.size(), kNodeCount * kComponentCount);
ASSERT_EQ(report.metrics.size(), kComponentCount);
ASSERT_EQ(report.metrics.size(), 2U);
ASSERT_EQ(report.vector_metrics.size(), kNodeCount);
EXPECT_TRUE(report.passed);
const std::size_t blocking_rows = static_cast<std::size_t>(std::count_if(
@@ -135,12 +135,35 @@ void ExpectComparisonCoverage(const fesa::test::Mitc4ComparisonReport& report) {
[](const fesa::test::Mitc4RowDecision& row) {
return !row.blocking || row.within_tolerance;
}));
EXPECT_EQ(report.warnings.size(),
static_cast<std::size_t>(
std::count_if(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) {
return !row.blocking && !row.within_tolerance;
})));
const auto rotation_metric =
std::find_if(report.metrics.begin(), report.metrics.end(),
[](const fesa::test::Mitc4ComponentMetrics& metric) {
return !metric.blocking;
});
ASSERT_NE(rotation_metric, report.metrics.end());
const std::size_t row_warning_count = static_cast<std::size_t>(
std::count_if(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) {
return !row.blocking && !row.within_tolerance;
}));
ASSERT_EQ(report.warnings.size(),
row_warning_count + (rotation_metric->rms_passed ? 0U : 1U));
std::size_t warning_index = 0U;
for (std::size_t row_index = 0U; row_index < report.rows.size();
++row_index) {
const auto& row = report.rows[row_index];
if (!row.blocking && !row.within_tolerance) {
EXPECT_EQ(report.warnings[warning_index].code,
"rotation-reference-exceedance");
EXPECT_EQ(report.warnings[warning_index].row, row_index);
++warning_index;
}
}
if (!rotation_metric->rms_passed) {
EXPECT_EQ(report.warnings[warning_index].code,
"rotation-reference-rms-exceedance");
EXPECT_EQ(report.warnings[warning_index].row, rotation_metric->worst_row);
}
}
// MITC4-E2E-S4-001
+154 -79
View File
@@ -24,6 +24,8 @@
#include <utility>
#include <vector>
#include "reference_tolerance_policy.h"
namespace fesa::test {
namespace {
@@ -32,7 +34,6 @@ constexpr const char* kDisplacementPath =
constexpr const char* kInternalFormulation = "FESA-MITC4";
constexpr const char* kIntegrationRule =
"2x2x2-gauss; mitc4-edge-midpoint-shear";
constexpr double kFixedAbsoluteTolerance = 1.0e-5;
constexpr std::array<const char*, 6> kComponents{"U1", "U2", "U3",
"UR1", "UR2", "UR3"};
const std::vector<std::string> expected_header{"Part Instance Name",
@@ -648,6 +649,18 @@ std::string FiniteText(const double value) {
return stream.str();
}
const char* BranchName(const ReferenceToleranceBranch branch) {
switch (branch) {
case ReferenceToleranceBranch::kNearZero:
return "near-zero";
case ReferenceToleranceBranch::kRelative:
return "relative";
case ReferenceToleranceBranch::kZeroScaleExact:
return "zero-scale-exact";
}
return "unknown";
}
std::string JsonEscape(const std::string& value) {
std::ostringstream stream;
for (const unsigned char character : value) {
@@ -687,6 +700,18 @@ std::string JsonEscape(const std::string& value) {
return stream.str();
}
void WriteJsonStrings(std::ostream& stream,
const std::vector<std::string>& values) {
stream << '[';
for (std::size_t index = 0U; index < values.size(); ++index) {
if (index != 0U) {
stream << ',';
}
stream << '"' << JsonEscape(values[index]) << '"';
}
stream << ']';
}
} // namespace
Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare(
@@ -713,14 +738,6 @@ Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare(
}
}
std::array<double, 6> scales{};
for (const auto& row : reference_rows) {
for (std::size_t component = 0U; component < scales.size(); ++component) {
scales[component] =
(std::max)(scales[component], std::abs(row.values[component]));
}
}
Mitc4ComparisonReport report{};
report.case_id = reference_case.case_id;
report.source_element_type = std::move(hdf5.source_element_type);
@@ -729,80 +746,126 @@ Result<Mitc4ComparisonReport> Mitc4ReferenceComparison::Compare(
report.passed = true;
report.rows.reserve(hdf5.rows.size() * kComponents.size());
report.vector_metrics.reserve(hdf5.rows.size());
std::array<double, 6> error_norms{};
std::array<double, 6> maximum_errors{};
std::array<double, 6> maximum_normalized{};
std::array<std::size_t, 6> worst_rows{};
double global_worst_normalized = -1.0;
std::array<std::vector<std::size_t>, 2> family_rows;
for (const auto& hdf5_row : hdf5.rows) {
const auto reference = reference_by_identity.find(hdf5_row.identity);
if (reference == reference_by_identity.end()) {
Fail("schema-mismatch", "A projected reference row is missing.");
}
std::array<double, 6> errors{};
for (std::size_t component = 0U; component < kComponents.size();
++component) {
const double tolerance = kFixedAbsoluteTolerance;
const double absolute_error = std::abs(
hdf5_row.values[component] - reference->second->values[component]);
const double normalized_error = absolute_error / tolerance;
if (!std::isfinite(tolerance) || !(tolerance > 0.0) ||
!std::isfinite(absolute_error) ||
!std::isfinite(normalized_error)) {
Fail("schema-mismatch",
"A finite row produced a nonfinite comparison metric.");
}
const bool blocking = component < 3U;
const bool within_tolerance = absolute_error <= tolerance;
const std::size_t row_index = report.rows.size();
report.rows.push_back(
{reference_case.case_id, hdf5_row.identity.instance_name,
hdf5_row.identity.source_node_label, kComponents[component],
hdf5_row.values[component], reference->second->values[component],
absolute_error, tolerance, normalized_error, blocking,
within_tolerance});
errors[component] = absolute_error;
error_norms[component] =
std::hypot(error_norms[component], absolute_error);
if (normalized_error > maximum_normalized[component]) {
maximum_normalized[component] = normalized_error;
maximum_errors[component] = absolute_error;
worst_rows[component] = row_index;
}
if (normalized_error > global_worst_normalized) {
global_worst_normalized = normalized_error;
report.worst_row = row_index;
}
if (blocking && !within_tolerance) {
report.passed = false;
} else if (!blocking && !within_tolerance) {
const std::string message =
"case=" + reference_case.case_id +
";instance=" + hdf5_row.identity.instance_name +
";node=" + std::to_string(hdf5_row.identity.source_node_label) +
";component=" + kComponents[component] +
";absolute_error=" + FiniteText(absolute_error) +
";tolerance=" + FiniteText(tolerance);
report.warnings.push_back(
{"rotation-reference-exceedance", row_index, message});
}
report.rows.push_back({reference_case.case_id,
hdf5_row.identity.instance_name,
hdf5_row.identity.source_node_label,
kComponents[component],
hdf5_row.values[component],
reference->second->values[component],
0.0,
0.0,
0.0,
0.0,
0.0,
false,
{},
blocking,
false});
family_rows[blocking ? 0U : 1U].push_back(row_index);
}
report.vector_metrics.push_back(
{hdf5_row.identity.instance_name, hdf5_row.identity.source_node_label,
std::hypot(errors[0U], errors[1U], errors[2U]),
std::hypot(errors[3U], errors[4U], errors[5U])});
}
report.metrics.reserve(kComponents.size());
const double row_count = static_cast<double>(hdf5.rows.size());
for (std::size_t component = 0U; component < kComponents.size();
++component) {
std::vector<ReferenceToleranceFamily> families;
families.reserve(2U);
for (std::size_t family_index = 0U; family_index < family_rows.size();
++family_index) {
ReferenceToleranceFamily family{};
family.identity = family_index == 0U ? "displacement-translation"
: "displacement-rotation";
family.components = family_index == 0U
? std::vector<std::string>{"U1", "U2", "U3"}
: std::vector<std::string>{"UR1", "UR2", "UR3"};
family.blocking = family_index == 0U;
family.rows.reserve(family_rows[family_index].size());
for (const std::size_t row_index : family_rows[family_index]) {
const auto& row = report.rows[row_index];
family.rows.push_back({row.fesa_value, row.reference_value});
}
families.push_back(std::move(family));
}
const auto evaluation = ReferenceTolerancePolicy::Evaluate(families);
if (!evaluation.valid || evaluation.families.size() != families.size()) {
Fail("schema-mismatch",
"A finite row produced a nonfinite comparison metric.");
}
report.metrics.reserve(evaluation.families.size());
double maximum_absolute_error = -1.0;
for (std::size_t family_index = 0U;
family_index < evaluation.families.size(); ++family_index) {
const auto& policy = evaluation.families[family_index];
for (std::size_t local = 0U; local < family_rows[family_index].size();
++local) {
const std::size_t row_index = family_rows[family_index][local];
auto& row = report.rows[row_index];
const auto& decision = policy.rows[local];
row.absolute_error = decision.absolute_error;
row.tolerance = decision.threshold;
row.normalized_error = decision.relative_error;
row.reference_scale = policy.reference_scale;
row.near_zero_band = policy.near_zero_band;
row.relative_error_applicable = decision.relative_error_applicable;
row.tolerance_branch = BranchName(decision.branch);
row.within_tolerance = decision.passed;
if (row.absolute_error > maximum_absolute_error) {
maximum_absolute_error = row.absolute_error;
report.worst_row = row_index;
}
if (!row.within_tolerance) {
if (row.blocking) {
report.passed = false;
} else {
const std::string message =
"case=" + reference_case.case_id +
";instance=" + row.instance_name +
";node=" + std::to_string(row.source_node_label) +
";component=" + row.component +
";absolute_error=" + FiniteText(row.absolute_error) +
";threshold=" + FiniteText(row.tolerance);
report.warnings.push_back(
{"rotation-reference-exceedance", row_index, message});
}
}
}
const std::size_t worst_row = family_rows[family_index][policy.worst_row];
report.metrics.push_back(
{kComponents[component], scales[component], kFixedAbsoluteTolerance,
maximum_errors[component], maximum_normalized[component],
error_norms[component] / std::sqrt(row_count),
error_norms[component], worst_rows[component]});
{policy.identity, policy.components, policy.blocking,
policy.reference_scale, policy.near_zero_band,
policy.near_zero_count, policy.maximum_absolute_error,
policy.relative_rms, worst_row, policy.rms_passed, policy.passed,
policy.diagnostic_code, policy.row_count});
if (policy.blocking && !policy.passed) {
report.passed = false;
}
if (!policy.blocking && !policy.rms_passed) {
report.warnings.push_back(
{"rotation-reference-rms-exceedance", worst_row,
"case=" + reference_case.case_id + ";family=" + policy.identity +
";relative_rms=" + FiniteText(policy.relative_rms)});
}
}
for (std::size_t node = 0U; node < hdf5.rows.size(); ++node) {
const std::size_t row_offset = node * kComponents.size();
report.vector_metrics.push_back(
{report.rows[row_offset].instance_name,
report.rows[row_offset].source_node_label,
std::hypot(report.rows[row_offset].absolute_error,
report.rows[row_offset + 1U].absolute_error,
report.rows[row_offset + 2U].absolute_error),
std::hypot(report.rows[row_offset + 3U].absolute_error,
report.rows[row_offset + 4U].absolute_error,
report.rows[row_offset + 5U].absolute_error)});
}
return Result<Mitc4ComparisonReport>::Success(std::move(report));
} catch (const ComparisonFailure& exception) {
@@ -845,8 +908,14 @@ Status Mitc4ReferenceComparison::WriteDeterministicJson(
<< "\",\"fesa_value\":" << row.fesa_value
<< ",\"reference_value\":" << row.reference_value
<< ",\"absolute_error\":" << row.absolute_error
<< ",\"tolerance\":" << row.tolerance
<< ",\"normalized_error\":" << row.normalized_error
<< ",\"threshold\":" << row.tolerance
<< ",\"relative_error\":" << row.normalized_error
<< ",\"reference_scale\":" << row.reference_scale
<< ",\"near_zero_band\":" << row.near_zero_band
<< ",\"relative_error_applicable\":"
<< (row.relative_error_applicable ? "true" : "false")
<< ",\"tolerance_branch\":\"" << JsonEscape(row.tolerance_branch)
<< "\""
<< ",\"blocking\":" << (row.blocking ? "true" : "false")
<< ",\"within_tolerance\":"
<< (row.within_tolerance ? "true" : "false") << '}';
@@ -857,15 +926,21 @@ Status Mitc4ReferenceComparison::WriteDeterministicJson(
stream << ',';
}
const auto& metric = report.metrics[index];
stream << "{\"component\":\"" << JsonEscape(metric.component)
<< "\",\"reference_scale\":" << metric.reference_scale
<< ",\"tolerance\":" << metric.tolerance
stream << "{\"family_identity\":\"" << JsonEscape(metric.family_identity)
<< "\",\"components\":";
WriteJsonStrings(stream, metric.components);
stream << ",\"blocking\":" << (metric.blocking ? "true" : "false")
<< ",\"reference_scale\":" << metric.reference_scale
<< ",\"near_zero_band\":" << metric.near_zero_band
<< ",\"near_zero_count\":" << metric.near_zero_count
<< ",\"row_count\":" << metric.row_count
<< ",\"maximum_absolute_error\":" << metric.maximum_absolute_error
<< ",\"maximum_normalized_error\":"
<< metric.maximum_normalized_error
<< ",\"rms_error\":" << metric.rms_error
<< ",\"vector_norm_error\":" << metric.vector_norm_error
<< ",\"worst_row\":" << metric.worst_row << '}';
<< ",\"relative_rms\":" << metric.relative_rms
<< ",\"worst_row\":" << metric.worst_row
<< ",\"rms_passed\":" << (metric.rms_passed ? "true" : "false")
<< ",\"passed\":" << (metric.passed ? "true" : "false")
<< ",\"diagnostic_code\":\"" << JsonEscape(metric.diagnostic_code)
<< "\"}";
}
stream << "],\"vector_metrics\":[";
for (std::size_t index = 0U; index < report.vector_metrics.size();
+14 -5
View File
@@ -29,19 +29,28 @@ struct Mitc4RowDecision {
double absolute_error;
double tolerance;
double normalized_error;
double reference_scale;
double near_zero_band;
bool relative_error_applicable;
std::string tolerance_branch;
bool blocking;
bool within_tolerance;
};
struct Mitc4ComponentMetrics {
std::string component;
std::string family_identity;
std::vector<std::string> components;
bool blocking;
double reference_scale;
double tolerance;
double near_zero_band;
std::size_t near_zero_count;
double maximum_absolute_error;
double maximum_normalized_error;
double rms_error;
double vector_norm_error;
double relative_rms;
std::size_t worst_row;
bool rms_passed;
bool passed;
std::string diagnostic_code;
std::size_t row_count{};
};
struct Mitc4VectorMetrics {
@@ -458,11 +458,11 @@ const fesa::test::Mitc4RowDecision* FindRow(
const fesa::test::Mitc4ComponentMetrics* FindMetric(
const fesa::test::Mitc4ComparisonReport& report,
const std::string& component) {
const std::string& family_identity) {
const auto found =
std::find_if(report.metrics.begin(), report.metrics.end(),
[&](const fesa::test::Mitc4ComponentMetrics& metric) {
return metric.component == component;
return metric.family_identity == family_identity;
});
return found == report.metrics.end() ? nullptr : &*found;
}
@@ -496,6 +496,9 @@ TEST(Mitc4ReferenceComparison,
EXPECT_EQ(report.rows[0U].component, "U1");
EXPECT_EQ(report.rows[5U].component, "UR3");
EXPECT_EQ(report.rows[6U].source_node_label, 2);
ASSERT_EQ(report.metrics.size(), 2U);
EXPECT_EQ(report.metrics[0U].row_count, 6U);
EXPECT_EQ(report.metrics[1U].row_count, 6U);
EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) {
return row.case_id == "shell-contract" &&
@@ -588,7 +591,7 @@ TEST(Mitc4ReferenceComparison, RejectsInvalidInventoryBeforeNumericComparison) {
// MITC4-REF-003
TEST(Mitc4ReferenceComparison,
AppliesFixedAbsoluteToleranceWithoutScaleClampOrRowDenominator) {
SharesTranslationScaleAcrossUComponentsAndUsesNearZeroBranches) {
ContractFixture fixture{"tolerance"};
auto reference = DefaultRows();
reference[0U].values[0U] = 0.0;
@@ -597,10 +600,10 @@ TEST(Mitc4ReferenceComparison,
reference[1U].values[1U] = 0.0;
WriteCsv(fixture.Csv(), reference);
auto fesa_values = reference;
fesa_values[0U].values[0U] = 0.9999e-5;
fesa_values[1U].values[0U] += 1.0001e-5;
fesa_values[0U].values[1U] = 0.9999e-5;
fesa_values[1U].values[1U] = 1.0001e-5;
fesa_values[0U].values[0U] = 0.999 * 8.0e-2;
fesa_values[1U].values[0U] += 1.001e-1;
fesa_values[0U].values[1U] = 0.999 * 8.0e-2;
fesa_values[1U].values[1U] = 1.001 * 8.0e-2;
WriteHdf5(fixture.Results(), fixture.Input(), fesa_values);
auto result =
@@ -616,25 +619,21 @@ TEST(Mitc4ReferenceComparison,
ASSERT_NE(u1_scaled, nullptr);
ASSERT_NE(u2_near, nullptr);
ASSERT_NE(u2_over, nullptr);
EXPECT_DOUBLE_EQ(u1_zero->tolerance, 1.0e-5);
EXPECT_DOUBLE_EQ(u1_zero->tolerance, 8.0e-2);
EXPECT_TRUE(u1_zero->within_tolerance);
EXPECT_NEAR(u1_zero->normalized_error, 0.9999, 1.0e-12);
EXPECT_FALSE(u1_zero->relative_error_applicable);
EXPECT_FALSE(u1_scaled->within_tolerance);
EXPECT_NEAR(u1_scaled->normalized_error, 1.0001, 2.0e-11);
EXPECT_DOUBLE_EQ(u2_near->tolerance, 1.0e-5);
EXPECT_NEAR(u1_scaled->normalized_error, 0.05005, 1.0e-12);
EXPECT_DOUBLE_EQ(u2_near->tolerance, 8.0e-2);
EXPECT_TRUE(u2_near->within_tolerance);
EXPECT_NEAR(u2_near->normalized_error, 0.9999, 1.0e-12);
EXPECT_FALSE(u2_near->relative_error_applicable);
EXPECT_FALSE(u2_over->within_tolerance);
EXPECT_NEAR(u2_over->normalized_error, 1.0001, 1.0e-12);
for (const auto& row : report.rows) {
EXPECT_DOUBLE_EQ(row.tolerance, 1.0e-5);
}
for (const auto& metric : report.metrics) {
EXPECT_DOUBLE_EQ(metric.tolerance, 1.0e-5);
}
const auto* u1_metric = FindMetric(report, "U1");
ASSERT_NE(u1_metric, nullptr);
EXPECT_DOUBLE_EQ(u1_metric->reference_scale, 2.0);
EXPECT_DOUBLE_EQ(u2_over->tolerance, 8.0e-2);
const auto* translation = FindMetric(report, "displacement-translation");
ASSERT_NE(translation, nullptr);
EXPECT_DOUBLE_EQ(translation->reference_scale, 8.0);
EXPECT_EQ(translation->components,
(std::vector<std::string>{"U1", "U2", "U3"}));
}
// MITC4-REF-004
@@ -650,8 +649,9 @@ TEST(Mitc4ReferenceComparison,
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
EXPECT_TRUE(report.passed);
ASSERT_EQ(report.warnings.size(), 1U);
ASSERT_EQ(report.warnings.size(), 2U);
EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance");
EXPECT_EQ(report.warnings[1U].code, "rotation-reference-rms-exceedance");
const auto* row = FindRow(report, 1, "UR1");
ASSERT_NE(row, nullptr);
EXPECT_FALSE(row->blocking);
@@ -663,6 +663,47 @@ TEST(Mitc4ReferenceComparison,
EXPECT_NE(report.warnings[0U].message.find("UR1"), std::string::npos);
}
TEST(Mitc4ReferenceComparison,
RotationRmsExceedanceWarnsWhenEveryRotationRowPasses) {
ContractFixture fixture{"rotation-rms"};
auto fesa_values = DefaultRows();
fesa_values[1U].values[3U] = 0.1049;
fesa_values[1U].values[4U] = -0.2098;
fesa_values[1U].values[5U] = 0.3147;
WriteHdf5(fixture.Results(), fixture.Input(), fesa_values);
auto result =
fesa::test::Mitc4ReferenceComparison::Compare(fixture.ReferenceCase());
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
EXPECT_TRUE(report.passed);
ASSERT_EQ(report.warnings.size(), 1U);
EXPECT_EQ(report.warnings[0U].code, "rotation-reference-rms-exceedance");
EXPECT_TRUE(std::all_of(report.rows.begin(), report.rows.end(),
[](const fesa::test::Mitc4RowDecision& row) {
return row.within_tolerance;
}));
}
TEST(Mitc4ReferenceComparison,
RotationRowAndRmsExceedancesProduceSeparateWarnings) {
ContractFixture fixture{"rotation-row-and-rms"};
auto fesa_values = DefaultRows();
fesa_values[0U].values[3U] = 1.0;
WriteHdf5(fixture.Results(), fixture.Input(), fesa_values);
auto result =
fesa::test::Mitc4ReferenceComparison::Compare(fixture.ReferenceCase());
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
EXPECT_TRUE(report.passed);
ASSERT_EQ(report.warnings.size(), 2U);
EXPECT_EQ(report.warnings[0U].code, "rotation-reference-exceedance");
EXPECT_EQ(report.warnings[1U].code, "rotation-reference-rms-exceedance");
EXPECT_LT(report.warnings[0U].row, report.rows.size());
EXPECT_LT(report.warnings[1U].row, report.rows.size());
}
// MITC4-REF-005
TEST(Mitc4ReferenceComparison,
ReportsMetricsVectorsWorstRowAndJsonDeterministically) {
@@ -678,7 +719,7 @@ TEST(Mitc4ReferenceComparison,
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
ASSERT_TRUE(report.passed);
ASSERT_EQ(report.metrics.size(), 6U);
ASSERT_EQ(report.metrics.size(), 2U);
ASSERT_EQ(report.vector_metrics.size(), 2U);
EXPECT_NEAR(report.vector_metrics[0U].displacement_norm_error,
std::sqrt(0.3125) * 1.0e-9, 1.0e-21);
@@ -687,10 +728,10 @@ TEST(Mitc4ReferenceComparison,
EXPECT_EQ(report.rows[report.worst_row].component, "UR1");
for (const auto& metric : report.metrics) {
EXPECT_TRUE(std::isfinite(metric.reference_scale));
EXPECT_TRUE(std::isfinite(metric.near_zero_band));
EXPECT_TRUE(std::isfinite(metric.maximum_absolute_error));
EXPECT_TRUE(std::isfinite(metric.maximum_normalized_error));
EXPECT_TRUE(std::isfinite(metric.rms_error));
EXPECT_TRUE(std::isfinite(metric.vector_norm_error));
EXPECT_TRUE(std::isfinite(metric.relative_rms));
EXPECT_FALSE(metric.components.empty());
EXPECT_LT(metric.worst_row, report.rows.size());
}
@@ -704,8 +745,10 @@ TEST(Mitc4ReferenceComparison,
.IsOk());
const std::string first = ReadBytes(json_a);
EXPECT_EQ(first, ReadBytes(json_b));
for (const char* key : {"\"rows\"", "\"metrics\"", "\"vector_metrics\"",
"\"warnings\"", "\"worst_row\"", "\"passed\""}) {
for (const char* key :
{"\"rows\"", "\"metrics\"", "\"vector_metrics\"", "\"warnings\"",
"\"family_identity\"", "\"near_zero_band\"", "\"relative_rms\"",
"\"row_count\"", "\"worst_row\"", "\"passed\""}) {
EXPECT_NE(first.find(key), std::string::npos) << key;
}
}
+255 -165
View File
@@ -29,7 +29,7 @@
#include "fesa/fem/dof_manager.h"
#include "fesa/io/abaqus/domain_mapper.h"
#include "fesa/io/abaqus/input_reader.h"
#include "fesa/results/result_recovery.h"
#include "reference_tolerance_policy.h"
namespace fesa::test {
namespace {
@@ -48,9 +48,6 @@ constexpr const char* kReactionPath = "/steps/Step-1/frames/0/nodal/reaction";
constexpr const char* kSectionPath =
"/steps/Step-1/frames/0/element/section_resultant";
constexpr const char* kStressPath = "/steps/Step-1/frames/0/element/stress_s11";
constexpr double kKinematicFloor = 1.0e-9;
constexpr double kForceMomentFloor = 1.0e-3;
constexpr double kRelativeCoefficient = 1.0e-6;
class ComparisonFailure final : public std::runtime_error {
public:
@@ -166,6 +163,17 @@ struct ReferenceTable {
std::vector<WideReferenceRow> rows;
};
struct SectionReferenceRow {
std::string instance_name;
std::int64_t source_element_label;
std::int64_t source_node_label;
std::vector<double> values;
};
struct SectionReferenceTable {
std::vector<SectionReferenceRow> rows;
};
ReferenceTable ReadReferenceCsv(
const std::filesystem::path& path,
const std::vector<std::string>& expected_header) {
@@ -223,6 +231,70 @@ ReferenceTable ReadReferenceCsv(
return table;
}
SectionReferenceTable ReadSectionReferenceCsv(
const std::filesystem::path& path) {
std::ifstream stream{path};
if (!stream) {
Fail("needs-reference-artifacts", "An approved reference CSV is missing.");
}
std::string line;
if (!std::getline(stream, line)) {
Fail("schema-mismatch", "An approved reference CSV is empty.");
}
if (!line.empty() && line.back() == '\r') {
line.pop_back();
}
const std::vector<std::string> expected_header = {
"Frame", "Part Instance Name",
"Element Label", "Node Label",
"SF-SF1", "SM-SM1",
"SM-SM2", "SM-SM3"};
if (SplitCsvLine(line) != expected_header) {
Fail("schema-mismatch", "An approved section CSV header is not exact.");
}
SectionReferenceTable table;
while (std::getline(stream, line)) {
if (!line.empty() && line.back() == '\r') {
line.pop_back();
}
if (line.empty()) {
Fail("schema-mismatch", "Blank reference CSV rows are not allowed.");
}
const auto fields = SplitCsvLine(line);
if (fields.size() != expected_header.size() ||
CollapseWhitespace(fields[0U]) != kFrameText || fields[1U].empty()) {
Fail("schema-mismatch",
"A reference section row has invalid schema or frame identity.");
}
SectionReferenceRow row{};
row.instance_name = fields[1U];
row.source_element_label = ParsePositiveLabel(fields[2U]);
row.source_node_label = ParsePositiveLabel(fields[3U]);
row.values.reserve(fields.size() - 4U);
for (std::size_t field = 4U; field < fields.size(); ++field) {
row.values.push_back(ParseFiniteDouble(fields[field]));
}
const auto duplicate = std::find_if(
table.rows.begin(), table.rows.end(),
[&](const SectionReferenceRow& existing) {
return AsciiLower(existing.instance_name) ==
AsciiLower(row.instance_name) &&
existing.source_element_label == row.source_element_label &&
existing.source_node_label == row.source_node_label;
});
if (duplicate != table.rows.end()) {
Fail("schema-mismatch",
"A reference section row identity is duplicated.");
}
table.rows.push_back(std::move(row));
}
if (table.rows.empty()) {
Fail("schema-mismatch", "An approved reference CSV has no data rows.");
}
return table;
}
void RequireExactArtifactInventory(
const std::filesystem::path& legacy_directory) {
std::error_code error;
@@ -894,76 +966,39 @@ std::vector<const WideReferenceRow*> OrderedRows(
return ordered;
}
double TableScale(const ReferenceTable& table, const std::size_t value_index) {
double scale = 0.0;
for (const auto& row : table.rows) {
if (value_index >= row.values.size()) {
Fail("schema-mismatch", "A reference row has the wrong component arity.");
}
scale = (std::max)(scale, std::abs(row.values[value_index]));
std::vector<const SectionReferenceRow*> OrderedSectionRows(
const SectionReferenceTable& table, const HdfProjection& hdf) {
const std::size_t endpoint_count = hdf.elements.size() * 2U;
if (table.rows.size() != endpoint_count) {
Fail("schema-mismatch", "The FESA/reference section row sets differ.");
}
return scale;
}
std::vector<NodeStationResultRow> NormalizeStations(
const Domain& domain, const HdfProjection& hdf,
const ReferenceTable& section_table) {
auto model_result = AnalysisModel::Create(domain);
if (!model_result.HasValue()) {
Fail("schema-mismatch",
"The approved input cannot create an analysis view.");
}
const AnalysisModel model = std::move(model_result.Value());
const std::array<double, 4> tolerances = {
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 0U),
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 3U),
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 1U),
kForceMomentFloor + kRelativeCoefficient * TableScale(section_table, 2U)};
std::vector<EndpointResultRow> endpoints;
endpoints.reserve(hdf.elements.size() * 2U);
for (std::size_t element = 0U; element < hdf.elements.size(); ++element) {
std::vector<const SectionReferenceRow*> ordered;
ordered.reserve(endpoint_count);
for (const auto& element : hdf.elements) {
for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) {
const auto node = static_cast<std::size_t>(
hdf.elements[element].node_internal_ids[endpoint]);
std::array<double, 4> values{};
for (std::size_t component = 0U; component < values.size(); ++component) {
values[component] =
hdf.section_resultants[(element * 2U + endpoint) * 4U + component];
const std::size_t node_index =
static_cast<std::size_t>(element.node_internal_ids[endpoint]);
if (node_index >= hdf.nodes.size()) {
Fail("schema-mismatch", "An HDF5 section endpoint references no node.");
}
endpoints.push_back({static_cast<EntityIndex>(element),
static_cast<int>(endpoint),
domain.Nodes()[node].source_id,
{},
values});
const auto& node = hdf.nodes[node_index];
const auto found = std::find_if(
table.rows.begin(), table.rows.end(),
[&](const SectionReferenceRow& row) {
return AsciiLower(row.instance_name) ==
AsciiLower(element.instance_name) &&
row.source_element_label == element.source_element_label &&
row.source_node_label == node.source_node_label;
});
if (found == table.rows.end() ||
found->instance_name != element.instance_name) {
Fail("schema-mismatch",
"A reference section endpoint identity does not match HDF5.");
}
ordered.push_back(&*found);
}
}
auto normalized = ResultRecovery::NormalizeSectionResultantsToNodeStations(
model, endpoints, tolerances);
if (!normalized.HasValue()) {
const auto& diagnostics = normalized.GetStatus().Diagnostics();
const std::string code =
diagnostics.empty() ? std::string{} : diagnostics[0U].code;
if (code == "node-station-tolerance-failure") {
Fail("tolerance-failure",
"Interior endpoint section resultants disagree.");
}
Fail("schema-mismatch",
"A node station is not eligible for legacy projection.");
}
return std::move(normalized.Value());
}
const NodeStationResultRow& FindStation(
const std::vector<NodeStationResultRow>& stations, const HdfNode& node) {
const auto found = std::find_if(
stations.begin(), stations.end(), [&](const NodeStationResultRow& row) {
return row.node.instance_name == node.instance_name &&
row.node.source_label == node.source_node_label;
});
if (found == stations.end()) {
Fail("schema-mismatch", "A projected HDF5 node station is missing.");
}
return *found;
return ordered;
}
CanonicalComparisonRow CanonicalRow(const HdfNode& node,
@@ -977,6 +1012,8 @@ CanonicalComparisonRow CanonicalRow(const HdfNode& node,
kFrameIndex,
node.instance_name,
node.source_node_label,
0,
-1,
quantity,
std::move(component),
value,
@@ -1003,93 +1040,122 @@ void AppendNodalRows(ComparisonReport& report, const HdfProjection& hdf,
CanonicalRow(hdf.nodes[node], quantity, components[component],
reference[node]->values[component], units[component],
"global-cartesian", dataset_path);
report.rows.push_back(
{std::move(fesa), std::move(abaqus), 0.0, 0.0, false});
report.rows.push_back({std::move(fesa), std::move(abaqus)});
}
}
}
void AppendSectionRows(ComparisonReport& report, const HdfProjection& hdf,
const std::vector<const WideReferenceRow*>& reference,
const std::vector<NodeStationResultRow>& stations) {
void AppendSectionRows(
ComparisonReport& report, const HdfProjection& hdf,
const std::vector<const SectionReferenceRow*>& reference) {
const std::array<std::string, 4> components = {"N", "T", "My", "Mz"};
const std::array<std::string, 4> units = {"force", "force*length",
"force*length", "force*length"};
const std::array<std::size_t, 4> reference_columns = {0U, 3U, 1U, 2U};
for (std::size_t node = 0U; node < hdf.nodes.size(); ++node) {
const auto& station = FindStation(stations, hdf.nodes[node]);
for (std::size_t component = 0U; component < components.size();
++component) {
auto fesa = CanonicalRow(
hdf.nodes[node], ComparisonQuantity::kSectionResultant,
components[component], station.section_resultant[component],
units[component], "beam-local", kSectionPath);
auto abaqus =
CanonicalRow(hdf.nodes[node], ComparisonQuantity::kSectionResultant,
components[component],
reference[node]->values[reference_columns[component]],
units[component], "beam-local", kSectionPath);
report.rows.push_back(
{std::move(fesa), std::move(abaqus), 0.0, 0.0, false});
std::size_t row = 0U;
for (std::size_t element = 0U; element < hdf.elements.size(); ++element) {
for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint, ++row) {
const auto& element_row = hdf.elements[element];
const std::size_t node_index =
static_cast<std::size_t>(element_row.node_internal_ids[endpoint]);
if (node_index >= hdf.nodes.size()) {
Fail("schema-mismatch", "An HDF5 section endpoint references no node.");
}
const auto& node = hdf.nodes[node_index];
const auto canonical = [&](const double value, std::string component,
std::string unit) {
return CanonicalComparisonRow{kModelId,
kStepName,
kFrameIndex,
element_row.instance_name,
node.source_node_label,
element_row.source_element_label,
static_cast<int>(endpoint),
ComparisonQuantity::kSectionResultant,
std::move(component),
value,
std::move(unit),
"beam-local",
kSectionPath};
};
for (std::size_t component = 0U; component < components.size();
++component) {
auto fesa = canonical(
hdf.section_resultants[(element * 2U + endpoint) * 4U + component],
components[component], units[component]);
auto abaqus =
canonical(reference[row]->values[reference_columns[component]],
components[component], units[component]);
report.rows.push_back({std::move(fesa), std::move(abaqus)});
}
}
}
}
double AbsoluteFloor(const ComparisonQuantity quantity, const std::string&) {
return quantity == ComparisonQuantity::kDisplacement ? kKinematicFloor
: kForceMomentFloor;
const char* BranchName(const ReferenceToleranceBranch branch) {
switch (branch) {
case ReferenceToleranceBranch::kNearZero:
return "near-zero";
case ReferenceToleranceBranch::kRelative:
return "relative";
case ReferenceToleranceBranch::kZeroScaleExact:
return "zero-scale-exact";
}
return "unknown";
}
void EvaluateGroup(ComparisonReport& report, const ComparisonQuantity quantity,
const std::string& component) {
void EvaluateFamily(ComparisonReport& report, const ComparisonQuantity quantity,
const std::string& identity,
const std::vector<std::string>& components,
const bool blocking) {
std::vector<std::size_t> row_indices;
for (std::size_t index = 0U; index < report.rows.size(); ++index) {
if (report.rows[index].reference.quantity == quantity &&
report.rows[index].reference.component == component) {
std::find(components.begin(), components.end(),
report.rows[index].reference.component) != components.end()) {
row_indices.push_back(index);
}
}
if (row_indices.empty()) {
Fail("schema-mismatch", "A canonical comparison component has no rows.");
Fail("schema-mismatch", "A canonical comparison family has no rows.");
}
double reference_scale = 0.0;
ReferenceToleranceFamily family{};
family.identity = identity;
family.components = components;
family.blocking = blocking;
family.rows.reserve(row_indices.size());
for (const std::size_t index : row_indices) {
reference_scale = (std::max)(reference_scale,
std::abs(report.rows[index].reference.value));
family.rows.push_back(
{report.rows[index].fesa.value, report.rows[index].reference.value});
}
const double tolerance = AbsoluteFloor(quantity, component) +
kRelativeCoefficient * reference_scale;
double maximum_absolute = -1.0;
double maximum_normalized = 0.0;
std::size_t worst_row = row_indices.front();
long double squared_error = 0.0L;
for (const std::size_t index : row_indices) {
auto& row = report.rows[index];
row.absolute_error = std::abs(row.fesa.value - row.reference.value);
if (!std::isfinite(row.absolute_error)) {
Fail("schema-mismatch", "A canonical row error is nonfinite.");
}
row.tolerance = tolerance;
row.passed = row.absolute_error <= tolerance;
report.passed = report.passed && row.passed;
const double normalized = row.absolute_error / tolerance;
if (row.absolute_error > maximum_absolute) {
maximum_absolute = row.absolute_error;
worst_row = index;
}
maximum_normalized = (std::max)(maximum_normalized, normalized);
const long double error = static_cast<long double>(row.absolute_error);
squared_error += error * error;
const auto evaluation =
ReferenceTolerancePolicy::Evaluate({std::move(family)});
if (!evaluation.valid || evaluation.families.size() != 1U) {
Fail("schema-mismatch", "A canonical comparison family is nonfinite.");
}
const double norm_error = std::sqrt(static_cast<double>(squared_error));
const double rms_error = std::sqrt(static_cast<double>(
squared_error / static_cast<long double>(row_indices.size())));
if (!std::isfinite(norm_error) || !std::isfinite(rms_error)) {
Fail("schema-mismatch", "A component aggregate error is nonfinite.");
const auto& policy = evaluation.families.front();
for (std::size_t local = 0U; local < row_indices.size(); ++local) {
auto& row = report.rows[row_indices[local]];
const auto& decision = policy.rows[local];
row.absolute_error = decision.absolute_error;
row.tolerance = decision.threshold;
row.reference_scale = policy.reference_scale;
row.near_zero_band = policy.near_zero_band;
row.relative_error = decision.relative_error;
row.relative_error_applicable = decision.relative_error_applicable;
row.tolerance_branch = BranchName(decision.branch);
row.passed = decision.passed;
}
const std::size_t worst_row = row_indices[policy.worst_row];
report.metrics.push_back({quantity, policy.identity, policy.components,
policy.reference_scale, policy.near_zero_band,
policy.near_zero_count,
policy.maximum_absolute_error, policy.relative_rms,
worst_row, policy.rms_passed, policy.passed,
policy.diagnostic_code, policy.row_count});
if (blocking && !policy.passed) {
report.passed = false;
}
report.metrics.push_back({quantity, component, reference_scale,
maximum_absolute, maximum_normalized, rms_error,
norm_error, worst_row});
}
PhysicsEvidence MakePhysicsEvidence(const Domain& domain,
@@ -1214,7 +1280,8 @@ void WriteCanonicalRow(std::ostream& stream,
stream << ",\"frame_index\":" << row.frame_index << ",\"instance_name\":";
WriteJsonString(stream, row.instance_name);
stream << ",\"source_node_label\":" << row.source_node_label
<< ",\"quantity\":";
<< ",\"source_element_label\":" << row.source_element_label
<< ",\"endpoint_index\":" << row.endpoint_index << ",\"quantity\":";
WriteJsonString(stream, QuantityName(row.quantity));
stream << ",\"component\":";
WriteJsonString(stream, row.component);
@@ -1231,6 +1298,18 @@ void WriteArray(std::ostream& stream, const std::array<double, 3>& values) {
stream << '[' << values[0U] << ',' << values[1U] << ',' << values[2U] << ']';
}
void WriteStrings(std::ostream& stream,
const std::vector<std::string>& values) {
stream << '[';
for (std::size_t index = 0U; index < values.size(); ++index) {
if (index != 0U) {
stream << ',';
}
WriteJsonString(stream, values[index]);
}
stream << ']';
}
bool FiniteReport(const ComparisonReport& report) {
const auto finite_array = [](const std::array<double, 3>& values) {
return std::all_of(values.begin(), values.end(),
@@ -1245,17 +1324,19 @@ bool FiniteReport(const ComparisonReport& report) {
}
for (const auto& row : report.rows) {
if (!std::isfinite(row.fesa.value) || !std::isfinite(row.reference.value) ||
!std::isfinite(row.absolute_error) || !std::isfinite(row.tolerance)) {
!std::isfinite(row.absolute_error) || !std::isfinite(row.tolerance) ||
!std::isfinite(row.reference_scale) ||
!std::isfinite(row.near_zero_band) ||
!std::isfinite(row.relative_error)) {
return false;
}
}
return std::all_of(report.metrics.begin(), report.metrics.end(),
[](const ComponentMetrics& metric) {
return std::isfinite(metric.reference_scale) &&
std::isfinite(metric.near_zero_band) &&
std::isfinite(metric.maximum_absolute_error) &&
std::isfinite(metric.maximum_normalized_error) &&
std::isfinite(metric.rms_error) &&
std::isfinite(metric.norm_error);
std::isfinite(metric.relative_rms);
});
}
@@ -1276,18 +1357,12 @@ Result<ComparisonReport> ReferenceComparison::Compare(
ReadReferenceCsv(legacy_reference_directory / kReactionName,
{"Frame", "Part Instance Name", "Node Label", "RF-RF1",
"RF-RF2", "RF-RF3", "RM-RM1", "RM-RM2", "RM-RM3"});
const ReferenceTable section =
ReadReferenceCsv(legacy_reference_directory / kSectionName,
{"Frame", "Part Instance Name", "Node Label", "SF-SF1",
"SM-SM1", "SM-SM2", "SM-SM3"});
const SectionReferenceTable section =
ReadSectionReferenceCsv(legacy_reference_directory / kSectionName);
HdfProjection hdf = ReadHdfProjection(results_hdf5, input, domain);
const auto displacement_rows = OrderedRows(displacement, hdf.nodes);
const auto reaction_rows = OrderedRows(reaction, hdf.nodes);
const auto section_rows = OrderedRows(section, hdf.nodes);
const auto stations = NormalizeStations(domain, hdf, section);
if (stations.size() != hdf.nodes.size()) {
Fail("schema-mismatch", "The HDF5 node-station row set is incomplete.");
}
const auto section_rows = OrderedSectionRows(section, hdf);
ComparisonReport report{};
report.passed = true;
@@ -1301,19 +1376,20 @@ Result<ComparisonReport> ReferenceComparison::Compare(
{"force", "force", "force", "force*length", "force*length",
"force*length"},
hdf.reaction, kReactionPath);
AppendSectionRows(report, hdf, section_rows, stations);
AppendSectionRows(report, hdf, section_rows);
for (const std::string& component :
{"UX", "UY", "UZ", "URX", "URY", "URZ"}) {
EvaluateGroup(report, ComparisonQuantity::kDisplacement, component);
}
for (const std::string& component :
{"RF1", "RF2", "RF3", "RM1", "RM2", "RM3"}) {
EvaluateGroup(report, ComparisonQuantity::kReaction, component);
}
for (const std::string& component : {"N", "T", "My", "Mz"}) {
EvaluateGroup(report, ComparisonQuantity::kSectionResultant, component);
}
EvaluateFamily(report, ComparisonQuantity::kDisplacement,
"displacement-translation", {"UX", "UY", "UZ"}, true);
EvaluateFamily(report, ComparisonQuantity::kDisplacement,
"displacement-rotation", {"URX", "URY", "URZ"}, true);
EvaluateFamily(report, ComparisonQuantity::kReaction, "reaction-force",
{"RF1", "RF2", "RF3"}, true);
EvaluateFamily(report, ComparisonQuantity::kReaction, "reaction-moment",
{"RM1", "RM2", "RM3"}, true);
EvaluateFamily(report, ComparisonQuantity::kSectionResultant,
"section-force", {"N"}, true);
EvaluateFamily(report, ComparisonQuantity::kSectionResultant,
"section-moment", {"T", "My", "Mz"}, true);
report.physics_evidence = MakePhysicsEvidence(domain, hdf);
report.stress_comparison_applicable = false;
report.stress_comparison_reason =
@@ -1366,8 +1442,15 @@ Status ReferenceComparison::WriteDeterministicJson(
stream << ",\"reference\":";
WriteCanonicalRow(stream, row.reference);
stream << ",\"absolute_error\":" << row.absolute_error
<< ",\"tolerance\":" << row.tolerance
<< ",\"passed\":" << (row.passed ? "true" : "false") << '}';
<< ",\"threshold\":" << row.tolerance
<< ",\"reference_scale\":" << row.reference_scale
<< ",\"near_zero_band\":" << row.near_zero_band
<< ",\"relative_error\":" << row.relative_error
<< ",\"relative_error_applicable\":"
<< (row.relative_error_applicable ? "true" : "false")
<< ",\"tolerance_branch\":";
WriteJsonString(stream, row.tolerance_branch);
stream << ",\"passed\":" << (row.passed ? "true" : "false") << '}';
}
stream << "],\"metrics\":[";
for (std::size_t index = 0U; index < report.metrics.size(); ++index) {
@@ -1377,15 +1460,22 @@ Status ReferenceComparison::WriteDeterministicJson(
const auto& metric = report.metrics[index];
stream << "{\"quantity\":";
WriteJsonString(stream, QuantityName(metric.quantity));
stream << ",\"component\":";
WriteJsonString(stream, metric.component);
stream << ",\"family_identity\":";
WriteJsonString(stream, metric.family_identity);
stream << ",\"components\":";
WriteStrings(stream, metric.components);
stream << ",\"reference_scale\":" << metric.reference_scale
<< ",\"near_zero_band\":" << metric.near_zero_band
<< ",\"near_zero_count\":" << metric.near_zero_count
<< ",\"row_count\":" << metric.row_count
<< ",\"maximum_absolute_error\":" << metric.maximum_absolute_error
<< ",\"maximum_normalized_error\":"
<< metric.maximum_normalized_error
<< ",\"rms_error\":" << metric.rms_error
<< ",\"norm_error\":" << metric.norm_error
<< ",\"worst_row\":" << metric.worst_row << '}';
<< ",\"relative_rms\":" << metric.relative_rms
<< ",\"worst_row\":" << metric.worst_row
<< ",\"rms_passed\":" << (metric.rms_passed ? "true" : "false")
<< ",\"passed\":" << (metric.passed ? "true" : "false")
<< ",\"diagnostic_code\":";
WriteJsonString(stream, metric.diagnostic_code);
stream << '}';
}
stream << "],\"physics_evidence\":{\"free_residual_norm\":"
<< report.physics_evidence.free_residual_norm << ",\"applied_force\":";
+16 -4
View File
@@ -20,6 +20,8 @@ struct CanonicalComparisonRow {
std::size_t frame_index;
std::string instance_name;
std::int64_t source_node_label;
std::int64_t source_element_label;
int endpoint_index;
ComparisonQuantity quantity;
std::string component;
double value;
@@ -33,18 +35,28 @@ struct RowDecision {
CanonicalComparisonRow reference;
double absolute_error;
double tolerance;
double reference_scale;
double near_zero_band;
double relative_error;
bool relative_error_applicable;
std::string tolerance_branch;
bool passed;
};
struct ComponentMetrics {
ComparisonQuantity quantity;
std::string component;
std::string family_identity;
std::vector<std::string> components;
double reference_scale;
double near_zero_band;
std::size_t near_zero_count;
double maximum_absolute_error;
double maximum_normalized_error;
double rms_error;
double norm_error;
double relative_rms;
std::size_t worst_row;
bool rms_passed;
bool passed;
std::string diagnostic_code;
std::size_t row_count{};
};
struct PhysicsEvidence {
+140 -68
View File
@@ -41,8 +41,8 @@ constexpr const char* kSectionName = "cantilever beam elemental forces.csv";
constexpr const char* kInstanceName = "PART-1_1-1";
constexpr std::size_t kNodeCount = 11U;
constexpr std::size_t kElementCount = 10U;
constexpr std::size_t kExpectedRowCount = 176U;
constexpr std::size_t kExpectedMetricCount = 16U;
constexpr std::size_t kExpectedRowCount = 212U;
constexpr std::size_t kExpectedMetricCount = 6U;
using NodalValues = std::array<std::array<double, 6>, kNodeCount>;
using EndpointValues =
@@ -315,14 +315,29 @@ const fesa::test::RowDecision* FindRow(
const fesa::test::ComponentMetrics* FindMetric(
const fesa::test::ComparisonReport& report,
const fesa::test::ComparisonQuantity quantity,
const std::string& family_identity) {
const auto found =
std::find_if(report.metrics.begin(), report.metrics.end(),
[&](const fesa::test::ComponentMetrics& metric) {
return metric.family_identity == family_identity;
});
return found == report.metrics.end() ? nullptr : &*found;
}
const fesa::test::RowDecision* FindSectionRow(
const fesa::test::ComparisonReport& report,
const std::int64_t source_element_label, const int endpoint_index,
const std::string& component) {
const auto found = std::find_if(
report.metrics.begin(), report.metrics.end(),
[&](const fesa::test::ComponentMetrics& metric) {
return metric.quantity == quantity && metric.component == component;
report.rows.begin(), report.rows.end(),
[&](const fesa::test::RowDecision& row) {
return row.reference.quantity ==
fesa::test::ComparisonQuantity::kSectionResultant &&
row.reference.source_element_label == source_element_label &&
row.reference.endpoint_index == endpoint_index &&
row.reference.component == component;
});
return found == report.metrics.end() ? nullptr : &*found;
return found == report.rows.end() ? nullptr : &*found;
}
void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) {
@@ -381,13 +396,25 @@ void ExpectExactRowInventory(const fesa::test::ComparisonReport& report) {
const std::array<std::string, 4> section_components = {"N", "T", "My", "Mz"};
const std::array<std::string, 4> section_units = {
"force", "force*length", "force*length", "force*length"};
for (std::size_t node = 0U; node < kNodeCount; ++node) {
for (std::size_t component = 0U; component < section_components.size();
++component) {
expect_row(fesa::test::ComparisonQuantity::kSectionResultant, node,
section_components[component], section_units[component],
"beam-local",
"/steps/Step-1/frames/0/element/section_resultant");
for (std::size_t element = 0U; element < kElementCount; ++element) {
for (std::size_t endpoint = 0U; endpoint < 2U; ++endpoint) {
for (std::size_t component = 0U; component < section_components.size();
++component) {
ASSERT_LT(row_index, report.rows.size());
const auto& row = report.rows[row_index++];
for (const auto* side : {&row.fesa, &row.reference}) {
EXPECT_EQ(side->source_element_label,
static_cast<std::int64_t>(element + 1U));
EXPECT_EQ(side->endpoint_index, static_cast<int>(endpoint));
EXPECT_EQ(side->source_node_label,
static_cast<std::int64_t>(element + endpoint + 1U));
EXPECT_EQ(side->component, section_components[component]);
EXPECT_EQ(side->unit_dimension, section_units[component]);
EXPECT_EQ(side->coordinate_system, "beam-local");
EXPECT_EQ(side->hdf5_dataset_path,
"/steps/Step-1/frames/0/element/section_resultant");
}
}
}
}
EXPECT_EQ(row_index, report.rows.size());
@@ -475,14 +502,46 @@ TEST(ReferenceComparisonContract,
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
{
ContractFixture fixture{"section-missing-endpoint", mismatched_values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_EQ(lines.size(), kElementCount * 2U + 1U);
lines.pop_back();
WriteLines(fixture.Legacy() / kSectionName, lines);
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
{
ContractFixture fixture{"section-duplicate-endpoint", mismatched_values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_EQ(lines.size(), kElementCount * 2U + 1U);
lines.push_back(lines[1U]);
WriteLines(fixture.Legacy() / kSectionName, lines);
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
{
ContractFixture fixture{"section-mismatched-endpoint", mismatched_values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_GT(lines.size(), 2U);
ReplaceFirst(lines[1U], ",1,1,", ",1,3,");
WriteLines(fixture.Legacy() / kSectionName, lines);
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(
fixture.Results(), fixture.Legacy()),
"schema-mismatch");
}
}
TEST(ReferenceComparisonContract,
AppliesAbaqusOnlyComponentScaleWithoutClampOrDrop) {
AppliesSharedFamilyScaleAndNearZeroBranchesWithoutAnAbsoluteGate) {
auto values = ReferenceValues();
values.displacement[1U][0U] = 0.999e-9;
values.displacement[2U][0U] = 1.001e-9;
values.section_resultants[0U][0U][2U] = 1.0e7 + 9.0;
const double translation_scale = 1.90476272e-2;
const double near_zero_band = 0.01 * translation_scale;
values.displacement[1U][0U] = 0.999 * near_zero_band;
values.displacement[2U][0U] = 1.001 * near_zero_band;
values.section_resultants[0U][0U][2U] = 1.0e7 + 4.999e5;
values.section_resultants[9U][1U][2U] = 0.0;
ContractFixture fixture{"tolerance", values};
@@ -505,28 +564,28 @@ TEST(ReferenceComparisonContract,
ASSERT_NE(deliberate_failure, nullptr);
EXPECT_DOUBLE_EQ(zero->reference.value, 0.0);
EXPECT_DOUBLE_EQ(zero->fesa.value, 0.0);
EXPECT_DOUBLE_EQ(zero->tolerance, 1.0e-9);
EXPECT_DOUBLE_EQ(zero->tolerance, near_zero_band);
EXPECT_TRUE(zero->passed);
EXPECT_TRUE(near_zero->passed);
EXPECT_FALSE(deliberate_failure->passed);
EXPECT_EQ(near_zero->tolerance_branch, "near-zero");
EXPECT_FALSE(near_zero->relative_error_applicable);
const auto* my_metric = FindMetric(
report, fesa::test::ComparisonQuantity::kSectionResultant, "My");
const auto* my_metric = FindMetric(report, "section-moment");
ASSERT_NE(my_metric, nullptr);
EXPECT_DOUBLE_EQ(my_metric->reference_scale, 1.0e7);
const auto* scaled = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "My");
const auto* residue = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 11, "My");
EXPECT_EQ(my_metric->components, (std::vector<std::string>{"T", "My", "Mz"}));
const auto* scaled = FindSectionRow(report, 1, 0, "My");
const auto* residue = FindSectionRow(report, 10, 1, "My");
ASSERT_NE(scaled, nullptr);
ASSERT_NE(residue, nullptr);
EXPECT_DOUBLE_EQ(scaled->tolerance, 10.001);
EXPECT_DOUBLE_EQ(scaled->absolute_error, 9.0);
EXPECT_DOUBLE_EQ(scaled->tolerance, 5.0e5);
EXPECT_DOUBLE_EQ(scaled->absolute_error, 4.999e5);
EXPECT_TRUE(scaled->passed);
EXPECT_DOUBLE_EQ(residue->reference.value, -1.56e-2);
EXPECT_DOUBLE_EQ(residue->reference.value, -1.5625e-2);
EXPECT_DOUBLE_EQ(residue->fesa.value, 0.0);
EXPECT_DOUBLE_EQ(residue->absolute_error, 1.56e-2);
EXPECT_DOUBLE_EQ(residue->tolerance, 10.001);
EXPECT_DOUBLE_EQ(residue->absolute_error, 1.5625e-2);
EXPECT_DOUBLE_EQ(residue->tolerance, 1.0e5);
EXPECT_TRUE(residue->passed);
}
@@ -548,19 +607,18 @@ TEST(ReferenceComparisonContract,
report.rows.begin(), report.rows.end(),
[](const fesa::test::RowDecision& row) { return row.passed; }));
const auto* metric =
FindMetric(report, fesa::test::ComparisonQuantity::kDisplacement, "UX");
const auto* metric = FindMetric(report, "displacement-translation");
const auto* worst =
FindRow(report, fesa::test::ComparisonQuantity::kDisplacement, 2, "UX");
ASSERT_NE(metric, nullptr);
ASSERT_NE(worst, nullptr);
EXPECT_DOUBLE_EQ(metric->reference_scale, 0.0);
EXPECT_EQ(metric->row_count, kNodeCount * 3U);
EXPECT_DOUBLE_EQ(metric->reference_scale, 1.90476272e-2);
EXPECT_DOUBLE_EQ(metric->maximum_absolute_error, 1.0e-9);
EXPECT_DOUBLE_EQ(metric->maximum_normalized_error, 1.0);
EXPECT_NEAR(metric->rms_error,
std::sqrt(1.25 / static_cast<double>(kNodeCount)) * 1.0e-9,
EXPECT_NEAR(metric->relative_rms,
std::sqrt(1.25 / static_cast<double>(kNodeCount * 3U)) * 1.0e-9 /
metric->reference_scale,
1.0e-21);
EXPECT_NEAR(metric->norm_error, std::sqrt(1.25) * 1.0e-9, 1.0e-21);
EXPECT_EQ(metric->worst_row,
static_cast<std::size_t>(worst - report.rows.data()));
@@ -589,7 +647,10 @@ TEST(ReferenceComparisonContract,
const std::string first = ReadBytes(json_a);
EXPECT_EQ(first, ReadBytes(json_b));
for (const char* required :
{"\"rows\"", "\"metrics\"", "\"stress_comparison_applicable\":false",
{"\"rows\"", "\"metrics\"", "\"family_identity\"", "\"components\"",
"\"near_zero_band\"", "\"relative_rms\"", "\"row_count\"",
"\"tolerance_branch\"", "\"relative_error_applicable\"",
"\"stress_comparison_applicable\":false",
"\"stress_comparison_reason\"", "\"physics_evidence\"",
"\"free_residual_norm\"", "\"applied_force\"", "\"reaction_force\"",
"\"applied_moment_about_origin\"", "\"reaction_moment_about_origin\"",
@@ -598,11 +659,10 @@ TEST(ReferenceComparisonContract,
}
}
TEST(ReferenceComparisonContract, NormalizesEligibleStationsWithoutAveraging) {
TEST(ReferenceComparisonContract,
PreservesSectionEndpointIdentityWithoutStationNormalization) {
auto values = ReferenceValues();
values.section_resultants[0U][0U] = {2.0e-4, 3.0e-4, 1.0e7, 4.0e-4};
values.section_resultants[0U][1U][2U] = 9.0e6 - 5.0;
values.section_resultants[1U][0U][2U] = 9.0e6 + 5.0;
ContractFixture fixture{"stations", values};
auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(),
@@ -612,35 +672,47 @@ TEST(ReferenceComparisonContract, NormalizesEligibleStationsWithoutAveraging) {
ASSERT_TRUE(report.passed);
EXPECT_TRUE(report.physics_evidence.endpoint_consistency_passed);
const auto* n = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "N");
const auto* t = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "T");
const auto* my = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "My");
const auto* mz = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 1, "Mz");
ASSERT_NE(n, nullptr);
ASSERT_NE(t, nullptr);
ASSERT_NE(my, nullptr);
ASSERT_NE(mz, nullptr);
EXPECT_DOUBLE_EQ(n->fesa.value, 2.0e-4);
EXPECT_DOUBLE_EQ(t->fesa.value, 3.0e-4);
EXPECT_DOUBLE_EQ(my->fesa.value, 1.0e7);
EXPECT_DOUBLE_EQ(mz->fesa.value, 4.0e-4);
const auto* interior = FindRow(
report, fesa::test::ComparisonQuantity::kSectionResultant, 2, "My");
const auto* interior = FindSectionRow(report, 1, 1, "My");
ASSERT_NE(interior, nullptr);
EXPECT_DOUBLE_EQ(interior->fesa.value, 9.0e6 - 5.0);
EXPECT_DOUBLE_EQ(interior->reference.value, 9.0e6);
EXPECT_DOUBLE_EQ(interior->absolute_error, 5.0);
auto mismatch_values = ReferenceValues();
mismatch_values.section_resultants[0U][1U][2U] = 9.0e6 - 6.0;
mismatch_values.section_resultants[1U][0U][2U] = 9.0e6 + 6.0;
ContractFixture mismatch{"station-mismatch", mismatch_values};
ExpectFailureCode(fesa::test::ReferenceComparison::Compare(mismatch.Results(),
mismatch.Legacy()),
"tolerance-failure");
const auto* adjacent = FindSectionRow(report, 2, 0, "My");
ASSERT_NE(adjacent, nullptr);
EXPECT_DOUBLE_EQ(adjacent->fesa.value, 9.0e6);
EXPECT_DOUBLE_EQ(adjacent->reference.value, 9.0e6);
}
TEST(ReferenceComparisonContract,
MapsEachSectionCsvComponentToItsDirectEndpointResultant) {
auto values = ReferenceValues();
values.section_resultants[0U][0U] = {11.0, 22.0, 33.0, 44.0};
ContractFixture fixture{"section-component-mapping", values};
auto lines = ReadLines(fixture.Legacy() / kSectionName);
ASSERT_EQ(lines.size(), kElementCount * 2U + 1U);
lines[1U] = "Increment 1: Step Time = 1.000,PART-1_1-1,1,1,11,33,44,22";
WriteLines(fixture.Legacy() / kSectionName, lines);
auto result = fesa::test::ReferenceComparison::Compare(fixture.Results(),
fixture.Legacy());
ASSERT_TRUE(result.HasValue());
const auto& report = result.Value();
ASSERT_TRUE(report.passed);
const auto* n = FindSectionRow(report, 1, 0, "N");
const auto* t = FindSectionRow(report, 1, 0, "T");
const auto* my = FindSectionRow(report, 1, 0, "My");
const auto* mz = FindSectionRow(report, 1, 0, "Mz");
ASSERT_NE(n, nullptr);
ASSERT_NE(t, nullptr);
ASSERT_NE(my, nullptr);
ASSERT_NE(mz, nullptr);
EXPECT_DOUBLE_EQ(n->reference.value, 11.0);
EXPECT_DOUBLE_EQ(t->reference.value, 22.0);
EXPECT_DOUBLE_EQ(my->reference.value, 33.0);
EXPECT_DOUBLE_EQ(mz->reference.value, 44.0);
EXPECT_DOUBLE_EQ(n->fesa.value, 11.0);
EXPECT_DOUBLE_EQ(t->fesa.value, 22.0);
EXPECT_DOUBLE_EQ(my->fesa.value, 33.0);
EXPECT_DOUBLE_EQ(mz->fesa.value, 44.0);
}
@@ -0,0 +1,115 @@
#include "reference_tolerance_policy.h"
#include <algorithm>
#include <cmath>
#include <limits>
namespace fesa::test {
namespace {
bool IsFiniteFamily(const ReferenceToleranceFamily& family) {
if (family.identity.empty() || family.components.empty() ||
family.rows.empty()) {
return false;
}
return std::all_of(family.rows.begin(), family.rows.end(),
[](const ReferenceToleranceRow& row) {
return std::isfinite(row.fesa_value) &&
std::isfinite(row.reference_value);
});
}
}
ReferenceToleranceEvaluation ReferenceTolerancePolicy::Evaluate(
const std::vector<ReferenceToleranceFamily>& families) {
ReferenceToleranceEvaluation evaluation{true, {}, true, {}};
evaluation.families.reserve(families.size());
for (const auto& family : families) {
if (!IsFiniteFamily(family)) {
return {false, "nonfinite-comparison-value", false, {}};
}
ReferenceToleranceFamilyDecision decision{};
decision.identity = family.identity;
decision.components = family.components;
decision.blocking = family.blocking;
decision.row_count = family.rows.size();
decision.rows.reserve(family.rows.size());
for (const auto& row : family.rows) {
decision.reference_scale =
(std::max)(decision.reference_scale, std::abs(row.reference_value));
}
decision.near_zero_band = kNearZeroRatio * decision.reference_scale;
double error_norm = 0.0;
for (std::size_t index = 0U; index < family.rows.size(); ++index) {
const auto& row = family.rows[index];
const double absolute_error =
std::abs(row.fesa_value - row.reference_value);
if (!std::isfinite(absolute_error)) {
return {false, "nonfinite-comparison-value", false, {}};
}
error_norm = std::hypot(error_norm, absolute_error);
if (!std::isfinite(error_norm)) {
return {false, "nonfinite-comparison-value", false, {}};
}
if (absolute_error > decision.maximum_absolute_error) {
decision.maximum_absolute_error = absolute_error;
decision.worst_row = index;
}
ReferenceToleranceRowDecision row_decision{};
row_decision.absolute_error = absolute_error;
if (decision.reference_scale == 0.0) {
++decision.near_zero_count;
row_decision.threshold = 0.0;
row_decision.branch = ReferenceToleranceBranch::kZeroScaleExact;
row_decision.passed = absolute_error == 0.0;
} else if (std::abs(row.reference_value) <= decision.near_zero_band) {
++decision.near_zero_count;
row_decision.threshold = decision.near_zero_band;
row_decision.branch = ReferenceToleranceBranch::kNearZero;
row_decision.passed = absolute_error <= row_decision.threshold;
} else {
row_decision.threshold =
kRelativeTolerance * std::abs(row.reference_value);
row_decision.relative_error =
absolute_error / std::abs(row.reference_value);
row_decision.relative_error_applicable = true;
row_decision.branch = ReferenceToleranceBranch::kRelative;
row_decision.passed = row_decision.relative_error <= kRelativeTolerance;
}
decision.rows.push_back(row_decision);
}
const bool rows_passed = std::all_of(
decision.rows.begin(), decision.rows.end(),
[](const ReferenceToleranceRowDecision& row) { return row.passed; });
if (decision.reference_scale == 0.0) {
decision.relative_rms = 0.0;
decision.rms_passed = rows_passed;
if (!rows_passed) {
decision.diagnostic_code = "zero-reference-scale-nonzero-error";
}
} else {
const double rms =
error_norm / std::sqrt(static_cast<double>(family.rows.size()));
decision.relative_rms = rms / decision.reference_scale;
if (!std::isfinite(decision.relative_rms)) {
return {false, "nonfinite-comparison-value", false, {}};
}
decision.rms_passed = decision.relative_rms <= kRelativeRmsTolerance;
}
decision.passed = rows_passed && decision.rms_passed;
if (decision.blocking && !decision.passed) {
evaluation.passed = false;
}
evaluation.families.push_back(std::move(decision));
}
return evaluation;
}
}
@@ -0,0 +1,73 @@
#ifndef FESA_TESTS_REFERENCE_REFERENCE_TOLERANCE_POLICY_H_
#define FESA_TESTS_REFERENCE_REFERENCE_TOLERANCE_POLICY_H_
#include <cstddef>
#include <string>
#include <vector>
namespace fesa::test {
enum class ReferenceToleranceBranch {
kNearZero,
kRelative,
kZeroScaleExact,
};
struct ReferenceToleranceRow {
double fesa_value;
double reference_value;
};
struct ReferenceToleranceFamily {
std::string identity;
std::vector<std::string> components;
bool blocking;
std::vector<ReferenceToleranceRow> rows;
};
struct ReferenceToleranceRowDecision {
double absolute_error;
double threshold;
double relative_error;
bool relative_error_applicable;
ReferenceToleranceBranch branch;
bool passed;
};
struct ReferenceToleranceFamilyDecision {
std::string identity;
std::vector<std::string> components;
bool blocking;
double reference_scale;
double near_zero_band;
std::size_t near_zero_count;
double relative_rms;
double maximum_absolute_error;
std::size_t worst_row;
bool rms_passed;
bool passed;
std::string diagnostic_code;
std::vector<ReferenceToleranceRowDecision> rows;
std::size_t row_count{};
};
struct ReferenceToleranceEvaluation {
bool valid;
std::string diagnostic_code;
bool passed;
std::vector<ReferenceToleranceFamilyDecision> families;
};
class ReferenceTolerancePolicy {
public:
static constexpr double kNearZeroRatio = 0.01;
static constexpr double kRelativeTolerance = 0.05;
static constexpr double kRelativeRmsTolerance = 0.01;
static ReferenceToleranceEvaluation Evaluate(
const std::vector<ReferenceToleranceFamily>& families);
};
}
#endif
@@ -0,0 +1,118 @@
#include "reference_tolerance_policy.h"
#include <gtest/gtest.h>
#include <algorithm>
#include <cmath>
#include <limits>
#include <string>
#include <utility>
#include <vector>
namespace {
using fesa::test::ReferenceToleranceBranch;
using fesa::test::ReferenceToleranceFamily;
using fesa::test::ReferenceTolerancePolicy;
using fesa::test::ReferenceToleranceRow;
ReferenceToleranceFamily MakeFamily(std::string identity,
std::vector<std::string> components,
bool blocking,
std::vector<ReferenceToleranceRow> rows) {
return {std::move(identity), std::move(components), blocking,
std::move(rows)};
}
TEST(ReferenceTolerancePolicy,
AppliesRelativeAndNearZeroThresholdsAtTheirInclusiveBoundaries) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate({MakeFamily(
"displacement-translation", {"UX", "UY", "UZ"}, true,
{{105.0, 100.0}, {105.001, 100.0}, {2.0, 1.0}, {2.001, 1.0}})});
ASSERT_TRUE(evaluation.valid);
ASSERT_EQ(evaluation.families.size(), 1U);
const auto& family = evaluation.families.front();
EXPECT_EQ(family.row_count, 4U);
EXPECT_DOUBLE_EQ(family.reference_scale, 100.0);
EXPECT_DOUBLE_EQ(family.near_zero_band, 1.0);
EXPECT_EQ(family.near_zero_count, 2U);
EXPECT_FALSE(family.passed);
ASSERT_EQ(family.rows.size(), 4U);
EXPECT_EQ(family.rows[0U].branch, ReferenceToleranceBranch::kRelative);
EXPECT_TRUE(family.rows[0U].relative_error_applicable);
EXPECT_DOUBLE_EQ(family.rows[0U].relative_error, 0.05);
EXPECT_DOUBLE_EQ(family.rows[0U].threshold, 5.0);
EXPECT_TRUE(family.rows[0U].passed);
EXPECT_FALSE(family.rows[1U].passed);
EXPECT_EQ(family.rows[2U].branch, ReferenceToleranceBranch::kNearZero);
EXPECT_FALSE(family.rows[2U].relative_error_applicable);
EXPECT_DOUBLE_EQ(family.rows[2U].threshold, 1.0);
EXPECT_TRUE(family.rows[2U].passed);
EXPECT_FALSE(family.rows[3U].passed);
}
TEST(ReferenceTolerancePolicy,
RejectsFamiliesWhoseScaleRelativeRmsExceedsOnePercent) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate({MakeFamily(
"reaction-force", {"RF1", "RF2", "RF3"}, true,
{{101.1, 100.0}, {101.1, 100.0}, {101.1, 100.0}, {101.1, 100.0}})});
ASSERT_TRUE(evaluation.valid);
const auto& family = evaluation.families.front();
EXPECT_EQ(family.row_count, 4U);
EXPECT_TRUE(std::all_of(family.rows.begin(), family.rows.end(),
[](const auto& row) { return row.passed; }));
EXPECT_NEAR(family.relative_rms, 0.011, 1.0e-15);
EXPECT_FALSE(family.rms_passed);
EXPECT_FALSE(family.passed);
EXPECT_FALSE(evaluation.passed);
}
TEST(ReferenceTolerancePolicy,
FailsClosedForZeroScaleNonzeroErrorWithoutNonfiniteMetrics) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate(
{MakeFamily("section-force", {"N"}, true, {{0.0, 0.0}, {1.0, 0.0}})});
ASSERT_TRUE(evaluation.valid);
const auto& family = evaluation.families.front();
EXPECT_EQ(family.row_count, 2U);
EXPECT_DOUBLE_EQ(family.reference_scale, 0.0);
EXPECT_DOUBLE_EQ(family.near_zero_band, 0.0);
EXPECT_EQ(family.near_zero_count, 2U);
EXPECT_EQ(family.diagnostic_code, "zero-reference-scale-nonzero-error");
EXPECT_EQ(family.rows[0U].branch, ReferenceToleranceBranch::kZeroScaleExact);
EXPECT_TRUE(family.rows[0U].passed);
EXPECT_FALSE(family.rows[1U].passed);
EXPECT_FALSE(family.rows[1U].relative_error_applicable);
EXPECT_TRUE(std::isfinite(family.relative_rms));
EXPECT_DOUBLE_EQ(family.relative_rms, 0.0);
EXPECT_FALSE(family.rms_passed);
EXPECT_FALSE(family.passed);
}
TEST(ReferenceTolerancePolicy,
KeepsWarningOnlyFamilyExceedanceOutOfBlockingVerdict) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate(
{MakeFamily("translation", {"U1", "U2", "U3"}, true, {{0.0, 0.0}}),
MakeFamily("rotation", {"UR1", "UR2", "UR3"}, false, {{1.0, 0.0}})});
ASSERT_TRUE(evaluation.valid);
ASSERT_EQ(evaluation.families.size(), 2U);
EXPECT_TRUE(evaluation.families[0U].passed);
EXPECT_FALSE(evaluation.families[1U].passed);
EXPECT_TRUE(evaluation.passed);
}
TEST(ReferenceTolerancePolicy,
RejectsNonfiniteInputsBeforeToleranceEvaluation) {
const auto evaluation = ReferenceTolerancePolicy::Evaluate(
{MakeFamily("displacement-translation", {"UX", "UY", "UZ"}, true,
{{std::numeric_limits<double>::quiet_NaN(), 0.0}})});
EXPECT_FALSE(evaluation.valid);
EXPECT_EQ(evaluation.diagnostic_code, "nonfinite-comparison-value");
EXPECT_TRUE(evaluation.families.empty());
}
}