Reference Verification Report 문서 작성 가이드

이 디렉터리는 Reference Verification Agent가 작성하거나 제안하는 기능별 reference comparison report를 보관하는 위치다.

Reference Verification Agent는 Build/Test Executor Agent 통과 후 generated solver results.h5와 Abaqus reference CSV files를 tolerance 기준으로 비교한다. Reference CSV는 동일한 Abaqus .inp 모델을 Abaqus로 해석해 추출한 변위, 반력, 내력, 응력 결과이며, FESA HDF5 dataset에서 파생된 파일이 아니다. 이 agent는 comparison과 report만 수행하며, source code, tests, CMake files, requirements, formulations, I/O contracts, reference artifacts, tolerance policies를 수정하지 않는다.

기본 문서명은 docs/reference-verifications/<feature-id>-reference-verification.md 형식을 사용한다.

Reference Verification Agent 역할

수행한다:

reference/<model-id>/ artifact bundle과 generated solver results.h5를 확인한다.
metadata.json, model.inp, required Abaqus reference CSV files, reference CSV schema version, FESA HDF5 schema version, units, coordinate system, step/frame identity, node/element ID matching rule, output location, component naming, tolerance policy를 확인한다.
FESA HDF5 dataset을 normalized row record로 읽고 Abaqus reference CSV row와 직접 비교한다.
comparison command가 FESA results.h5에서 deterministic CSV view를 materialize할 수 있지만, 이 파일은 debugging/review용 derived artifact일 뿐 reference artifact가 아니다.
upstream 문서가 요구할 때만 <model-id>_strains.csv, <model-id>_energy_or_residual.csv, 또는 <model-id>_<quantity>.csv를 추가 비교한다.
max absolute error, max relative error, RMS error, norm error, worst id/component, missing rows, extra rows, pass/fail을 보고한다.
실패를 missing-reference-artifact, missing-solver-output, schema-mismatch, id-mismatch, unit-or-coordinate-mismatch, tolerance-failure, nonfinite-result, upstream-contract, environment로 분류한다.

수행하지 않는다:

source code를 수정하지 않는다.
tests를 수정하지 않는다.
CMake files를 수정하지 않는다.
requirements, formulations, I/O contracts, reference model contracts를 수정하지 않는다.
reference artifacts 또는 tolerance policies를 수정하지 않는다.
Abaqus, Nastran 또는 reference solver를 실행하지 않는다.
Abaqus reference CSV 파일을 생성하거나 수정하지 않는다.
solver output을 tolerance에 맞추기 위해 보정하지 않는다.
physics validation success 또는 release readiness를 승인하지 않는다.

실행 순서

Reference Verification Agent는 항상 다음 순서를 따른다.

ARTIFACT CHECK -> COMPARE -> CLASSIFY -> REPORT

ARTIFACT CHECK에서 다음 항목이 없으면 비교를 시작하지 않는다.

metadata.json
model.inp
generated solver results.h5
reference/<model-id>/<model-id>_displacements.csv
reference/<model-id>/<model-id>_reactions.csv
reference/<model-id>/<model-id>_internalforces.csv
reference/<model-id>/<model-id>_stresses.csv
reference CSV schema version
FESA HDF5 schema version
units
coordinate system
step/frame identity
node/element ID matching rule
output location
component naming
tolerance policy

비교 대상

quantity	fesa_hdf5_dataset	reference_csv
displacement	/steps//frames//field_outputs/U	reference//_displacements.csv
reaction	/steps//frames//field_outputs/RF	reference//_reactions.csv
internal force	/steps//frames//field_outputs/element_forces	reference//_internalforces.csv
stress	/steps//frames//field_outputs/S	reference//_stresses.csv

선택 Abaqus reference CSV:

reference/<model-id>/<model-id>_strains.csv: strain이 acceptance criteria에 포함된 경우
reference/<model-id>/<model-id>_energy_or_residual.csv: energy, residual, convergence quantity가 acceptance criteria에 포함된 경우
reference/<model-id>/<model-id>_<quantity>.csv: feature-specific quantity가 acceptance criteria에 포함된 경우

Failure Classification

missing-reference-artifact: required Abaqus reference CSV file 또는 provenance가 없다.
missing-solver-output: generated solver results.h5 또는 comparison command가 없다.
schema-mismatch: FESA HDF5 또는 reference CSV row schema가 다르다.
id-mismatch: node id, element id, step/frame, integration point, component matching이 실패했다.
unit-or-coordinate-mismatch: units 또는 coordinate system이 비교 가능하지 않다.
tolerance-failure: schema와 matching은 유효하지만 error가 tolerance를 초과했다.
nonfinite-result: NaN 또는 infinite value가 발견됐다.
upstream-contract: tolerance, schema, units, output location, ID matching policy가 누락 또는 충돌한다.
environment: 로컬 실행 환경 문제로 비교가 불가능하다.

문서 템플릿

# <feature title> Reference Verification Report

## Metadata
- feature_id: <feature-id>
- source_build_test_report: docs/build-test-reports/<feature-id>-build-test.md
- source_reference_models: docs/reference-models/<feature-id>-reference-models.md
- source_io_definition: docs/io-definitions/<feature-id>-io.md
- source_implementation_report: <path or N/A>
- status: pass-for-physics-evaluation | needs-correction | needs-reference-artifacts | needs-solver-results | needs-upstream-decision | blocked
- owner_agent: reference-verification-agent
- date: <YYYY-MM-DD>

## Artifact Inventory

| item | path | status | notes |
| --- | --- | --- | --- |
| reference_model_dir | reference/<model-id>/ | present | missing | <notes> |
| reference_input | reference/<model-id>/model.inp | present | missing | <input summary> |
| metadata | reference/<model-id>/metadata.json | present | missing | <provenance summary> |
| reference_displacements_csv | reference/<model-id>/<model-id>_displacements.csv | present | missing | <row/schema summary> |
| reference_reactions_csv | reference/<model-id>/<model-id>_reactions.csv | present | missing | <row/schema summary> |
| reference_internalforces_csv | reference/<model-id>/<model-id>_internalforces.csv | present | missing | <row/schema summary> |
| reference_stresses_csv | reference/<model-id>/<model-id>_stresses.csv | present | missing | <row/schema summary> |
| solver_hdf5 | <solver output directory>/results.h5 | present | missing | <schema summary> |
| solver_debug_csv_view | <solver output directory>/csv/ | present | missing | <optional, derived from results.h5 only> |

## Comparison Contract
- hdf5_schema_version: <version>
- reference_csv_schema_version: <version>
- id_matching: node_id | element_id | step/frame | integration_point | component
- units: <unit system>
- coordinate_system: <global/local convention>
- output_location: nodal | element | integration_point | centroid | recovery_location
- component_naming: <component naming policy>
- row_sort_order: <step, frame, id, location, component order>
- tolerance_source: <requirement/reference model/I/O document>
- tolerance_policy: absolute | relative | norm-based | combined
- zero_reference_policy: <policy or N/A>

## Quantity Results

| quantity | model_id | fesa_hdf5_dataset | reference_csv | compared_rows | missing_rows | extra_rows | max_abs_error | max_rel_error | rms_error | norm_error | worst_id | worst_component | result |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| displacement | <model-id> | /steps/<step>/frames/<frame>/field_outputs/U | reference/<model-id>/<model-id>_displacements.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <node id> | <component> | pass | fail |
| reaction | <model-id> | /steps/<step>/frames/<frame>/field_outputs/RF | reference/<model-id>/<model-id>_reactions.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <node id> | <component> | pass | fail |
| internal force | <model-id> | /steps/<step>/frames/<frame>/field_outputs/element_forces | reference/<model-id>/<model-id>_internalforces.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <element id> | <component> | pass | fail |
| stress | <model-id> | /steps/<step>/frames/<frame>/field_outputs/S | reference/<model-id>/<model-id>_stresses.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <element/ip id> | <component> | pass | fail |

## Failure Classification
- classification: missing-reference-artifact | missing-solver-output | schema-mismatch | id-mismatch | unit-or-coordinate-mismatch | tolerance-failure | nonfinite-result | upstream-contract | environment | N/A
- primary_failure: <short summary>
- evidence: <short relevant excerpt or computed metric>

## Handoff Recommendation

| target_agent | reason | required_input |
| --- | --- | --- |
| Correction Agent | <implementation-owned mismatch or nonfinite result> | <comparison metrics and failing quantity> |
| Reference Model Agent | <missing or invalid reference artifact/provenance> | <artifact inventory> |
| I/O Definition Agent | <FESA HDF5 schema, reference CSV schema, units, coordinate, output location mismatch> | <contract mismatch> |
| Physics Evaluation Agent | <reference comparisons passed> | <quantity results and report> |
| Coordinator Agent | <blocked or repeated ambiguity> | <classification and open issue> |

## No-Change Assertion
- source_files_modified: false
- test_files_modified: false
- cmake_files_modified: false
- reference_artifacts_modified: false
- tolerance_policies_modified: false
- notes: <observed no-change evidence or exception>

## Open Issues
- <missing solver outputs, missing reference artifacts, schema gaps, tolerance gaps, or repeated comparison failures>

상태 값

pass-for-physics-evaluation: required reference comparisons가 모두 통과했고 Physics Evaluation Agent로 넘길 수 있다.
needs-correction: implementation-owned solver result mismatch 또는 nonfinite result가 있다.
needs-reference-artifacts: required Abaqus reference CSV 또는 provenance가 누락됐다.
needs-solver-results: generated solver results.h5 또는 comparison command가 없다.
needs-upstream-decision: schema, tolerance, units, coordinate system, output location, ID matching policy가 누락 또는 충돌한다.
blocked: 사용자 또는 Coordinator Agent 결정 없이는 안전하게 진행할 수 없다.

품질 기준

모든 must requirement의 reference-comparison 항목은 model id, compared quantity, FESA HDF5 dataset, reference CSV, tolerance에 trace되어야 한다.
reference artifact는 읽기 전용이다. model.inp, metadata.json, reference/<model-id>/<model-id>_*.csv를 수정하지 않는다.
FESA results.h5가 authoritative solver output이고 Abaqus reference CSV files가 authoritative reference result다.
solver debug CSV view는 행 정렬과 사람이 검토 가능한 비교 view일 뿐이며 tolerance에 맞추기 위해 후처리 보정하지 않는다.
stress/strain은 element id, integration point 또는 recovery location, component naming이 일치할 때만 비교한다.
nodal displacement/reaction은 node id, DOF/component, coordinate system, unit이 일치할 때만 비교한다.
missing rows와 extra rows를 숨기지 않고 보고한다.
NaN 또는 infinite value는 nonfinite-result로 분류한다.
pass는 reference tolerance 통과만 의미한다.
physics validation과 release readiness는 각각 Physics Evaluation Agent와 Release Agent가 판정한다.

12 KiB Raw Blame History