FESADev/docs/reference-verifications/README.md

Reference Verification Report 문서 작성 가이드

이 디렉터리는 Reference Verification Agent가 작성하거나 제안하는 기능별 reference comparison report를 보관하는 위치다.

Reference Verification Agent는 Build/Test Executor Agent 통과 후 generated solver result CSV와 stored Abaqus reference CSV artifacts를 tolerance 기준으로 비교한다. 이 agent는 comparison과 report만 수행하며, source code, tests, CMake files, requirements, formulations, I/O contracts, reference artifacts, tolerance policies를 수정하지 않는다.

기본 문서명은 docs/reference-verifications/<feature-id>-reference-verification.md 형식을 사용한다.

Reference Verification Agent 역할

수행한다:

• references/<feature-id>/<model-id>/ artifact bundle과 generated solver result CSV를 확인한다.
• metadata.json, schema version, units, coordinate system, step/frame identity, node/element IDs, output location, tolerance source를 확인한다.
• displacements.csv, reactions.csv, element_forces.csv, stresses.csv를 기본 비교 대상으로 삼는다.
• upstream 문서가 요구할 때만 strains.csv, energy_or_residual.csv를 추가 비교한다.
• max absolute error, max relative error, RMS error, norm error, worst node/element/component, missing rows, extra rows, pass/fail을 보고한다.
• 실패를 missing-reference-artifact, missing-solver-output, schema-mismatch, id-mismatch, unit-or-coordinate-mismatch, tolerance-failure, nonfinite-result, upstream-contract, environment로 분류한다.

수행하지 않는다:

• source code를 수정하지 않는다.
• tests를 수정하지 않는다.
• CMake files를 수정하지 않는다.
• requirements, formulations, I/O contracts, reference model contracts를 수정하지 않는다.
• reference artifacts 또는 tolerance policies를 수정하지 않는다.
• Abaqus, Nastran 또는 reference solver를 실행하지 않는다.
• reference CSV를 생성하지 않는다.
• solver output CSV를 tolerance에 맞추기 위해 보정하지 않는다.
• physics validation success 또는 release readiness를 승인하지 않는다.

실행 순서

Reference Verification Agent는 항상 다음 순서를 따른다.

ARTIFACT CHECK -> COMPARE -> CLASSIFY -> REPORT

ARTIFACT CHECK에서 다음 항목이 없으면 비교를 시작하지 않는다.

• metadata.json
• required reference CSV files
• generated solver result CSV files
• schema version
• units
• coordinate system
• step/frame identity
• node/element ID matching rule
• output location
• tolerance policy

비교 대상

기본 비교 대상:

• displacements.csv: nodal displacement
• reactions.csv: nodal reaction force
• element_forces.csv: element internal force
• stresses.csv: element stress

선택 비교 대상:

• strains.csv: strain이 acceptance criteria에 포함된 경우
• energy_or_residual.csv: energy, residual, convergence quantity가 acceptance criteria에 포함된 경우

Failure Classification

• missing-reference-artifact: required stored reference file 또는 provenance가 없다.
• missing-solver-output: generated solver result CSV 또는 comparison command가 없다.
• schema-mismatch: reference CSV와 solver CSV column/schema가 다르다.
• id-mismatch: node id, element id, step/frame, integration point, component matching이 실패했다.
• unit-or-coordinate-mismatch: units 또는 coordinate system이 비교 가능하지 않다.
• tolerance-failure: schema와 matching은 유효하지만 error가 tolerance를 초과했다.
• nonfinite-result: NaN 또는 infinite value가 발견됐다.
• upstream-contract: tolerance, schema, units, output location, ID matching policy가 누락 또는 충돌한다.
• environment: 로컬 실행 환경 문제로 비교가 불가능하다.

문서 템플릿

# <feature title> Reference Verification Report

## Metadata
- feature_id: <feature-id>
- source_build_test_report: docs/build-test-reports/<feature-id>-build-test.md
- source_reference_models: docs/reference-models/<feature-id>-reference-models.md
- source_io_definition: docs/io-definitions/<feature-id>-io.md
- source_implementation_report: <path or N/A>
- status: pass-for-physics-evaluation | needs-correction | needs-reference-artifacts | needs-solver-results | needs-upstream-decision | blocked
- owner_agent: reference-verification-agent
- date: <YYYY-MM-DD>

## Artifact Inventory

| item | path | status | notes |
| --- | --- | --- | --- |
| reference_bundle | references/<feature-id>/<model-id>/ | present | missing | <notes> |
| metadata | references/<feature-id>/<model-id>/metadata.json | present | missing | <provenance summary> |
| reference_displacements | references/<feature-id>/<model-id>/displacements.csv | present | missing | <notes> |
| reference_reactions | references/<feature-id>/<model-id>/reactions.csv | present | missing | <notes> |
| reference_element_forces | references/<feature-id>/<model-id>/element_forces.csv | present | missing | <notes> |
| reference_stresses | references/<feature-id>/<model-id>/stresses.csv | present | missing | <notes> |
| solver_outputs | <solver output directory> | present | missing | <notes> |

## Comparison Contract
- schema_version: <version>
- id_matching: node_id | element_id | step/frame | integration_point | component
- units: <unit system>
- coordinate_system: <global/local convention>
- output_location: nodal | element | integration_point | centroid | recovery_location
- component_naming: <component naming policy>
- tolerance_source: <requirement/reference model/I/O document>
- tolerance_policy: absolute | relative | norm-based | combined
- zero_reference_policy: <policy or N/A>

## Quantity Results

| quantity | model_id | artifact_file | compared_rows | missing_rows | extra_rows | max_abs_error | max_rel_error | rms_error | norm_error | worst_id | worst_component | result |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| displacement | <model-id> | displacements.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <node id> | <component> | pass | fail |
| reaction | <model-id> | reactions.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <node id> | <component> | pass | fail |
| element force | <model-id> | element_forces.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <element id> | <component> | pass | fail |
| stress | <model-id> | stresses.csv | <n> | <n> | <n> | <value> | <value> | <value> | <value or N/A> | <element/ip id> | <component> | pass | fail |

## Failure Classification
- classification: missing-reference-artifact | missing-solver-output | schema-mismatch | id-mismatch | unit-or-coordinate-mismatch | tolerance-failure | nonfinite-result | upstream-contract | environment | N/A
- primary_failure: <short summary>
- evidence: <short relevant excerpt or computed metric>

## Handoff Recommendation

| target_agent | reason | required_input |
| --- | --- | --- |
| Correction Agent | <implementation-owned mismatch or nonfinite result> | <comparison metrics and failing quantity> |
| Reference Model Agent | <missing or invalid reference artifact/provenance> | <artifact inventory> |
| I/O Definition Agent | <schema, units, coordinate, output location mismatch> | <contract mismatch> |
| Physics Evaluation Agent | <reference comparisons passed> | <quantity results and report> |
| Coordinator Agent | <blocked or repeated ambiguity> | <classification and open issue> |

## No-Change Assertion
- source_files_modified: false
- test_files_modified: false
- cmake_files_modified: false
- reference_artifacts_modified: false
- tolerance_policies_modified: false
- notes: <observed no-change evidence or exception>

## Open Issues
- <missing solver outputs, missing reference artifacts, schema gaps, tolerance gaps, or repeated comparison failures>

상태 값

• pass-for-physics-evaluation: required reference comparisons가 모두 통과했고 Physics Evaluation Agent로 넘길 수 있다.
• needs-correction: implementation-owned solver result mismatch 또는 nonfinite result가 있다.
• needs-reference-artifacts: required reference artifact 또는 provenance가 누락됐다.
• needs-solver-results: generated solver result CSV 또는 comparison command가 없다.
• needs-upstream-decision: schema, tolerance, units, coordinate system, output location, ID matching policy가 누락 또는 충돌한다.
• blocked: 사용자 또는 Coordinator Agent 결정 없이는 안전하게 진행할 수 없다.

품질 기준

• 모든 must requirement의 reference-comparison 항목은 model id, compared quantity, artifact file, tolerance에 trace되어야 한다.
• reference artifact는 읽기 전용이다. model.inp, metadata.json, reference CSV를 수정하지 않는다.
• solver output CSV는 비교 입력일 뿐이며 tolerance에 맞추기 위해 후처리 보정하지 않는다.
• stress/strain은 element id, integration point 또는 recovery location, component naming이 일치할 때만 비교한다.
• nodal displacement/reaction은 node id, DOF/component, coordinate system, unit이 일치할 때만 비교한다.
• missing rows와 extra rows를 숨기지 않고 보고한다.
• NaN 또는 infinite value는 nonfinite-result로 분류한다.
• pass는 reference tolerance 통과만 의미한다.
• physics validation과 release readiness는 각각 Physics Evaluation Agent와 Release Agent가 판정한다.