김경종
786 lines
30 KiB
Markdown
786 lines
30 KiB
Markdown
# Reference CSV Comparison Contract Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Re-align FESA reference comparison contracts so FESA solver output remains authoritative HDF5, while Abaqus-derived reference results are CSV files compared directly against the FESA `results.h5` datasets.
|
|
|
|
**Architecture:** The active contract becomes `FESA results.h5 -> comparison normalizer -> Abaqus reference CSV`. Reference CSV files are authoritative reference artifacts; they are not deterministic CSV views derived from `reference.h5`, and `reference.h5` is no longer required. Existing documentation, skills, agent prompts, and Python contract tests must all describe the same flat reference folder shape.
|
|
|
|
**Tech Stack:** Markdown docs, Codex skill `SKILL.md`, agent TOML/YAML metadata, Python `unittest`, existing `scripts/validate_workspace.py`.
|
|
|
|
---
|
|
|
|
## Target Contract
|
|
|
|
Use this exact vocabulary across active contracts:
|
|
|
|
- FESA solver result: HDF5 file, normally `<solver-output-dir>/results.h5`.
|
|
- Reference result: Abaqus-generated CSV files created by running the same Abaqus `.inp` model outside the agent workflow.
|
|
- Comparison: read FESA HDF5 datasets and compare their normalized rows directly with reference CSV rows.
|
|
- Reference root: `reference/`, singular, following the user-provided structure.
|
|
|
|
Required reference structure:
|
|
|
|
```text
|
|
reference/
|
|
<model-id>/
|
|
<model-id>_displacements.csv
|
|
<model-id>_reactions.csv
|
|
<model-id>_internalforces.csv
|
|
<model-id>_stresses.csv
|
|
<model-id>_<quantity>.csv
|
|
```
|
|
|
|
Quantity naming:
|
|
|
|
| quantity | reference CSV | FESA HDF5 source |
|
|
| --- | --- | --- |
|
|
| displacement | `reference/<model-id>/<model-id>_displacements.csv` | `/steps/<step>/frames/<frame>/field_outputs/U` |
|
|
| reaction | `reference/<model-id>/<model-id>_reactions.csv` | `/steps/<step>/frames/<frame>/field_outputs/RF` |
|
|
| internal force | `reference/<model-id>/<model-id>_internalforces.csv` | `/steps/<step>/frames/<frame>/field_outputs/element_forces` or feature-defined equivalent |
|
|
| stress | `reference/<model-id>/<model-id>_stresses.csv` | `/steps/<step>/frames/<frame>/field_outputs/S` |
|
|
|
|
Allowed concept:
|
|
|
|
- A comparison command may materialize a deterministic CSV view from FESA HDF5 for debugging or review, but it is derived from `results.h5`, not an official solver output and not the reference artifact.
|
|
|
|
Prohibited active-contract claims:
|
|
|
|
- `reference.h5` is required.
|
|
- `results.h5` and `reference.h5` are both authoritative comparison inputs.
|
|
- Reference CSV files are deterministic views derived from `reference.h5`.
|
|
- Reference bundles live only under `references/<feature-id>/<model-id>/`.
|
|
|
|
Historical plan files under `docs/superpowers/plans/` may mention older contracts, but active docs, skills, agents, and tests must not.
|
|
|
|
## File Structure
|
|
|
|
Modify these active contract files:
|
|
|
|
- `AGENTS.md`: project-level reference comparison rules.
|
|
- `docs/ProjectInitialPlanNote.md`: initial plan notes where the old HDF5/reference-HDF5 decision was recorded.
|
|
- `docs/PRD.md`: user-facing product/verification expectations.
|
|
- `docs/ARCHITECTURE.md`: result and reference data-flow architecture.
|
|
- `docs/ADR.md`: architecture decision record for HDF5 solver output and CSV reference artifacts.
|
|
- `docs/SOLVER_AGENT_DESIGN.md`: agent workflow and gates.
|
|
- `docs/SOLVER_SKILL_DESIGN.md`: skill workflow overview.
|
|
- `docs/reference-models/README.md`: reference artifact bundle contract.
|
|
- `docs/reference-verifications/README.md`: comparison report template and status rules.
|
|
- `docs/io-definitions/README.md`: FESA HDF5 schema and comparison row schema handoff.
|
|
- `docs/implementation-plans/README.md`: implementation planning data-flow contract.
|
|
- `docs/physics-evaluations/README.md`: physics evidence inventory.
|
|
- `docs/requirements/README.md`: reference artifact requirements section.
|
|
|
|
Modify these skills and UI metadata:
|
|
|
|
- `.codex/skills/fesa-reference-models/SKILL.md`
|
|
- `.codex/skills/fesa-reference-models/agents/openai.yaml`
|
|
- `.codex/skills/fesa-reference-comparison/SKILL.md`
|
|
- `.codex/skills/fesa-reference-comparison/agents/openai.yaml`
|
|
- `.codex/skills/fesa-io-contract/SKILL.md`
|
|
- `.codex/skills/fesa-physics-sanity/SKILL.md`
|
|
- `.codex/skills/fesa-cpp-msvc-tdd/SKILL.md`
|
|
- `.codex/skills/fesa-release-readiness/SKILL.md`
|
|
- `.codex/skills/fesa-requirements-baseline/SKILL.md`
|
|
|
|
Modify these agent configs:
|
|
|
|
- `.codex/agents/reference-model-agent.toml`
|
|
- `.codex/agents/reference-verification-agent.toml`
|
|
- `.codex/agents/io-definition-agent.toml`
|
|
- `.codex/agents/implementation-planning-agent.toml`
|
|
- `.codex/agents/implementation-agent.toml`
|
|
- `.codex/agents/physics-evaluation-agent.toml`
|
|
- `.codex/agents/release-agent.toml`
|
|
- `.codex/agents/requirement-agent.toml`
|
|
- `.codex/agents/coordinator-agent.toml`
|
|
- Other `.codex/agents/*-agent.toml` files only where they contain stale `reference.h5` or `reference HDF5` claims.
|
|
|
|
Modify these Python contract tests:
|
|
|
|
- `scripts/test_fesa_solver_skills.py`
|
|
- `scripts/test_reference_model_agent_config.py`
|
|
- `scripts/test_reference_verification_agent_config.py`
|
|
- `scripts/test_physics_evaluation_agent_config.py`
|
|
- `scripts/test_io_definition_agent_config.py`
|
|
- `scripts/test_implementation_planning_agent_config.py`
|
|
- `scripts/test_requirement_agent_config.py`
|
|
- `scripts/test_release_agent_config.py`
|
|
- Any other `scripts/test_*_agent_config.py` that still asserts `Do not generate reference HDF5 files or deterministic CSV views.`
|
|
|
|
---
|
|
|
|
### Task 1: Update Contract Tests First
|
|
|
|
**Files:**
|
|
- Modify: `scripts/test_fesa_solver_skills.py`
|
|
- Modify: `scripts/test_reference_model_agent_config.py`
|
|
- Modify: `scripts/test_reference_verification_agent_config.py`
|
|
- Modify: `scripts/test_physics_evaluation_agent_config.py`
|
|
- Modify: `scripts/test_io_definition_agent_config.py`
|
|
- Modify: `scripts/test_implementation_planning_agent_config.py`
|
|
- Modify: `scripts/test_requirement_agent_config.py`
|
|
- Modify: `scripts/test_release_agent_config.py`
|
|
|
|
- [ ] **Step 1: Replace reference-HDF5 expectations in reference model tests**
|
|
|
|
In `scripts/test_reference_model_agent_config.py`, change the artifact contract expectations from `reference.h5` and nested `csv/` files to the flat user-provided structure.
|
|
|
|
Use assertions equivalent to:
|
|
|
|
```python
|
|
for required_text in (
|
|
"FESA reference models use Abaqus input files.",
|
|
"reference/<model-id>/",
|
|
"model.inp",
|
|
"metadata.json",
|
|
"<model-id>_displacements.csv",
|
|
"<model-id>_reactions.csv",
|
|
"<model-id>_internalforces.csv",
|
|
"<model-id>_stresses.csv",
|
|
):
|
|
self.assertIn(required_text, instructions)
|
|
|
|
self.assertNotIn("reference.h5", instructions)
|
|
self.assertNotIn("references/<feature-id>/<model-id>/", instructions)
|
|
```
|
|
|
|
Change the boundary expectation from:
|
|
|
|
```python
|
|
"Do not generate reference HDF5 files or deterministic CSV views.",
|
|
```
|
|
|
|
to:
|
|
|
|
```python
|
|
"Do not generate or modify Abaqus reference CSV files.",
|
|
```
|
|
|
|
- [ ] **Step 2: Replace reference-HDF5 expectations in reference verification tests**
|
|
|
|
In `scripts/test_reference_verification_agent_config.py`, assert that the agent compares FESA HDF5 to Abaqus reference CSV:
|
|
|
|
```python
|
|
for required_text in (
|
|
"results.h5",
|
|
"Abaqus reference CSV",
|
|
"reference/<model-id>/",
|
|
"<model-id>_displacements.csv",
|
|
"<model-id>_reactions.csv",
|
|
"<model-id>_internalforces.csv",
|
|
"<model-id>_stresses.csv",
|
|
"metadata.json",
|
|
):
|
|
self.assertIn(required_text, instructions)
|
|
|
|
self.assertNotIn("reference.h5", instructions)
|
|
self.assertNotIn("stored reference HDF5", instructions)
|
|
```
|
|
|
|
Keep output section assertions for `Artifact Inventory`, `Comparison Contract`, `Quantity Results`, `Failure Classification`, `Handoff Recommendation`, and `No-Change Assertion`.
|
|
|
|
- [ ] **Step 3: Update solver skill contract tests**
|
|
|
|
In `scripts/test_fesa_solver_skills.py`, update the `fesa-reference-models` `body_terms` to:
|
|
|
|
```python
|
|
"reference/<model-id>/",
|
|
"model.inp",
|
|
"metadata.json",
|
|
"<model-id>_displacements.csv",
|
|
"<model-id>_reactions.csv",
|
|
"<model-id>_internalforces.csv",
|
|
"<model-id>_stresses.csv",
|
|
"Coverage Matrix",
|
|
"Do not generate or modify Abaqus reference CSV files.",
|
|
```
|
|
|
|
Update the `fesa-reference-comparison` `body_terms` to:
|
|
|
|
```python
|
|
"docs/reference-verifications/<feature-id>-reference-verification.md",
|
|
"ARTIFACT CHECK -> COMPARE -> CLASSIFY -> REPORT",
|
|
"results.h5",
|
|
"Abaqus reference CSV",
|
|
"reference/<model-id>/",
|
|
"<model-id>_displacements.csv",
|
|
"<model-id>_reactions.csv",
|
|
"<model-id>_internalforces.csv",
|
|
"<model-id>_stresses.csv",
|
|
"max absolute error",
|
|
"max relative error",
|
|
"RMS error",
|
|
"missing rows",
|
|
"extra rows",
|
|
"pass-for-physics-evaluation",
|
|
"Do not change tolerance policies.",
|
|
```
|
|
|
|
Keep `HDF5` in the description terms for FESA output. Replace `CSV view` description terms with `CSV` or `reference CSV` where the old phrase implies a reference HDF5-derived view.
|
|
|
|
- [ ] **Step 4: Update shared boundary tests**
|
|
|
|
For each affected `scripts/test_*_agent_config.py`, replace the old boundary assertion:
|
|
|
|
```python
|
|
"Do not generate reference HDF5 files or deterministic CSV views.",
|
|
```
|
|
|
|
with:
|
|
|
|
```python
|
|
"Do not generate or modify Abaqus reference CSV files.",
|
|
```
|
|
|
|
Use this only for agents whose scope mentions reference artifacts. Do not add the phrase to unrelated tests unless the corresponding agent config already carries a reference artifact boundary.
|
|
|
|
- [ ] **Step 5: Run the updated tests and confirm RED**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python -m unittest scripts.test_reference_model_agent_config scripts.test_reference_verification_agent_config scripts.test_fesa_solver_skills -v
|
|
```
|
|
|
|
Expected: FAIL. Failures should point to missing `reference/<model-id>/`, missing `<model-id>_internalforces.csv`, or stale `reference.h5` text in active contracts.
|
|
|
|
Commit after this task only if using a clean branch:
|
|
|
|
```powershell
|
|
git add scripts/test_fesa_solver_skills.py scripts/test_reference_model_agent_config.py scripts/test_reference_verification_agent_config.py scripts/test_physics_evaluation_agent_config.py scripts/test_io_definition_agent_config.py scripts/test_implementation_planning_agent_config.py scripts/test_requirement_agent_config.py scripts/test_release_agent_config.py
|
|
git commit -m "test: redefine reference comparison contract"
|
|
```
|
|
|
|
### Task 2: Update Project-Level Documentation
|
|
|
|
**Files:**
|
|
- Modify: `AGENTS.md`
|
|
- Modify: `docs/ProjectInitialPlanNote.md`
|
|
- Modify: `docs/PRD.md`
|
|
- Modify: `docs/ARCHITECTURE.md`
|
|
- Modify: `docs/ADR.md`
|
|
|
|
- [ ] **Step 1: Update `AGENTS.md` project identity**
|
|
|
|
Replace the current reference comparison statement with:
|
|
|
|
```markdown
|
|
- 공식 solver output은 HDF5 `results.h5`이다.
|
|
- reference 결과는 FESA와 같은 Abaqus `.inp` 모델을 Abaqus로 해석해 생성한 CSV 파일이다.
|
|
- reference comparison은 FESA `results.h5`의 변위, 반력, 내력, 응력 dataset을 `reference/<model-id>/<model-id>_*.csv` 파일과 비교한다.
|
|
- CSV는 FESA 공식 output이 아니며, FESA HDF5에서 추출한 deterministic CSV view는 비교 디버깅/검토용 보조 artifact로만 둔다.
|
|
```
|
|
|
|
Replace any active `references/<feature-id>/<model-id>/` reference artifact examples with:
|
|
|
|
```text
|
|
reference/<model-id>/<model-id>_displacements.csv
|
|
reference/<model-id>/<model-id>_reactions.csv
|
|
reference/<model-id>/<model-id>_internalforces.csv
|
|
reference/<model-id>/<model-id>_stresses.csv
|
|
```
|
|
|
|
- [ ] **Step 2: Update `docs/ProjectInitialPlanNote.md`**
|
|
|
|
Revise the section currently stating that `reference.h5` is the stored authoritative reference. The replacement must say:
|
|
|
|
```markdown
|
|
## HDF5 결과와 Abaqus reference CSV 비교 결정
|
|
|
|
FESA solver의 공식 해석 결과는 `results.h5` HDF5 파일이다. Reference 결과는 동일한 Abaqus `.inp` 모델을 Abaqus에서 해석해 추출한 CSV 파일이며, `reference/<model-id>/` 아래에 물리량별 파일로 저장한다.
|
|
|
|
Reference comparison은 FESA HDF5 dataset을 deterministic row record로 읽어 `reference/<model-id>/<model-id>_displacements.csv`, `<model-id>_reactions.csv`, `<model-id>_internalforces.csv`, `<model-id>_stresses.csv`와 직접 비교한다.
|
|
```
|
|
|
|
- [ ] **Step 3: Update PRD/architecture/ADR**
|
|
|
|
Ensure the active decision in `docs/PRD.md`, `docs/ARCHITECTURE.md`, and `docs/ADR.md` contains these three facts:
|
|
|
|
```markdown
|
|
- FESA solver writes `results.h5`.
|
|
- Abaqus reference results are CSV files under `reference/<model-id>/`.
|
|
- Verification compares FESA HDF5 rows with Abaqus reference CSV rows using documented IDs, components, units, coordinate system, step/frame identity, and tolerance.
|
|
```
|
|
|
|
Remove or rewrite claims that `reference.h5` is required or that reference CSV files are generated from reference HDF5.
|
|
|
|
- [ ] **Step 4: Run targeted stale-text check**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
rg -n "reference\.h5|stored reference HDF5|results\.h5.*reference\.h5|derived from reference\.h5|references/<feature-id>/<model-id>/" AGENTS.md docs\ProjectInitialPlanNote.md docs\PRD.md docs\ARCHITECTURE.md docs\ADR.md
|
|
```
|
|
|
|
Expected: no matches in these active project-level docs.
|
|
|
|
Commit:
|
|
|
|
```powershell
|
|
git add AGENTS.md docs/ProjectInitialPlanNote.md docs/PRD.md docs/ARCHITECTURE.md docs/ADR.md
|
|
git commit -m "docs: define HDF5 to reference CSV comparison"
|
|
```
|
|
|
|
### Task 3: Update Reference Model Contracts
|
|
|
|
**Files:**
|
|
- Modify: `docs/reference-models/README.md`
|
|
- Modify: `.codex/skills/fesa-reference-models/SKILL.md`
|
|
- Modify: `.codex/skills/fesa-reference-models/agents/openai.yaml`
|
|
- Modify: `.codex/agents/reference-model-agent.toml`
|
|
|
|
- [ ] **Step 1: Rewrite the artifact bundle structure**
|
|
|
|
Use this structure in all four files:
|
|
|
|
```text
|
|
reference/
|
|
<model-id>/
|
|
model.inp
|
|
metadata.json
|
|
<model-id>_displacements.csv
|
|
<model-id>_reactions.csv
|
|
<model-id>_internalforces.csv
|
|
<model-id>_stresses.csv
|
|
README.md
|
|
```
|
|
|
|
Optional files:
|
|
|
|
```text
|
|
<model-id>_strains.csv
|
|
<model-id>_energy_or_residual.csv
|
|
<model-id>_<quantity>.csv
|
|
notes.md
|
|
```
|
|
|
|
- [ ] **Step 2: Rename the reference result section**
|
|
|
|
Change headings and output contracts from `Reference HDF5 and CSV View Requirements` to:
|
|
|
|
```markdown
|
|
## Abaqus Reference CSV Requirements
|
|
```
|
|
|
|
For `docs/reference-models/README.md`, include this exact required-file wording:
|
|
|
|
```markdown
|
|
- `<model-id>_displacements.csv`: required when nodal displacement is a verification quantity.
|
|
- `<model-id>_reactions.csv`: required when constrained DOF reactions or global equilibrium are verification quantities.
|
|
- `<model-id>_internalforces.csv`: required when element internal force is a verification quantity.
|
|
- `<model-id>_stresses.csv`: required when stress is a verification quantity.
|
|
```
|
|
|
|
- [ ] **Step 3: Update metadata contract**
|
|
|
|
Replace reference HDF5 metadata fields with CSV/reference provenance fields:
|
|
|
|
```json
|
|
{
|
|
"feature_id": "<feature-id>",
|
|
"model_id": "<model-id>",
|
|
"artifact_status": "draft | needs-reference-artifacts | ready-for-implementation-planning | blocked",
|
|
"input_file": "model.inp",
|
|
"abaqus_version": "<version or needs-user-decision>",
|
|
"generation_owner": "<person/procedure>",
|
|
"generation_date": "<YYYY-MM-DD>",
|
|
"source_documents": ["docs/requirements/<feature-id>.md"],
|
|
"units": "<unit system>",
|
|
"coordinate_system": "global Cartesian unless otherwise documented",
|
|
"analysis_type": "<analysis type>",
|
|
"element_types": ["<Abaqus element type>"],
|
|
"material_values": {},
|
|
"boundary_condition_summary": "<summary>",
|
|
"load_summary": "<summary>",
|
|
"output_requests": ["U", "RF", "S", "<feature-specific quantities>"],
|
|
"reference_csv_schema_version": "<version>",
|
|
"reference_csv_files": [
|
|
"<model-id>_displacements.csv",
|
|
"<model-id>_reactions.csv",
|
|
"<model-id>_internalforces.csv",
|
|
"<model-id>_stresses.csv"
|
|
],
|
|
"tolerance_policy": "<absolute/relative/norm policy>",
|
|
"limitations": ["<known limitation>"]
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Update coverage matrix**
|
|
|
|
Change coverage matrix columns from `hdf5_dataset` and `csv_view` to:
|
|
|
|
```markdown
|
|
| requirement_id | model_id | compared_quantity | fesa_hdf5_dataset | reference_csv | tolerance | verification_method | status |
|
|
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
| <req-id> | <model-id> | displacement | /steps/<step>/frames/<frame>/field_outputs/U | reference/<model-id>/<model-id>_displacements.csv | <policy> | hdf5-to-reference-csv | draft |
|
|
| <req-id> | <model-id> | reaction | /steps/<step>/frames/<frame>/field_outputs/RF | reference/<model-id>/<model-id>_reactions.csv | <policy> | hdf5-to-reference-csv | draft |
|
|
| <req-id> | <model-id> | internal force | /steps/<step>/frames/<frame>/field_outputs/element_forces | reference/<model-id>/<model-id>_internalforces.csv | <policy> | hdf5-to-reference-csv | draft |
|
|
| <req-id> | <model-id> | stress | /steps/<step>/frames/<frame>/field_outputs/S | reference/<model-id>/<model-id>_stresses.csv | <policy> | hdf5-to-reference-csv | draft |
|
|
```
|
|
|
|
- [ ] **Step 5: Verify Task 3 tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python -m unittest scripts.test_reference_model_agent_config scripts.test_fesa_solver_skills -v
|
|
```
|
|
|
|
Expected: reference model tests PASS; reference comparison skill tests may still fail until Task 4.
|
|
|
|
Commit:
|
|
|
|
```powershell
|
|
git add docs/reference-models/README.md .codex/skills/fesa-reference-models/SKILL.md .codex/skills/fesa-reference-models/agents/openai.yaml .codex/agents/reference-model-agent.toml
|
|
git commit -m "docs: update reference model CSV artifact contract"
|
|
```
|
|
|
|
### Task 4: Update Reference Verification Contracts
|
|
|
|
**Files:**
|
|
- Modify: `docs/reference-verifications/README.md`
|
|
- Modify: `.codex/skills/fesa-reference-comparison/SKILL.md`
|
|
- Modify: `.codex/skills/fesa-reference-comparison/agents/openai.yaml`
|
|
- Modify: `.codex/agents/reference-verification-agent.toml`
|
|
|
|
- [ ] **Step 1: Rewrite the mission**
|
|
|
|
Use this comparison statement:
|
|
|
|
```markdown
|
|
Reference Verification Agent는 Build/Test Executor Agent 통과 후 generated solver `results.h5`와 Abaqus reference CSV files를 tolerance 기준으로 비교한다. Reference CSV는 동일한 Abaqus `.inp` 모델을 Abaqus로 해석해 추출한 변위, 반력, 내력, 응력 결과이며, FESA HDF5 dataset에서 파생된 파일이 아니다.
|
|
```
|
|
|
|
- [ ] **Step 2: Update artifact check**
|
|
|
|
Artifact check must require:
|
|
|
|
```markdown
|
|
- `metadata.json`
|
|
- `model.inp`
|
|
- generated solver `results.h5`
|
|
- `reference/<model-id>/<model-id>_displacements.csv`
|
|
- `reference/<model-id>/<model-id>_reactions.csv`
|
|
- `reference/<model-id>/<model-id>_internalforces.csv`
|
|
- `reference/<model-id>/<model-id>_stresses.csv`
|
|
- reference CSV schema version
|
|
- FESA HDF5 schema version
|
|
- units
|
|
- coordinate system
|
|
- step/frame identity
|
|
- node/element ID matching rule
|
|
- output location
|
|
- component naming
|
|
- tolerance policy
|
|
```
|
|
|
|
- [ ] **Step 3: Update comparison targets**
|
|
|
|
Replace authoritative HDF5-vs-HDF5 rows with:
|
|
|
|
```markdown
|
|
| quantity | fesa_hdf5_dataset | reference_csv |
|
|
| --- | --- | --- |
|
|
| displacement | /steps/<step>/frames/<frame>/field_outputs/U | reference/<model-id>/<model-id>_displacements.csv |
|
|
| reaction | /steps/<step>/frames/<frame>/field_outputs/RF | reference/<model-id>/<model-id>_reactions.csv |
|
|
| internal force | /steps/<step>/frames/<frame>/field_outputs/element_forces | reference/<model-id>/<model-id>_internalforces.csv |
|
|
| stress | /steps/<step>/frames/<frame>/field_outputs/S | reference/<model-id>/<model-id>_stresses.csv |
|
|
```
|
|
|
|
- [ ] **Step 4: Update report template**
|
|
|
|
Change `Artifact Inventory` rows to:
|
|
|
|
```markdown
|
|
| item | path | status | notes |
|
|
| --- | --- | --- | --- |
|
|
| reference_model_dir | reference/<model-id>/ | present | missing | <notes> |
|
|
| reference_input | reference/<model-id>/model.inp | present | missing | <input summary> |
|
|
| metadata | reference/<model-id>/metadata.json | present | missing | <provenance summary> |
|
|
| reference_displacements_csv | reference/<model-id>/<model-id>_displacements.csv | present | missing | <row/schema summary> |
|
|
| reference_reactions_csv | reference/<model-id>/<model-id>_reactions.csv | present | missing | <row/schema summary> |
|
|
| reference_internalforces_csv | reference/<model-id>/<model-id>_internalforces.csv | present | missing | <row/schema summary> |
|
|
| reference_stresses_csv | reference/<model-id>/<model-id>_stresses.csv | present | missing | <row/schema summary> |
|
|
| solver_hdf5 | <solver output directory>/results.h5 | present | missing | <schema summary> |
|
|
| solver_debug_csv_view | <solver output directory>/csv/ | present | missing | <optional, derived from results.h5 only> |
|
|
```
|
|
|
|
Change `Quantity Results` columns to:
|
|
|
|
```markdown
|
|
| quantity | model_id | fesa_hdf5_dataset | reference_csv | compared_rows | missing_rows | extra_rows | max_abs_error | max_rel_error | rms_error | norm_error | worst_id | worst_component | result |
|
|
```
|
|
|
|
- [ ] **Step 5: Update skill and UI metadata prompts**
|
|
|
|
Set `.codex/skills/fesa-reference-comparison/agents/openai.yaml` default prompt to:
|
|
|
|
```yaml
|
|
default_prompt: "Use $fesa-reference-comparison to compare FESA solver results.h5 against Abaqus reference CSV files."
|
|
```
|
|
|
|
The skill `description` should include:
|
|
|
|
```text
|
|
FESA solver HDF5 results against Abaqus reference CSV files
|
|
```
|
|
|
|
- [ ] **Step 6: Verify Task 4 tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python -m unittest scripts.test_reference_verification_agent_config scripts.test_fesa_solver_skills -v
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
Commit:
|
|
|
|
```powershell
|
|
git add docs/reference-verifications/README.md .codex/skills/fesa-reference-comparison/SKILL.md .codex/skills/fesa-reference-comparison/agents/openai.yaml .codex/agents/reference-verification-agent.toml
|
|
git commit -m "docs: compare solver HDF5 with Abaqus reference CSV"
|
|
```
|
|
|
|
### Task 5: Update Upstream and Downstream Workflow Docs
|
|
|
|
**Files:**
|
|
- Modify: `docs/SOLVER_AGENT_DESIGN.md`
|
|
- Modify: `docs/SOLVER_SKILL_DESIGN.md`
|
|
- Modify: `docs/io-definitions/README.md`
|
|
- Modify: `docs/implementation-plans/README.md`
|
|
- Modify: `docs/physics-evaluations/README.md`
|
|
- Modify: `docs/requirements/README.md`
|
|
- Modify: `.codex/skills/fesa-io-contract/SKILL.md`
|
|
- Modify: `.codex/skills/fesa-physics-sanity/SKILL.md`
|
|
- Modify: `.codex/skills/fesa-cpp-msvc-tdd/SKILL.md`
|
|
- Modify: `.codex/skills/fesa-release-readiness/SKILL.md`
|
|
- Modify: `.codex/skills/fesa-requirements-baseline/SKILL.md`
|
|
- Modify: `.codex/agents/io-definition-agent.toml`
|
|
- Modify: `.codex/agents/implementation-planning-agent.toml`
|
|
- Modify: `.codex/agents/implementation-agent.toml`
|
|
- Modify: `.codex/agents/physics-evaluation-agent.toml`
|
|
- Modify: `.codex/agents/release-agent.toml`
|
|
- Modify: `.codex/agents/requirement-agent.toml`
|
|
- Modify: `.codex/agents/coordinator-agent.toml`
|
|
|
|
- [ ] **Step 1: Update I/O contract wording**
|
|
|
|
Keep FESA `results.h5` authoritative. Reframe CSV schema sections as comparison row schema, not reference HDF5-derived view:
|
|
|
|
```markdown
|
|
## FESA HDF5 to Reference CSV Comparison Schema
|
|
|
|
FESA solver output is `results.h5`. Comparison tooling reads required HDF5 datasets and maps them to deterministic row records that can be matched against Abaqus reference CSV files under `reference/<model-id>/`.
|
|
```
|
|
|
|
Use `reference_csv_schema_version` for reference CSV contracts and `hdf5_schema_version` for FESA output contracts.
|
|
|
|
- [ ] **Step 2: Update implementation planning data flow**
|
|
|
|
Change data-flow contract to:
|
|
|
|
```markdown
|
|
1. Abaqus `.inp` input follows docs/io-definitions/<feature-id>-io.md.
|
|
2. Parser/I/O path maps model data and history data into the internal semantic model.
|
|
3. Solver path produces authoritative `results.h5` with displacement, reaction, internal force, stress, or feature-specific result datasets.
|
|
4. Reference artifacts are Abaqus-generated CSV files under `reference/<model-id>/`.
|
|
5. Reference comparison tests compare FESA `results.h5` rows against `reference/<model-id>/<model-id>_*.csv` rows.
|
|
```
|
|
|
|
- [ ] **Step 3: Update physics evidence inventory**
|
|
|
|
Change physics evidence references from solver/reference HDF5 to:
|
|
|
|
```markdown
|
|
- checked solver HDF5 file: `<solver output directory>/results.h5`
|
|
- checked Abaqus reference CSV files under `reference/<model-id>/`
|
|
- optional FESA deterministic CSV view derived from `results.h5` for review only
|
|
```
|
|
|
|
Artifact inventory should not include `reference_hdf5`.
|
|
|
|
- [ ] **Step 4: Update global agent/skill boundary text**
|
|
|
|
Replace stale boundary sentences with:
|
|
|
|
```text
|
|
Do not generate or modify Abaqus reference CSV files.
|
|
```
|
|
|
|
For agents that mention deterministic CSV views derived from FESA HDF5, use:
|
|
|
|
```text
|
|
Do not treat FESA debug CSV views as authoritative solver output or reference artifacts.
|
|
```
|
|
|
|
Only add the second sentence where the agent actually discusses debug/exported FESA CSV views.
|
|
|
|
- [ ] **Step 5: Verify upstream/downstream tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python -m unittest scripts.test_io_definition_agent_config scripts.test_implementation_planning_agent_config scripts.test_physics_evaluation_agent_config scripts.test_requirement_agent_config scripts.test_release_agent_config -v
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
Commit:
|
|
|
|
```powershell
|
|
git add docs/SOLVER_AGENT_DESIGN.md docs/SOLVER_SKILL_DESIGN.md docs/io-definitions/README.md docs/implementation-plans/README.md docs/physics-evaluations/README.md docs/requirements/README.md .codex/skills/fesa-io-contract/SKILL.md .codex/skills/fesa-physics-sanity/SKILL.md .codex/skills/fesa-cpp-msvc-tdd/SKILL.md .codex/skills/fesa-release-readiness/SKILL.md .codex/skills/fesa-requirements-baseline/SKILL.md .codex/agents/io-definition-agent.toml .codex/agents/implementation-planning-agent.toml .codex/agents/implementation-agent.toml .codex/agents/physics-evaluation-agent.toml .codex/agents/release-agent.toml .codex/agents/requirement-agent.toml .codex/agents/coordinator-agent.toml
|
|
git commit -m "docs: align workflow agents with reference CSV artifacts"
|
|
```
|
|
|
|
### Task 6: Add Stale Contract Guard
|
|
|
|
**Files:**
|
|
- Modify: `scripts/test_fesa_solver_skills.py`
|
|
|
|
- [ ] **Step 1: Add a stale phrase scanner for active contract files**
|
|
|
|
Add this test to `scripts/test_fesa_solver_skills.py`:
|
|
|
|
```python
|
|
ACTIVE_CONTRACT_FILES = (
|
|
ROOT / "AGENTS.md",
|
|
ROOT / "docs" / "ProjectInitialPlanNote.md",
|
|
ROOT / "docs" / "PRD.md",
|
|
ROOT / "docs" / "ARCHITECTURE.md",
|
|
ROOT / "docs" / "ADR.md",
|
|
ROOT / "docs" / "SOLVER_AGENT_DESIGN.md",
|
|
ROOT / "docs" / "SOLVER_SKILL_DESIGN.md",
|
|
ROOT / "docs" / "reference-models" / "README.md",
|
|
ROOT / "docs" / "reference-verifications" / "README.md",
|
|
ROOT / "docs" / "io-definitions" / "README.md",
|
|
ROOT / "docs" / "implementation-plans" / "README.md",
|
|
ROOT / "docs" / "physics-evaluations" / "README.md",
|
|
ROOT / "docs" / "requirements" / "README.md",
|
|
ROOT / ".codex" / "agents" / "reference-model-agent.toml",
|
|
ROOT / ".codex" / "agents" / "reference-verification-agent.toml",
|
|
ROOT / ".codex" / "agents" / "io-definition-agent.toml",
|
|
ROOT / ".codex" / "agents" / "implementation-planning-agent.toml",
|
|
ROOT / ".codex" / "agents" / "implementation-agent.toml",
|
|
ROOT / ".codex" / "agents" / "physics-evaluation-agent.toml",
|
|
ROOT / ".codex" / "agents" / "release-agent.toml",
|
|
ROOT / ".codex" / "agents" / "requirement-agent.toml",
|
|
ROOT / ".codex" / "agents" / "coordinator-agent.toml",
|
|
ROOT / ".codex" / "skills" / "fesa-reference-models" / "SKILL.md",
|
|
ROOT / ".codex" / "skills" / "fesa-reference-comparison" / "SKILL.md",
|
|
ROOT / ".codex" / "skills" / "fesa-io-contract" / "SKILL.md",
|
|
ROOT / ".codex" / "skills" / "fesa-physics-sanity" / "SKILL.md",
|
|
ROOT / ".codex" / "skills" / "fesa-cpp-msvc-tdd" / "SKILL.md",
|
|
ROOT / ".codex" / "skills" / "fesa-release-readiness" / "SKILL.md",
|
|
ROOT / ".codex" / "skills" / "fesa-requirements-baseline" / "SKILL.md",
|
|
)
|
|
|
|
STALE_REFERENCE_CONTRACT_PHRASES = (
|
|
"reference.h5",
|
|
"stored reference HDF5",
|
|
"reference HDF5 artifact",
|
|
"results.h5 and reference.h5",
|
|
"results.h5` and `reference.h5",
|
|
"derived from reference.h5",
|
|
"references/<feature-id>/<model-id>/",
|
|
)
|
|
|
|
def test_active_contracts_do_not_require_reference_hdf5(self):
|
|
for path in ACTIVE_CONTRACT_FILES:
|
|
with self.subTest(path=str(path.relative_to(ROOT))):
|
|
text = path.read_text(encoding="utf-8")
|
|
for stale_phrase in STALE_REFERENCE_CONTRACT_PHRASES:
|
|
self.assertNotIn(stale_phrase, text)
|
|
```
|
|
|
|
This intentionally excludes `docs/superpowers/plans/` so old implementation plans remain historical records.
|
|
|
|
- [ ] **Step 2: Verify stale guard catches active contract drift**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python -m unittest scripts.test_fesa_solver_skills.FesaSolverSkillTests.test_active_contracts_do_not_require_reference_hdf5 -v
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
Commit:
|
|
|
|
```powershell
|
|
git add scripts/test_fesa_solver_skills.py
|
|
git commit -m "test: guard against reference HDF5 contract drift"
|
|
```
|
|
|
|
### Task 7: Full Verification
|
|
|
|
**Files:**
|
|
- No new edits expected.
|
|
|
|
- [ ] **Step 1: Run all Python contract tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python -m unittest discover -s scripts -p "test_*.py"
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 2: Run workspace validation**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
python scripts/validate_workspace.py
|
|
```
|
|
|
|
Expected: exit code 0. If no C++ targets exist, the current acceptable message is:
|
|
|
|
```text
|
|
No C++ validation commands configured.
|
|
Add CMakeLists.txt or set HARNESS_VALIDATION_COMMANDS.
|
|
```
|
|
|
|
- [ ] **Step 3: Run manual stale-text audit across active files**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
rg -n "reference\.h5|stored reference HDF5|reference HDF5 artifact|results\.h5.*reference\.h5|derived from reference\.h5|references/<feature-id>/<model-id>/" AGENTS.md docs .codex scripts
|
|
```
|
|
|
|
Expected:
|
|
|
|
- No matches in active docs, active skills, active agents, or tests.
|
|
- Matches are acceptable only in `docs/superpowers/plans/` historical plan files if they document old work.
|
|
|
|
- [ ] **Step 4: Review final diff**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git diff --stat
|
|
git diff --check
|
|
```
|
|
|
|
Expected: no whitespace errors. Review that every changed file traces to replacing the old `reference.h5`/reference-HDF5 premise with FESA-HDF5-to-Abaqus-reference-CSV comparison.
|
|
|
|
Commit:
|
|
|
|
```powershell
|
|
git add AGENTS.md docs .codex scripts
|
|
git commit -m "docs: finalize reference CSV comparison contract"
|
|
```
|
|
|
|
## Completion Criteria
|
|
|
|
- Active contracts say FESA solver output is HDF5 `results.h5`.
|
|
- Active contracts say Abaqus reference output is CSV under `reference/<model-id>/`.
|
|
- Active contracts compare FESA HDF5 directly with reference CSV files for displacement, reaction, internal force, and stress.
|
|
- Active contracts state the reference CSVs come from Abaqus solving the same Abaqus `.inp` model used by FESA.
|
|
- `reference.h5` is not required anywhere in active docs, skills, agents, or contract tests.
|
|
- Python tests and workspace validation pass.
|
|
|