177 lines
7.1 KiB
Markdown
177 lines
7.1 KiB
Markdown
# Verification Plan
|
|
|
|
## Purpose
|
|
This document defines how FESA will be verified against small focused tests and stored reference artifacts.
|
|
|
|
The project rule is strict: solver quality is maintained by comparing against reference examples and reference results.
|
|
|
|
## Source Basis
|
|
- The original Dvorkin-Bathe four-node shell paper presents a non-flat quadrilateral shell element for thin and thick shells and includes test/demonstration problems: https://web.mit.edu/kjb/www/Publications_Prior_to_1998/A_Continuum_Mechanics_Based_Four-Node_Shell_Element_for_General_Nonlinear_Analysis.pdf
|
|
- The MITC3+/MITC4+ benchmark paper states that MITC methods address shell locking and uses widely-used shell benchmarks with convergence studies: https://web.mit.edu/kjb/www/Principal_Publications/Performance_of_the_MITC3%2B_and_MITC4%2B_shell_elements_in_widely_used_benchmark_problems.pdf
|
|
- COMSOL's Scordelis-Lo example documents the benchmark geometry, material, load, constraints, and reference displacement context: https://doc.comsol.com/5.6/doc/com.comsol.help.models.sme.scordelis_lo_roof/scordelis_lo_roof.html
|
|
- NAFEMS benchmark surveys list nonlinear shell and structural benchmark classes for future phases: https://www.nafems.org/publications/pubguide/benchmarks/Page6/
|
|
|
|
## Verification Philosophy
|
|
FESA verification uses multiple layers:
|
|
|
|
1. Unit tests for low-level math, parser, and DOF management.
|
|
2. Element tests for MITC4 stiffness, rigid body modes, patch behavior, and drilling stabilization.
|
|
3. Assembly tests for small known systems.
|
|
4. Solver tests for constrained/reduced systems and reaction recovery.
|
|
5. Input-to-result integration tests using Abaqus-style `.inp` files.
|
|
6. Reference comparisons against stored Abaqus result artifacts under `reference/`.
|
|
|
|
A single large benchmark is not enough. Phase 1 should use many small models that isolate failure causes.
|
|
|
|
## Abaqus Availability
|
|
Abaqus is not available locally and must not be required by CI.
|
|
|
|
Rules:
|
|
- The user provides `.inp` files and solved reference result artifacts.
|
|
- FESA tests compare against stored artifacts only.
|
|
- Reference artifacts are treated as the numerical source of truth once accepted into `reference/`.
|
|
|
|
## Proposed Reference Folder Contract
|
|
Recommended layout:
|
|
|
|
```text
|
|
reference/
|
|
README.md
|
|
phase1-linear-static/
|
|
manifest.json
|
|
single-element-membrane/
|
|
model.inp
|
|
expected.json
|
|
notes.md
|
|
single-element-bending/
|
|
model.inp
|
|
expected.json
|
|
notes.md
|
|
cantilever-strip/
|
|
model.inp
|
|
expected.json
|
|
notes.md
|
|
scordelis-lo-roof/
|
|
model.inp
|
|
expected.json
|
|
notes.md
|
|
```
|
|
|
|
`manifest.json` should include:
|
|
|
|
```json
|
|
{
|
|
"schema_version": 1,
|
|
"cases": [
|
|
{
|
|
"name": "single-element-membrane",
|
|
"input": "single-element-membrane/model.inp",
|
|
"expected": "single-element-membrane/expected.json",
|
|
"analysis_type": "linear_static",
|
|
"element": "MITC4",
|
|
"source_solver": "Abaqus",
|
|
"source_solver_version": "provided-by-user",
|
|
"unit_system_note": "self-consistent",
|
|
"tags": ["phase1", "element", "membrane"]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Expected Result Artifact
|
|
Preferred `expected.json` structure:
|
|
|
|
```json
|
|
{
|
|
"schema_version": 1,
|
|
"case_name": "single-element-membrane",
|
|
"source": {
|
|
"solver": "Abaqus",
|
|
"version": "provided-by-user",
|
|
"created_by": "provided-by-user"
|
|
},
|
|
"units": {
|
|
"note": "self-consistent"
|
|
},
|
|
"comparisons": [
|
|
{
|
|
"path": "/results/steps/Step-1/frames/0/fieldOutputs/U",
|
|
"entity": {"type": "node", "id": 4},
|
|
"component": "UX",
|
|
"expected": 0.0,
|
|
"abs_tol": 1e-9,
|
|
"rel_tol": 1e-7
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
CSV or `.rpt` files may be stored as raw source artifacts, but structured JSON or HDF5 comparison artifacts should be used by automated tests.
|
|
|
|
## Tolerance Policy
|
|
Use absolute and relative tolerance:
|
|
|
|
```text
|
|
abs_error = abs(actual - expected)
|
|
rel_error = abs_error / max(abs(expected), reference_scale)
|
|
pass if abs_error <= abs_tol or rel_error <= rel_tol
|
|
```
|
|
|
|
Initial guidance:
|
|
- Parser and exact metadata tests: exact match.
|
|
- DOF mapping tests: exact integer match.
|
|
- Small linear algebra tests: `abs_tol = 1e-12`, `rel_tol = 1e-10`.
|
|
- Element stiffness symmetry: matrix norm tolerance around `1e-10` to `1e-9`, adjusted by stiffness scale.
|
|
- Reference displacement tests: start with `rel_tol = 1e-5` and refine after baseline agreement.
|
|
- Reaction equilibrium tests: use force-scale-based tolerances.
|
|
|
|
Final benchmark tolerances must be stored with each reference case.
|
|
|
|
## Phase 1 Benchmark Matrix
|
|
| Case | Purpose | Required Output |
|
|
|---|---|---|
|
|
| Parser smoke model | Verify Phase 1 keyword subset | Domain object counts, set membership |
|
|
| Single MITC4 membrane | Constant strain behavior | `U`, element strains/stresses if available |
|
|
| Single MITC4 bending | Bending stiffness sanity | `U`, rotations |
|
|
| Rigid body mode check | Verify near-zero internal forces without constraints | eigen/nullspace or controlled diagnostic |
|
|
| Constrained static sanity | Verify constrained DOF elimination | `U`, `RF` |
|
|
| Reaction balance | Verify full-vector reaction recovery | total load vs total reaction |
|
|
| Cantilever shell strip | Bending-dominated displacement convergence | tip `U`, support `RF` |
|
|
| Simply supported square plate | Thin shell/plate locking sensitivity | center displacement |
|
|
| Scordelis-Lo roof | Curved shell benchmark | midpoint vertical displacement |
|
|
| Pinched cylinder | Curved shell bending/membrane behavior | loaded-point displacement |
|
|
| Twisted beam | Warped shell and drilling sensitivity | tip displacement/rotation |
|
|
|
|
## Minimum Phase 1 Acceptance
|
|
Before MITC4 Phase 1 is considered credible:
|
|
- All parser smoke tests pass.
|
|
- `DofManager` free/constrained mapping tests pass.
|
|
- Reduced-system solve reconstructs full `U`.
|
|
- `RF = K_full * U_full - F_full` is tested.
|
|
- MITC4 element stiffness is symmetric within tolerance for linear elastic Phase 1.
|
|
- Rigid body modes do not create artificial membrane/bending stiffness beyond documented drilling stabilization effects.
|
|
- At least three stored Abaqus reference models pass: one single-element case, one simple multi-element plate/shell case, and one curved shell benchmark.
|
|
|
|
## Singular System Verification
|
|
Required negative tests:
|
|
- Model with no boundary conditions should fail with a singular-system diagnostic.
|
|
- Model with missing property should fail before assembly.
|
|
- Model with load on missing node or set should fail during input/model validation.
|
|
- Model with an unconstrained isolated node should identify free untouched DOFs.
|
|
- Model with only drilling DOF instability should mention weak or unconstrained rotational DOFs.
|
|
|
|
## What Not To Verify In Phase 1
|
|
- Mesh quality metrics such as aspect ratio, skew, warpage, Jacobian quality, or distortion warnings.
|
|
- Pressure loads.
|
|
- RBE2/RBE3.
|
|
- Nonlinear increments.
|
|
- Time integration.
|
|
- Thermal-stress coupling.
|
|
|
|
## User Inputs Needed
|
|
- First accepted `reference/` folder layout.
|
|
- At least one Abaqus `.inp` with solved displacement and reaction output.
|
|
- Preferred raw reference format: `.dat`, `.rpt`, `.csv`, `.json`, or `.h5`.
|
|
- Abaqus version used to generate the reference.
|
|
- Whether reference files are generated from Abaqus `S4` only for Phase 1.
|