AbaqusSubroutineDev/docs/AbaqusUserSubroutineManual/INDEX_DESIGN.md

# Abaqus User Subroutine Manual Index Design and Implementation Plan

> **For agentic workers:** When regenerating this index, preserve the schema and run every validation check in this document before treating the result as usable retrieval metadata.

**Goal:** Provide an AI-agent-oriented routing index for every numbered leaf section in the chunked Abaqus User Subroutine Manual.

**Architecture:** `INDEX.md` is a Markdown document containing one searchable heading and one fenced YAML record per manual section. The externally supplied local manual chunks remain authoritative; the index stores concise retrieval metadata and explicit file-local source spans so an agent can load only the relevant text.

**Tech stack:** Markdown, YAML-compatible fenced records, PowerShell/Python-compatible line numbering, and repository-local validation commands.

## Global constraints

- Do not modify `AbaqusUserSubroutineManual_*.md` while building the index.
- Line numbers are 1-based and both `start_line` and `end_line` are inclusive.
- A multi-file section must list every source file span explicitly; agents must not infer omitted intermediate files.
- The manual text is authoritative. `title`, `summary`, `keywords`, and `content_anchors` are retrieval aids only.
- The 69 manual chunks are an external-local prerequisite and are not bundled by the index. An agent must stop if they are absent or their combined digest does not match `source_sha256`.
- Companion files under `images/` are required when referenced by a selected source span. They are excluded from `source_sha256` because they do not affect Markdown line locations; agents must check referenced asset existence separately.
- Section IDs are strings so values such as `1.1.10` and `A.1` retain their exact form.
- The index covers numbered leaf headings matching `N.N.N` and appendix leaf headings matching `A.N`.

## Files

- `INDEX_DESIGN.md`: schema, boundary rules, generation algorithm, and acceptance checks.
- `INDEX.md`: agent usage instructions followed by the generated section records.

No persistent generator is introduced. Regeneration may use a temporary script, but the resulting index must be verified independently against the source chunks.

## Record schema

Each record is introduced by a Markdown heading of the form `## <section_id> — <display_name>`. The fenced YAML object uses these fields:

| Field | Required | Meaning |
|---|---:|---|
| `section_id` | yes | Exact manual leaf section identifier. |
| `parent_section` | yes | Parent identifier, such as `1.1` or `A`. |
| `symbol` | yes | Unescaped subroutine symbol; `null` for utility-topic and appendix records without one symbol. |
| `products` | yes | Ordered applicability list containing one or more of `Abaqus/Standard`, `Abaqus/Explicit`, `Abaqus/CFD`, or `Abaqus/Aqua`. |
| `entry_type` | yes | `user_subroutine`, `utility_topic`, or `appendix`. |
| `title` | yes | Exact text following the symbol in the source heading, or the full utility/appendix heading. |
| `summary` | yes | One concise routing sentence derived from the heading without adding unsupported behavior. |
| `keywords` | yes | Lowercase search terms plus the exact symbol when one exists. |
| `source_ranges` | yes | Ordered list of file-local inclusive spans containing the section body. |
| `source_pages` | yes | First and last source-page markers associated with nonblank indexed content; `null` if absent. |
| `heading` | yes | Exact source file, line, and unescaped heading text used as a stable fallback anchor. |
| `content_anchors` | yes | Ordered locations of high-value headings such as Overview, Interface, Variables, and Examples. May be empty. |

Markdown escaping is removed from metadata values. For example, `FRIC\_COEF` in a source heading is stored as `FRIC_COEF`.

The document-level metadata also records `schema_version`, generation date, line/range conventions, source prerequisite status, source file and section counts, and `source_sha256`. The SHA-256 digest is calculated over each numerically ordered source filename and its bytes, separated by null bytes, so agents can detect missing or stale line metadata after a manual chunk changes.

## Section boundary algorithm

1. Sort `AbaqusUserSubroutineManual_NNN.md` by numeric suffix.
2. Scan the files in that order and collect leaf headings matching `^# (\d+\.\d+\.\d+|A\.\d+) `.
3. Start each record at its leaf heading line.
4. End the record immediately before the next leaf heading or structural boundary. Structural boundaries include numbered chapter headings such as `# 2. Utility Routines`, numbered parent headings such as `# 1.2 Abaqus/Explicit subroutines`, `# Appendix A:`, and `# About SIMULIA`. In particular, `# 2. Utility Routines` ends Section `1.3.2`, `# Appendix A:` ends Section `2.1.23`, and `# About SIMULIA` ends Section `A.2`.
5. Walk backward from the raw boundary and exclude trailing blank lines and trailing `<!-- source-page: ... -->` markers. This makes `end_line` point to substantive section content.
6. If the section crosses a chunk boundary, emit the start-file tail, each complete intermediate file, and the final-file prefix as separate `source_ranges` entries.
7. Determine `source_pages.start` from the nearest source-page marker at or before the heading. Determine `source_pages.end` from the last marker at or before the final substantive line.
8. Parse an explicit source `Product:` or `Products:` declaration for every record and retain every listed product in source order. When a user-subroutine heading has no explicit declaration, fall back to its numbered parent. Appendix records apply to Standard, Explicit, and CFD. This rule preserves the additional Aqua applicability declared by UWAVE and VWAVE.

For example, Section `1.1.1` starts at line 96 of `_002.md`; its explicit spans are `_002.md:96-249` and `_003.md:1-390`. The trailing page marker and blank line are not part of the content span.

## Summary and keyword rules

- For a user subroutine, retain the exact manual purpose statement as `title`. Convert only its leading grammar for `summary`, such as `User subroutine to define ...` to `Defines ...`.
- For a utility topic, summarize the heading as `Describes utility routines for ...`. A concise high-value detail may be included only when explicitly supported by the section Overview.
- Derive keywords from the symbol, applicable products, and meaningful title terms. Exclude common English stop words.
- Do not infer supported procedures, element families, tensor conventions, or update responsibilities into a summary. Agents must read the referenced source ranges for those facts.

## Content-anchor rules

Record high-value H1 headings within a section when they identify one of the following:

- references or overview;
- an interface;
- variables supplied, defined, returned, or passed for information;
- an example;
- compile/link guidance;
- thread-local or global array guidance.

Do not index every argument-name heading. That would enlarge the index without improving routing.

## Agent retrieval contract

1. Confirm that the 69 required source chunks exist beside the index and reproduce the document-level `source_sha256`; stop on absence or mismatch. If the selected spans reference `images/...`, also confirm those companion assets exist.
2. Search `INDEX.md` by exact `symbol`, `section_id`, product, or keyword.
3. Read the complete YAML record.
4. Load every listed `source_ranges` span in order.
5. Use `content_anchors` to jump to implementation-relevant subsections when appropriate.
6. Treat the source manual as authoritative and cite the manual span, not the generated summary, in downstream engineering artifacts.

## Validation and acceptance criteria

The index is accepted only when all checks pass:

- source scan count equals index record count;
- all required external-local source files exist and their digest matches `source_sha256`;
- every companion image referenced by a source span selected for use exists under `images/`;
- every source leaf heading appears exactly once in the index;
- section IDs are unique and numerically ordered within each parent;
- every file exists and every range satisfies `1 <= start_line <= end_line <= file_line_count`;
- source ranges are ordered, nonoverlapping, and cover the section from its heading through its final substantive line;
- each `heading.file` and `heading.line` resolves to the indexed heading text;
- all content anchors fall inside the record's source ranges and match their source heading;
- CREEP regression check resolves to `_002.md:96-249` and `_003.md:1-390`;
- `SMACfdUserVelocityBC` regression check resolves to `_059.md:178-374` and `_060.md:1-13`;
- `ALLOCATABLE ARRAYS` regression check resolves to `_066.md:99-245`, `_067.md:1-383`, and `_068.md:1-222`;
- `git diff --check` reports no whitespace errors in the created documents.

## Implementation checklist

- [x] Extract and classify all numbered leaf headings.
- [x] Calculate explicit source spans and source-page bounds.
- [x] Generate summaries, keywords, and selected content anchors.
- [x] Write `INDEX.md` using one Markdown heading plus one YAML record per section.
- [x] Run structural, range, heading, anchor, and regression validation.
- [x] Record the verification commands and results in `PROGRESS.md`.

## Maintenance

Regenerate the complete file when chunk contents change; do not hand-adjust isolated line numbers. Increment `schema_version` only when field meaning or required fields change. Update `generated_on` for content-only regeneration and verify the complete index again. This design uses schema version 2, which replaces the original scalar `product` field with the source-backed `products` list.