add files

2026-04-30 17:05:19 +09:00
parent f3e01b5a8c
commit 7e985ae94a
135 changed files with 41205 additions and 0 deletions
@@ -0,0 +1,26 @@
+{
+  "project": "PDFtoMD",
+  "phase": "3-formula-pipeline",
+  "steps": [
+    {
+      "step": 0,
+      "name": "formula-block-detection",
+      "status": "pending"
+    },
+    {
+      "step": 1,
+      "name": "nougat-command-adapter",
+      "status": "pending"
+    },
+    {
+      "step": 2,
+      "name": "latex-validation-repair",
+      "status": "pending"
+    },
+    {
+      "step": 3,
+      "name": "formula-reference-links",
+      "status": "pending"
+    }
+  ]
+}
@@ -0,0 +1,37 @@
+# Step 0: formula-block-detection
+
+## Read First
+- /AGENTS.md
+- /PLAN.md
+- /PROGRESS.md
+- /docs/HARNESS.md
+- /docs/IMPLEMENTATION_PLAN.md
+- /docs/CONVERSION_POLICY.md
+- /phases/2-marker-adapter/step2.md
+
+## Task
+Implement formula candidate detection from normalized Marker blocks.
+
+Detect Marker equation blocks and text-pattern candidates while classifying inline versus block formulas based on block role and layout hints.
+
+## Sprint Contract
+- Done means: formula candidates are represented as internal objects ready for Nougat or Marker fallback.
+- Hard thresholds: ordinary currency-like dollar text is not blindly treated as math; inline/block distinction is tested; no Nougat invocation occurs yet.
+- Files owned: `src/pdftomd/formulas.py`, tests, `PROGRESS.md`, `phases/3-formula-pipeline/index.json`.
+- Dependencies: Phase 2 block normalization.
+
+## Acceptance Criteria
+```powershell
+python scripts\validate_workspace.py
+.\venv\python.exe -m pytest tests
+```
+
+## Verification
+1. Run the acceptance commands.
+2. Confirm tests include inline and block formula candidates.
+3. Update `PROGRESS.md` and this phase index.
+
+## Do Not
+- Do not call Nougat.
+- Do not render Markdown math.
+- Do not make regex the only source when structured block role exists.
@@ -0,0 +1,38 @@
+# Step 1: nougat-command-adapter
+
+## Read First
+- /AGENTS.md
+- /PLAN.md
+- /PROGRESS.md
+- /docs/HARNESS.md
+- /docs/IMPLEMENTATION_PLAN.md
+- /docs/TOOLCHAIN.md
+- /docs/CONVERSION_POLICY.md
+- /phases/3-formula-pipeline/step0.md
+
+## Task
+Implement the Nougat formula-only adapter boundary.
+
+The adapter should accept formula candidates and return LaTeX candidates or structured failure results. It should support a configured Nougat command path and be mockable in unit tests.
+
+## Sprint Contract
+- Done means: Nougat execution is isolated behind a testable command adapter and never becomes the primary document parser.
+- Hard thresholds: failures preserve Marker fallback text; tests do not require GPU/model execution by default; command path handling works on Windows.
+- Files owned: `src/pdftomd/formulas.py`, optional `src/pdftomd/nougat_adapter.py`, tests, `PROGRESS.md`, phase index.
+- Dependencies: Step 0 formula candidates and Phase 1 options.
+
+## Acceptance Criteria
+```powershell
+python scripts\validate_workspace.py
+.\venv\python.exe -m pytest tests
+```
+
+## Verification
+1. Run the acceptance commands.
+2. Confirm `.\venv\Scripts\nougat.exe --help` remains documented as an environment check, not a unit-test requirement.
+3. Update `PROGRESS.md` and this phase index.
+
+## Do Not
+- Do not parse whole PDFs with Nougat.
+- Do not require model downloads for normal unit tests.
+- Do not discard Marker source text on failure.
@@ -0,0 +1,38 @@
+# Step 2: latex-validation-repair
+
+## Read First
+- /AGENTS.md
+- /PLAN.md
+- /PROGRESS.md
+- /docs/HARNESS.md
+- /docs/IMPLEMENTATION_PLAN.md
+- /docs/CONVERSION_POLICY.md
+- /phases/0-harness-foundation/step3.md
+- /phases/3-formula-pipeline/step1.md
+
+## Task
+Implement LaTeX and Markdown math validation for formula outputs, plus explicit repair helpers for safe cases.
+
+Validation should cover delimiter balance and common `\begin{...}` / `\end{...}` pairs.
+
+## Sprint Contract
+- Done means: formula output validation returns actionable diagnostics and tested repairs for narrow, deterministic cases.
+- Hard thresholds: validation does not silently mutate math; unrepairable failures fall back to Marker text; delimiter tests include both inline and block math.
+- Files owned: `src/pdftomd/formulas.py`, `src/pdftomd/quality.py`, tests, `PROGRESS.md`, phase index.
+- Dependencies: Phase 0 quality gates and Step 1 Nougat adapter.
+
+## Acceptance Criteria
+```powershell
+python scripts\validate_workspace.py
+.\venv\python.exe -m pytest tests
+```
+
+## Verification
+1. Run the acceptance commands.
+2. Confirm broken delimiter and environment examples are covered.
+3. Update `PROGRESS.md` and this phase index.
+
+## Do Not
+- Do not build a broad LaTeX parser from scratch.
+- Do not use LLM repair.
+- Do not hide validation failures.
@@ -0,0 +1,37 @@
+# Step 3: formula-reference-links
+
+## Read First
+- /AGENTS.md
+- /PLAN.md
+- /PROGRESS.md
+- /docs/HARNESS.md
+- /docs/IMPLEMENTATION_PLAN.md
+- /docs/CONVERSION_POLICY.md
+- /phases/3-formula-pipeline/step2.md
+
+## Task
+Preserve formula numbering and body references as internal Markdown link targets when confidence is sufficient.
+
+Support common English and Korean reference patterns such as `Eq. (3)` and `식 (5)`.
+
+## Sprint Contract
+- Done means: formula anchors and reference rewrites are modeled and tested independently from final Markdown rendering.
+- Hard thresholds: low-confidence matches remain plain text; duplicate formula numbers do not create unstable anchors; references never point to missing anchors.
+- Files owned: `src/pdftomd/formulas.py`, reference model/tests, `PROGRESS.md`, phase index.
+- Dependencies: Steps 0 through 2.
+
+## Acceptance Criteria
+```powershell
+python scripts\validate_workspace.py
+.\venv\python.exe -m pytest tests
+```
+
+## Verification
+1. Run the acceptance commands.
+2. Confirm duplicate and missing reference cases are tested.
+3. Update `PROGRESS.md` and this phase index.
+
+## Do Not
+- Do not rewrite ambiguous references.
+- Do not render final Markdown chunks.
+- Do not remove the original formula number text.