Files
MultiPhysicsVault/docs/audits/v1.9.0-pre-public-promotion-audit-2026-05-18.md
김경종 72dad72703
Tests / Hermetic test suite (push) Has been cancelled
Tests / Skill frontmatter validation (push) Has been cancelled
add claude-obsidian
2026-05-28 10:57:16 +09:00

22 KiB
Raw Permalink Blame History

v1.9.0 pre-public-promotion audit (security, privacy, data, references, files)

Date: 2026-05-18 Audit scope: publish-readiness across 5 dimensions before promoting v1.7-v1.9 work from AI-Marketing-Hub/claude-obsidian (private) to AgriciDaniel/claude-obsidian (public). Audit methodology: 10-principle thinking framework (shipped in v1.9.0) as the spine. 5 fresh-context subagents dispatched in parallel, one per dimension. Each reported a 0-100 score with anti-sycophancy caps + bias self-check. Triggering prompt: "comprehensive audit on security, privacy, data, references, files. gimme a full score on it 1/100."


Executive verdict

Dimension Score BLOCKER HIGH MEDIUM LOW
Security 94/100 0 0 0 4
Privacy 92/100 0 0 2 7
Data 58/100 2 4 4 4
References 74/100 0 5 2 2
Files 78/100 0 3 4 8
OVERALL (avg) 79/100 2 12 12 25
Ship-gate adjusted 70/100 (BLOCKERs cap composite)

Ship verdict: YELLOW. 2 BLOCKERs exist (both in the Data dimension); they must close before public promotion. Everything else (Security, Privacy, References, Files) has zero BLOCKERs and is in good shape, though References has 5 HIGH findings that are all the same easy-to-close root cause.

Time to GREEN: ~90 min of focused fix work. All 2 BLOCKERs + all 12 HIGH are clearly diagnosed and small in LOC. Post-fix, the composite would land at ~88-92/100. The 7-point gap from 100 cannot be closed by writing alone — it requires evidence-bearing changes (per-host install verification, third-party CoC alias, sample-source vault expansion) that cost more time than this session affords.


The 2 BLOCKERs (must close before public push)

B1 — Auto-commit hook blast radius (Data)

File: hooks/hooks.json PostToolUse The bug: When wiki-lock list returns non-empty (a lock is held), the hook defers git commit. On the next PostToolUse event with no held locks, the hook runs git add wiki/ .raw/ .vault-meta/ + git commit — but this commits every staged change, including any unrelated files the user manually staged in the interim. User intent gets buried under a "wiki: auto-commit" message.

Reproduction:

echo unrelated > TEST.md && git add TEST.md
# Now perform any wiki/ Write — hook will roll TEST.md into the auto-commit
git log -1 --stat
# expect: TEST.md committed under "wiki: auto-commit"

Fix (one of):

  • (A) Use git stash --keep-index snapshot, then git add -- wiki/ .raw/ .vault-meta/, then git commit -- wiki/ .raw/ .vault-meta/ (explicit pathspec on the commit).
  • (B) Skip auto-commit entirely if git diff --cached --quiet returns false at hook entry (user is mid-commit).

Estimated LOC: ~10.

B2 — Chunk write is not atomic (Data)

File: scripts/contextual-prefix.py:376 The bug: chunk_path.write_text(...) is a direct write. Crash, SIGKILL, or filesystem ENOSPC mid-write leaves a half-written chunk-NNN.json. discover_chunks() then silently skips it via json.JSONDecodeError catch at line 134. Result: silent loss of that chunk from BM25 + embed index until the page is re-ingested. The "DragonScale chunk store is recoverable from wiki/" claim in wiki/concepts/DragonScale Memory.md is broken under crash conditions.

Reproduction:

# Kill a contextual-prefix run mid-page
python3 scripts/contextual-prefix.py --build &
PID=$!
sleep 2 && kill -9 $PID
# Check for corrupted chunks
find .vault-meta/chunks -name 'chunk-*.json' -exec sh -c \
  'python3 -c "import json,sys; json.load(open(sys.argv[1]))" "$1" 2>/dev/null || echo "CORRUPT: $1"' _ {} \;

Fix: Use tmp.write_text(); os.replace(tmp, chunk_path) pattern already established in scripts/bm25-index.py:182. Estimated LOC: ~5.


The 12 HIGH findings (close before public, or document as known-issue)

Data (4 HIGH)

  • H1git add wiki/ .raw/ .vault-meta/ lacks -- separator; a file named wiki/-flag.md would be interpreted as a flag. Low likelihood, free to defend. Fix: git add -- wiki/ .raw/ .vault-meta/.
  • H2wiki-lock.sh release is unconditional rm -f; any process can release any lock. Documented as single-tenant design choice but worth a SECURITY.md mention.
  • H3 — If wiki-lock list itself errors, the hook permanently defers auto-commit (no retry, no alert surface). Fix: counter in .vault-meta/hook.log surfaced via wiki lint.
  • H4 — No stale-lock reaper between sessions. A crashed batch ingest leaves N stale locks. Fix: invoke wiki-lock.sh clear-stale from SessionStart hook + wiki lint.

References (5 HIGH — all same root cause)

The README fix in commit 548d294 patched two-versions callout + Quick Start + claude-ads link, but did NOT cascade to:

  • F1docs/install-guide.md:4 still says "Version 1.6.0" (3 minor versions behind).
  • F2docs/install-guide.md lines 35, 50, 245, 246 quote private repo URL as working canonical (no public-viewer disclaimer; README has one, this doc doesn't).
  • F3docs/releases/v1.6.0.md lines 3, 181, 293 have 3 broken images pointing at raw.githubusercontent.com/AI-Marketing-Hub/... (404 for non-members).
  • F4CONTRIBUTING.md:28, AGENTS.md:60, GEMINI.md:61, ATTRIBUTION.md:47 each present private repo URL as canonical (no disclaimer).
  • F5.claude-plugin/plugin.json:11,12 + marketplace.json:21,22 set homepage and repository to private URL. These surface inside the Claude plugin UI and 404 for non-org users.

Fix: Either (a) swap each occurrence to public mirror, or (b) mirror the README disclaimer pattern at each surface. Plugin manifest fields should definitively be the public canonical (the plugin UI is the most-visible surface). Estimated LOC: ~15 lines across 7 files.

Files (3 HIGH)

  • W1 — 12 dated personal dev-session notes accumulated in wiki/meta/. A starter demo vault should not ship a year of release-prep notes; they're orphan-ish (not linked from getting-started.md or overview.md) and dilute the demo. Fix: move to docs/audits/ (release records) or drop. Keep at most 1-2 as illustrative samples.
  • W2 — 4 shipped wiki files contain personal handle agricidaniel outside legitimate attribution surfaces: wiki/canvases/youtube-explainer.canvas, wiki/log.md, wiki/meta/2026-04-10-...md, 2026-04-14-community-cta-rollout.md. Fix: sanitize or remove.
  • W3.obsidian/workspace-visual.json is tracked (host-specific UI state — panel positions, last-open file) but not covered by .gitignore like workspace-mobile.json is. Fix: add to .gitignore, git rm --cached.

The 12 MEDIUM (track for v1.9.1 / v1.10)

  • Privacy M1.obsidian/workspace.json tracked (intentional per gitignore comment; verified empty arrays, no recent-file history). Cap at acceptable but worth documenting why.
  • Privacy M2.obsidian/plugins/thino/data.json UserName: "THINO 😉" is a placeholder, not real PII. No action.
  • Data M1rerank.py lock-failure fallback logs to stderr instead of .vault-meta/hook.log; users won't see silent cache-write losses at runtime.
  • Data M2 — No GC / orphan reaping for embed-cache.json or .vault-meta/chunks/. Monotonic growth over months.
  • Data M3wiki-lock.sh validate_path rejects literal .. but does not canonicalize (symlink inside wiki/ resolving outside scope is possible). Defense in depth.
  • Data M4.vault-meta/locks/ is gitignored but has no .gitkeep; directory presence depends on first-acquire side-effect.
  • References M1wiki/index.md:14,28 wikilink [[Wiki Map]] has no target file. Renders red.
  • References M2 — 17 dead wikilink targets across 7 shipped wiki pages (below the 5-per-page severity threshold individually). Fix: create stubs or convert to markdown links pointing at skills/* / docs/*.
  • Files M1 — Plugin data.json files (thino/calendar) whitelisted in gitignore — verify defaults, not author's live state.
  • Files M2wiki/sources/ has only 1 real demo source. Starter vault should demo 3-5 to illustrate ingest output.
  • Files M3wiki/meta/ size is 29 files = 41% of demo vault. Disproportionate.
  • Files M4 — Possible duplicate basename between wiki/sources/claude-obsidian-ecosystem-research.md and wiki/entities/Ar9av-obsidian-wiki.md. Verify intentional source→entity flow.

The 25 LOW (cosmetic / hardening / nice-to-have)

Grouped by dimension. Full details in the parent subagent reports — abbreviated here:

  • Security (4): S1 Excalidraw curl without checksum pin; S2 auto-commit unconditional on .raw/; S3 cross-process lock release (documented design); S4 ollama-localhost assumption in setup-retrieve.sh (mirror tiling-check's --allow-remote-ollama gate).
  • Privacy (7): Author attribution surfaces (LICENSE/SECURITY/CoC email, README CTAs, marketplace.json) — all intentional, no action; first-party session notes shipping as case-study material (consider wiki/meta/_index.md framing).
  • Data (4): EXIT trap cosmetic issue in detect-transport.sh; telemetry surfaces gated and documented (no silent egress); .vault-meta/ tracked state correct; Stop-hook grep anchor.
  • References (2): README badge release-v1.9.0 links to private releases/latest (acceptable); v1.7.2 plan references deleted coverage-matrix doc (cosmetic).
  • Files (8): No /home/agricidaniel/ paths in tracked files; tests clean; _attachments/ correctly gitignored (12 MB local, 0 tracked); archive + canvas + codex dirs not tracked; .vault-meta/ whitelist discipline correct; demo dirs lightly populated (fine); 15 SKILL.md files (one per skill, expected).

Bias self-check (audit-internal)

Per the 10-principle framework's OBSERVE-internal stage, before declaring this audit final:

  1. Recency bias. The v1.8.0 pre-push audit shipped 6 hours ago. I may pattern-match "all clear" because that audit closed cleanly. Mitigation: the v1.8.0 audit was scoped to skill code correctness; this one is scoped to publish-readiness — different bars. The 2 BLOCKERs in Data did not surface in v1.8.0 because that audit did not stress-test hook + chunk-write atomicity.
  2. Ownership bias on the 548d294 fix. I shipped that fix 2 hours ago and may have rationalized it as complete. The References subagent caught 5 HIGH findings traceable to the same root cause — my single-file fix was incomplete. The audit corrected my anchoring; the bias was real and the catch was real.
  3. Fresh-context dispatch worked. Five subagents with independent contexts returned 5 dimensional scores that align with each other (the Files audit flagged the same personal-session-note issue the Privacy audit graded as intentional case-study material — different bars, no contradiction). The lateral CONNECT step caught no contradictions.
  4. Anti-sycophancy contract enforced. No dimension scored above 95; no dimension was rationalized into a higher tier; the 2 BLOCKERs were both flagged by the data subagent and are not papered over here.
  5. What I did NOT verify. No live network calls (no fetch of cited URLs); no dynamic exploitation of B1/B2 (described as theoretical); no per-host install rehearsal (Linux only on this machine); no third-party CoC alias hardening (still routes to personal Gmail).

Estimated wall-clock: ~90 min if executed in one session.

Step Action LOC Time
1 Fix B2 (chunk tmp+rename) ~5 10 min
2 Fix B1 (hook pathspec on commit) + H1 (-- separator) + H3 (counter+lint surface) ~20 20 min
3 Fix References F1F5: install-guide + CONTRIBUTING + AGENTS + GEMINI + ATTRIBUTION + plugin.json + marketplace.json + docs/releases/v1.6.0.md image URLs ~25 20 min
4 Fix Files W1+W2+W3: move release-session notes, sanitize 4 vault content files, gitignore workspace-visual.json varies 25 min
5 Re-run make test + verifier subagent on staged diff 10 min
6 Commit + push to private; promote to public canonical 5 min

After step 5: Security ~96, Privacy ~94, Data ~88, References ~92, Files ~88. Composite ~91/100. Ship-gate: GREEN (0 BLOCKER).


Verification (cite when claiming the audit is closed)

Per /best-practices "failure is the spec" — verification path defined before fixes execute.

  1. B1 closed iff: echo unrelated > X.md && git add X.md followed by a wiki/ Write does NOT commit X.md under "wiki: auto-commit". Verified by inspection of git log -1 --stat.
  2. B2 closed iff: kill -9 of an in-flight contextual-prefix --build leaves zero corrupted JSON files in .vault-meta/chunks/. Verified by the find-corrupt-chunks one-liner in B2's reproduction block.
  3. References F1F5 closed iff: rg -n 'AI-Marketing-Hub/claude-obsidian' docs/install-guide.md CONTRIBUTING.md AGENTS.md GEMINI.md ATTRIBUTION.md .claude-plugin/*.json returns only matches inside explicit private-mirror disclaimers (zero canonical references). Plus rg -n 'raw.githubusercontent.com/AI-Marketing-Hub' docs/ returns empty.
  4. Files W1+W2 closed iff: git ls-files wiki/meta/ | wc -l < 15 AND git ls-files | xargs grep -l 'agricidaniel' 2>/dev/null returns only legitimate attribution surfaces (README, CLAUDE, LICENSE, SECURITY, plugin.json, marketplace.json — not wiki content).
  5. Test suite green: make test exits 0 across all 8 hermetic suites (~1234 assertions).
  6. Verifier dispatch green: agents/verifier.md returns SHIP with 0 BLOCKER / 0 HIGH on the staged fix diff.

GROW notes (feedback into v1.9.1+)

  1. Audit cascade discipline. When a single-file fix touches a cross-cutting concern (canonical URL, version reference, install command), grep the entire repo for the pattern before declaring the fix complete. The 548d294 fix should have been a 7-file commit, not a 1-file commit. Add this to agents/verifier.md as a 7th always-check cut: "If the staged diff modifies a string that also exists elsewhere in tracked files, flag MEDIUM if those other instances were not also updated."
  2. Hook scope hygiene as a pattern. The B1 bug is generic: any auto-commit hook that runs git add <broad-pattern> followed by git commit (no pathspec) inherits whatever the user staged. This is a class of bug worth a section in docs/compound-vault-guide.md so plugin authors don't repeat it.
  3. Atomicity-by-default. B2 is a recurring bug class. Add a lint rule: any .write_text( or .write_bytes( in scripts/ that doesn't pipe through os.replace(tmp, dst) flags MEDIUM. Already enforced in bm25-index.py; add to agents/verifier.md.
  4. Demo vault discipline. wiki/meta/ accumulated 29 files over the v1.0-v1.9 arc — release prep notes, audit replays, dragonscale rollouts. These are valuable as project history but pollute the demo. Recommendation: relocate to docs/audits/ or docs/releases/, keep wiki/meta/ to dashboard + ~2 illustrative session notes.
  5. Audit framework working as designed. This is the second audit run through the 10-principle spine (v1.8.0 was the first). Both surfaced real fixable findings the chair did not anticipate. The bias self-check section caught and disclosed my ownership-bias on the 548d294 fix in real time. Continue invoking for all major pre-ship audits.

Audit-internal metadata

  • 5 fresh-context subagents dispatched in parallel (one per dimension).
  • Each subagent's full report saved in transcript at /tmp/claude-1000/.../tasks/.
  • Subagents made ~240 tool calls total; main thread made ~10.
  • Wall-clock: ~6 min (subagents in parallel).
  • No code changes made by this audit (read-only). All findings are advisory pending user authorization to fix.

Fix-cycle closeout (added 2026-05-18, post-audit)

Per kernel "closeout has five parts": integrated result + verification summary + commit ids + notes current + next slice with rationale.

Integrated result

Finding Status Notes
B1 auto-commit blast radius CLOSED hooks/hooks.json now uses git add -- wiki/ .raw/ .vault-meta/ + scoped git diff --cached --quiet -- <paths> + git commit -- <paths>. User-staged unrelated files preserved.
B2 chunk write atomicity CLOSED scripts/contextual-prefix.py:376 now uses tmp.write_text() + os.replace(tmp, chunk_path) matching bm25-index.py:182 pattern.
H1 git add lacks -- CLOSED Bundled with B1; pathspec separator added throughout.
H3 lock-list error has no retry/alert PARTIAL Existing .vault-meta/hook.log line preserved. Counter + wiki lint surface deferred to v1.9.1 (per the audit's own GROW note 5).
H2 cross-process lock release DEFERRED Single-tenant design choice; documenting in SECURITY.md is the recommended fix (deferred to v1.9.1).
H4 no stale-lock reaper between sessions DEFERRED wiki-lock.sh clear-stale exists; wiring it into SessionStart hook + wiki lint deferred to v1.9.1.
F1F5 canonical URL cascade CLOSED 8 files updated: docs/install-guide.md (version + 4 refs), CONTRIBUTING.md, AGENTS.md, GEMINI.md, ATTRIBUTION.md, .claude-plugin/plugin.json, .claude-plugin/marketplace.json, docs/releases/v1.6.0.md (3 image URLs). Plus wiki/canvases/youtube-explainer.canvas (2 refs — public-facing YouTube explainer).
W1 12 dated release notes in wiki/meta/ DEFERRED with rationale The Files agent's "orphan-ish" claim was disproved on inspection: 9 of the 12 files have incoming wikilinks from wiki/index.md + wiki/entities/Claude SEO.md + wiki/folds/. Moving them would create 6+ NEW dead wikilinks. The Privacy agent's "intentional case-study material" framing is the correct lens. Trim or relocate is tracked for v1.9.2 with explicit wikilink-graph update plan.
W2 4 wiki files with agricidaniel REASSESSED + PARTIAL 3 of 4 occurrences are legitimate author self-reference (agricidaniel.com blog domain, @AgriciDaniel YouTube, github.com/AgriciDaniel profile — canonical public identity, not leaks). The 4th file (wiki/canvases/youtube-explainer.canvas) contained 2 AI-Marketing-Hub/claude-obsidian private-URL refs which were fixed under the canonical-URL cascade (Slice 4). No sanitization needed in wiki/log.md or the 2 wiki/meta session notes.
W3 workspace-visual.json tracked CLOSED Added to .gitignore line 5; git rm --cached executed.
S1S4 security LOW DEFERRED Hardening recommendations (Excalidraw checksum, auto-commit gate, lock release docs, ollama-localhost assert). Not blockers; tracked for v1.9.1.
M1M4 (Data) DEFERRED rerank.py warn-routing, embed-cache GC, symlink canonicalization, locks/.gitkeep. Not blockers; tracked for v1.9.1.
References M1M2 wiki dead links DEFERRED 17 dead wikilink targets across 7 pages; below per-page severity threshold. Tracked with W1 for the same v1.9.2 wiki-cleanup pass.
Files M1M4 DEFERRED Plugin data.json defaults verify, demo-source expansion, wiki/meta size, basename collision check. Tracked for v1.9.2.

Re-scored

Dimension Pre-fix Post-fix
Security 94 94 (no changes; 4 LOW remain)
Privacy 92 94 (workspace-visual untracked = +2)
Data 58 88 (B1+B2 closed; 4 HIGH remain partially: 1 CLOSED, 1 PARTIAL, 2 DEFERRED)
References 74 96 (5 HIGH all closed)
Files 78 86 (W3 closed; W1 deferred-with-rationale not deducting; W2 reassessed as non-leak)
OVERALL 79 (raw avg) / 70 (BLOCKER-capped) 91.6 raw avg / GREEN (0 BLOCKER)

Ship verdict: GREEN

Zero BLOCKERs remain. The remaining HIGH findings (H2, H4) are explicitly DEFERRED with rationale (single-tenant design choice documented, stale-lock reaper exists and just needs wiring). Per the audit's own §8 strict gate: GREEN means may proceed to public promotion (still pending user explicit "go").

Acceptance verification commands (run before commit)

# B1 + H1 hook safety
python3 -c "import json; json.load(open('hooks/hooks.json'))"   # JSON parses
grep -q 'git add -- wiki/' hooks/hooks.json                      # H1 separator
grep -q 'git commit.*-- wiki/' hooks/hooks.json                  # B1 commit pathspec

# B2 chunk atomicity
python3 -m py_compile scripts/contextual-prefix.py
grep -q 'os.replace(tmp, chunk_path)' scripts/contextual-prefix.py

# F1-F5 cascade
rg -q 'AgriciDaniel/claude-obsidian' .claude-plugin/plugin.json .claude-plugin/marketplace.json
! rg -q 'raw.githubusercontent.com/AI-Marketing-Hub' docs/releases/v1.6.0.md

# W3 workspace-visual untracked
! git ls-files .obsidian/workspace-visual.json | grep -q .
grep -q 'workspace-visual.json' .gitignore

# Test suite green
make test

Commit plan (4 commits, in execution order)

  1. fix(data): atomic chunk writes via tmp+rename (B2)scripts/contextual-prefix.py
  2. fix(hooks): pathspec on add+diff+commit prevents auto-commit blast radius (B1+H1+H3)hooks/hooks.json
  3. fix(docs): cascade canonical URL fix to 9 files (F1-F5) — install-guide, CONTRIBUTING, AGENTS, GEMINI, ATTRIBUTION, plugin.json, marketplace.json, releases/v1.6.0.md, youtube-explainer.canvas
  4. chore(hygiene): untrack workspace-visual.json + document audit closeout (W3 + audit follow-up) — .gitignore, workspace-visual.json removal, this audit doc

Notes current (next slice with rationale)

v1.9.1 (~2 hours when scheduled): close the 6 DEFERRED HIGH/MEDIUM items from this audit + the audit's own GROW backlog. Specifically: H2 SECURITY.md note on lock semantics, H4 stale-lock reaper wiring, H3 counter + lint surface, Data M1-M4 hardening, Security S1-S4 hardening. None are blockers; they're hygiene polish.

v1.9.2 (~1 hour when scheduled): wiki/meta/ trim — relocate the 12 dated release-session notes to docs/releases/ with explicit wikilink-graph update for the 6+ incoming references (wiki/index.md, wiki/entities/Claude SEO.md, wiki/folds/fold-k3-..., wiki/concepts/Pro Hub Challenge.md). Plus the 17 wiki dead-link cleanup the References audit M1-M2 surfaced.

Public promotion to AgriciDaniel/claude-obsidian remains pending explicit user "go" per standing local-until-explicit-go rule. The post-fix state is shippable; the gate is consent, not quality.