motief/docs/solutions/best-practices/verify-transient-artifacts-against-canonical-sources.md

title	date	category	module	problem_type	component	severity	applies_when	tags
Verify Transient Session Artifacts Against Canonical Sources Before Compounding	2026-04-24	docs/solutions/best-practices	documentation	best_practice	documentation	medium	[Merging session ledgers or other transient artifacts into durable documentation Creating or updating docs/solutions/ entries from agent session outputs Extracting code constants, labels, or configurations from non-canonical files Compounding knowledge from temporary workspace artifacts]	[compound-documentation canonical-sources session-ledgers svd-labels verification transient-artifacts]

Verify Transient Session Artifacts Against Canonical Sources Before Compounding

Context

The ce-compound workflow involves merging session ledgers from thoughts/ledgers/ into durable documentation under docs/solutions/. During one such session, an agent was instructed to create a compounding doc based on a ledger file. The agent extracted SVD component labels directly from the ledger and wrote them into a new docs/solutions/ file.

The problem: the labels in the ledger were outdated. They had since been updated in the canonical source (analysis/config.py SVD_THEMES). The agent did not cross-check the ledger content against the canonical codebase before creating the durable doc. The user had to manually catch the discrepancy, instruct the agent to verify against canonical sources, and the inaccurate doc was deleted.

Guidance

Always cross-check transient artifacts against canonical codebase sources before creating or updating compounding documentation.

When merging session ledgers or any transient artifact into docs/solutions/:

1. Identify the canonical source for every factual claim

   • Code constants → check the defining module (e.g., analysis/config.py for SVD labels)
   • Data figures → check the pipeline output or database
   • Configuration → check the committed config file, not session notes

2. Do not treat ledger content as ground truth

   • Ledgers capture agent reasoning at a point in time
   • Code evolves after the ledger is written
   • A ledger is a memory aid, not a canonical reference

3. Diff the artifact against the canonical source

   • Read the current canonical file explicitly
   • Compare values, labels, constants, or conclusions
   • If they differ, use the canonical source and note the update

4. Flag discrepancies instead of silently using stale data

   • If the ledger contradicts the codebase, document the divergence
   • Explain when and why the canonical source changed
   • Do not propagate outdated information into durable docs

Why This Matters

Compounding documentation is meant to reduce future cognitive load. If it embeds stale or inaccurate information:

• Future agents (and humans) will trust it as truth. docs/solutions/ is explicitly referenced in AGENTS.md as a source of guidance. An inaccurate doc becomes a source of repeated errors.
• Outdated labels or constants propagate downstream. In this case, outdated SVD labels would have misled every future agent working on SVD analysis, visualization, or blog updates.
• Correcting a published doc costs more than verifying before writing. Deleting and rewriting a doc is cheap; discovering and fixing a stale doc months later requires archaeology.

When to Apply

Apply this guidance whenever you are:

• Creating a new docs/solutions/ entry from a session ledger, conversation log, or agent memory
• Updating an existing doc with insights from a transient artifact
• Extracting code snippets, constants, labels, or configurations from any file that is not the canonical definition
• Summarizing a debugging session where code was modified — the final committed code is canonical, not the session narrative

Examples

What Happened (Incorrect)

An agent read a session ledger containing SVD component labels and wrote them directly into a new docs/solutions/ file without checking analysis/config.py:

# ❌ INCORRECT: labels taken directly from stale ledger
Component 1: "Sociale zekerheid vs economische liberalisering"

The canonical source (analysis/config.py SVD_THEMES) had since been updated to reflect voting-pattern-based labels. The doc was inaccurate and had to be deleted.

What Should Have Happened (Correct)

# ✅ CORRECT: verify ledger claims against canonical source
1. Read analysis/config.py SVD_THEMES
2. Compare ledger labels with current SVD_THEMES values
3. Use the canonical labels from config.py
4. If the ledger contained useful context (e.g., reasoning about why labels changed),
   preserve that narrative but anchor all factual claims to the canonical source

Verification Pattern

# When documenting SVD labels, always read the canonical config
from analysis.config import SVD_THEMES

for comp_num, theme in SVD_THEMES.items():
    print(f"Component {comp_num}: {theme['label']}")
    # Use these values in the doc, not ledger-cached values

Related

• docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md — how SVD labels should be derived from voting patterns
• docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md — deriving quantitative claims from canonical pipeline outputs
• analysis/config.py — canonical source for SVD themes and other constants