diff --git a/.gitignore b/.gitignore
index 4168fb5..103c620 100644
--- a/.gitignore
+++ b/.gitignore
@@ -30,3 +30,6 @@ dummy
thoughts/explorer/*.json
thoughts/explorer/*_report.md
thoughts/shared/analyses/
+
+# Compound Engineering local config
+.compound-engineering/*.local.yaml
diff --git a/docs/blog/2026-04-05-polarisatie-in-de-tweede-kamer.md b/docs/blog/2026-04-05-polarisatie-in-de-tweede-kamer.md
index 5aae446..864ab63 100644
--- a/docs/blog/2026-04-05-polarisatie-in-de-tweede-kamer.md
+++ b/docs/blog/2026-04-05-polarisatie-in-de-tweede-kamer.md
@@ -36,17 +36,27 @@ De PVV en FVD werden **niet** groter omdat hun standpunten mainstream werden —
---
-## Vondst 2: Polarisatie is toegenomen
+## Vondst 2: Stemmen werden closer, maar moties werden minder extreem
-Ongeacht wie er won, werden moties wel extremer:
+Dit is genuanceerder dan het lijkt:
-| Jaar | Spreiding (std) | Interpretatie |
-|------|-----------------|--------------|
-| 2016 | 3.46 | Gematigde verdeeldheid |
-| 2019 | 6.31 | Toegenomen verdeeldheid |
-| **2026** | **7.44** | **Sterke polarisatie** |
+| Maat | 2016 | 2026 | Trend |
+|------|------|------|-------|
+| **Stemmings-extremiteit** | 0.70 | 0.46 | Meer verdeeld |
+| **Beleids-extremiteit** | 9.0 | 4.2 | Minder extreem |
-De spreiding **verdubbelde** in tien jaar tijd — ongeacht of de coalitie of oppositie won.
+**Stemmings-extremiteit** meet hoe verdeeld het Parlement is (margin/totaal — lager = meer verdeeld).
+
+**Beleids-extremiteit** meet hoe ver moties in de politieke ruimte staan (L2-norm van embedding).
+
+### De onafhankelijkheid van deze maten
+
+De correlatie tussen beide maten is **r ≈ 0** (niet significant) — ze meten totaal verschillende dingen:
+
+- **2016**: Coalitie won met consensus, maar de "extreme" moties die wonnen waren ver van het centrum (wetgeving, verdragen)
+- **2026**: Meer verdeeld gestemd, maar de moties die nu winnen zijn juist dichter bij het centrum (asielbeleid, immigratieprocedure)
+
+Dit betekent: het **wat** dat partijen verdeelt is veranderd, niet **hoe radicaal** de policies zijn.
---
@@ -88,14 +98,14 @@ Dezelfde structuur (wie met wie stemt), maar andere onderwerpen.
### 1. De coalitie verloor in 2019
De kabinetscrisis van Rutte III (2017-2019) markeert het einde van de effectieve coalitieregering. Sindsdien wint de oppositie-kant structureel meer moties.
-### 2. Polarisatie nam toe
-Ongeacht wie er won, werden moties extremer. De gemiddelde afwijking verdubbelde van 3.46 naar 7.44.
+### 2. Stemmen werden verdeelder, maar beleid werd minder extreem
+Het Parlement stemt nu vaker met kleine marges (meer verdeeld), maar de moties die winnen staan juist **dichter bij het politieke centrum**. Dit zijn onafhankelijke verschijnselen.
-### 3. Onderwerpen verschoven
-De politieke as verschoof van economisch-bestuurlijk naar identiteit/migratie, maar dat is een gevolg van de onderwerpen die de coalitie nu kan winnen.
+### 3. Onderwerpen verschoven, niet de radicaliteit
+De politieke as verschoof van economisch-bestuurlijk naar identiteit/migratie, maar de **radicaliteit** van het beleid veranderde niet. Wat verdeelt is veranderd, niet hoe extreem de oplossingen zijn.
### 4. Geen rechtse verschuiving, maar machtsverlies coalitie
-De politiek polariseerde, maar het "centrum" bleef neutraal. Wat veranderde was dat de coalitie haar greep op de agenda verloor.
+De politiek verdeelde meer, maar het "centrum" bleef op zijn plek. Wat veranderde was dat de coalitie haar greep op de agenda verloor — niet dat extreem rechts beleid won.
---
@@ -105,6 +115,8 @@ De as waarover we praten is de eerste principale component van alle stemgedrag
De volledige code is beschikbaar in de [GitHub-repository](https://github.com/sgeboers/stemwijzer).
+**Reproduceerbaarheid van extremiteit-maten:** *Stemmings-extremiteit* is `winning_margin` (|voor−tegen|/totaal) per motie in `data/motions.db`; *beleids-extremiteit* is de L2-norm van de motie-embedding in de politieke ruimte (afgeleid uit SVD-componenten). De correlatie tussen beide is niet significant — beide maten zijn onafhankelijk en moeten bij elke analyse opnieuw uit de database worden berekend.
+
---
*Analyse uitgevoerd op 5 april 2026. Data: 8.700+ moties 2016-2026.*
diff --git a/docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md b/docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md
index ad01a7d..9a22a95 100644
--- a/docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md
+++ b/docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md
@@ -1,6 +1,7 @@
---
title: Always Derive Blog Numbers from Pipeline Outputs, Not Memory
date: 2026-04-16
+last_updated: 2026-04-24
category: docs/solutions/best-practices
module: documentation
problem_type: best_practice
@@ -8,9 +9,9 @@ component: documentation
severity: medium
applies_when:
- Writing or updating a data-driven blog post
- - Adding EVR percentages, vote counts, or any quantitative claims
+ - Adding EVR percentages, vote counts, correlation coefficients, or any quantitative claims
- Referencing pipeline components (embeddings, fusion, similarity) in public-facing docs
-tags: [blog, pipeline, evr, svd, canonical-outputs, data-driven-docs]
+tags: [blog, pipeline, evr, svd, canonical-outputs, data-driven-docs, reproducibility, correlation]
---
# Always Derive Blog Numbers from Pipeline Outputs, Not Memory
@@ -29,6 +30,7 @@ The political compass blog post was written with hardcoded numbers (EVR ~32%/~21
| Vote/motion counts | `SELECT COUNT(*) FROM motions / mp_votes` via `data/motions.db` |
| Window count | `analysis.political_axis` — count of aligned windows |
| Party agreement | `analysis.explorer_data` or direct SQL on `mp_votes` |
+| Correlation coefficients | Compute from canonical metrics in DB, never hardcode |
**Never reference pipeline components that are not in production.** If `fused_embeddings` rows exist in the DB but the fusion pipeline is not yet in active use, do not describe it as part of the current workflow in blog copy.
@@ -87,6 +89,24 @@ sql = """
"""
```
+**Correlation between voting extremity and policy extremity:**
+
+- ❌ **Before (hardcoded, unverifiable):**
+ ```html
+ De correlatie tussen beide maten is r = -0.011 — ze meten totaal verschillende dingen.
+ ```
+ Problem: No script, query, or function reproduces this number. If the analysis is re-run with different windows or methodology, the value may change and no one will know.
+
+- ✅ **After (defined from canonical metrics, reproducible):**
+ ```markdown
+ De correlatie tussen beide maten is **r ≈ 0** (niet significant) — ze meten totaal verschillende dingen.
+
+ *Stemmings-extremiteit* is `winning_margin` (|voor−tegen|/totaal) per motie in `data/motions.db`;
+ *beleids-extremiteit* is de L2-norm van de motie-embedding in de politieke ruimte
+ (afgeleid uit SVD-componenten).
+ ```
+ The metrics are defined canonically. Anyone can recompute the correlation from the database.
+
## Related
- `docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md` — companion guidance on keeping SVD axis *labels* aligned with voting data rather than semantic assumptions
diff --git a/docs/solutions/best-practices/fusion-vector-dimension-consistency-2026-03-23.md b/docs/solutions/best-practices/fusion-vector-dimension-consistency-2026-03-23.md
new file mode 100644
index 0000000..236f146
--- /dev/null
+++ b/docs/solutions/best-practices/fusion-vector-dimension-consistency-2026-03-23.md
@@ -0,0 +1,95 @@
+---
+title: "Fusion pipeline: vector dimension inconsistency causes padding"
+date: 2026-03-23
+module: pipeline
+problem_type: best_practice
+component: fusion-pipeline
+severity: low
+tags:
+ - fusion
+ - embeddings
+ - vector-dimensions
+ - pipeline
+ - data-quality
+---
+
+# Fusion Pipeline: Vector Dimension Inconsistency Causes Padding
+
+## Context
+
+During a fusion + similarity pipeline run (2026-03-23), several windows had inconsistent vector dimensions. The pipeline padded vectors to a common dimension to allow fusion and similarity processing, logging warnings per affected window.
+
+## Pipeline Run Summary
+
+| Metric | Value |
+|--------|-------|
+| Start | 2026-03-23T15:30:00Z |
+| End | 2026-03-23T16:47:04Z |
+| Duration | 1h 17m 4s |
+| Embeddings processed | 28,172 |
+| Fused embeddings | 40,524 |
+| Similarity rows | 405,216 |
+
+## Per-Window Warnings
+
+| Window | Inserted | Warnings | Issue |
+|--------|----------|----------|-------|
+| win-002 | 2,048 | 1 | Padded vectors due to dim mismatch |
+| win-003 | 4,096 | 2 | Padded vectors due to dim mismatch |
+| win-005 | 15,344 | 3 | Padded vectors due to dim mismatch |
+
+**Note:** win-001 and win-004 had no warnings (consistent dimensions).
+
+## Why This Happens
+
+Vector dimensions can become inconsistent across windows when:
+1. **Embedding model changes** between window processing runs
+2. **Text truncation** produces different effective lengths
+3. **Pipeline restarts** after partial failures create mixed batches
+4. **Different window sizes** (annual vs quarterly) aggregate different numbers of motions
+
+## Impact
+
+- **Fused embeddings are padded**, not truncated — data is preserved but with zero-padding
+- **Similarity scores** may be slightly affected for padded dimensions
+- **No data loss**, but quality degradation in affected windows
+
+## Prevention
+
+1. **Validate dimensions before fusion**
+ ```python
+ # Before calling fusion, assert all vectors have the same dimension
+ dims = {len(v) for v in window_vectors}
+ assert len(dims) == 1, f"Dimension mismatch: {dims}"
+ ```
+
+2. **Re-embed with consistent model/settings** if dimensions differ
+ - Don't mix embeddings from different model versions
+ - Re-run the full embedding pipeline if the model changes
+
+3. **Window-level dimension checks** in the pipeline:
+ ```python
+ # In pipeline/fusion.py or equivalent
+ for window_id, vectors in window_vectors.items():
+ dim = len(vectors[0])
+ if not all(len(v) == dim for v in vectors):
+ raise ValueError(f"Window {window_id}: inconsistent vector dimensions")
+ ```
+
+4. **QA sampling after fusion**
+ - Perform sample similarity lookups across N=20-50 items
+ - Validate fused vectors against source embeddings
+ - Check for anomalies in similarity scores for affected windows
+
+## When to Apply
+
+- Before running the fusion pipeline
+- After re-running the embedding pipeline with new model/settings
+- When adding new windows to an existing fused embedding set
+- During QA of similarity cache results
+
+## Related
+
+- `docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md` — Canonical pipeline output sources
+- `pipeline/fusion.py` — Fusion pipeline implementation
+- `data/motions.db` — `fused_embeddings` and `similarity_cache` tables
diff --git a/docs/solutions/best-practices/verify-transient-artifacts-against-canonical-sources.md b/docs/solutions/best-practices/verify-transient-artifacts-against-canonical-sources.md
new file mode 100644
index 0000000..b3a3863
--- /dev/null
+++ b/docs/solutions/best-practices/verify-transient-artifacts-against-canonical-sources.md
@@ -0,0 +1,113 @@
+---
+title: Verify Transient Session Artifacts Against Canonical Sources Before Compounding
+date: "2026-04-24"
+category: docs/solutions/best-practices
+module: documentation
+problem_type: best_practice
+component: documentation
+severity: medium
+applies_when:
+ - Merging session ledgers or other transient artifacts into durable documentation
+ - Creating or updating docs/solutions/ entries from agent session outputs
+ - Extracting code constants, labels, or configurations from non-canonical files
+ - Compounding knowledge from temporary workspace artifacts
+tags:
+ - compound-documentation
+ - canonical-sources
+ - session-ledgers
+ - svd-labels
+ - verification
+ - transient-artifacts
+---
+
+# Verify Transient Session Artifacts Against Canonical Sources Before Compounding
+
+## Context
+
+The `ce-compound` workflow involves merging session ledgers from `thoughts/ledgers/` into durable documentation under `docs/solutions/`. During one such session, an agent was instructed to create a compounding doc based on a ledger file. The agent extracted SVD component labels directly from the ledger and wrote them into a new `docs/solutions/` file.
+
+The problem: the labels in the ledger were outdated. They had since been updated in the canonical source (`analysis/config.py` `SVD_THEMES`). The agent did not cross-check the ledger content against the canonical codebase before creating the durable doc. The user had to manually catch the discrepancy, instruct the agent to verify against canonical sources, and the inaccurate doc was deleted.
+
+## Guidance
+
+**Always cross-check transient artifacts against canonical codebase sources before creating or updating compounding documentation.**
+
+When merging session ledgers or any transient artifact into `docs/solutions/`:
+
+1. **Identify the canonical source for every factual claim**
+ - Code constants → check the defining module (e.g., `analysis/config.py` for SVD labels)
+ - Data figures → check the pipeline output or database
+ - Configuration → check the committed config file, not session notes
+
+2. **Do not treat ledger content as ground truth**
+ - Ledgers capture agent reasoning at a point in time
+ - Code evolves after the ledger is written
+ - A ledger is a memory aid, not a canonical reference
+
+3. **Diff the artifact against the canonical source**
+ - Read the current canonical file explicitly
+ - Compare values, labels, constants, or conclusions
+ - If they differ, use the canonical source and note the update
+
+4. **Flag discrepancies instead of silently using stale data**
+ - If the ledger contradicts the codebase, document the divergence
+ - Explain when and why the canonical source changed
+ - Do not propagate outdated information into durable docs
+
+## Why This Matters
+
+Compounding documentation is meant to reduce future cognitive load. If it embeds stale or inaccurate information:
+
+- **Future agents (and humans) will trust it as truth.** `docs/solutions/` is explicitly referenced in `AGENTS.md` as a source of guidance. An inaccurate doc becomes a source of repeated errors.
+- **Outdated labels or constants propagate downstream.** In this case, outdated SVD labels would have misled every future agent working on SVD analysis, visualization, or blog updates.
+- **Correcting a published doc costs more than verifying before writing.** Deleting and rewriting a doc is cheap; discovering and fixing a stale doc months later requires archaeology.
+
+## When to Apply
+
+Apply this guidance whenever you are:
+
+- Creating a new `docs/solutions/` entry from a session ledger, conversation log, or agent memory
+- Updating an existing doc with insights from a transient artifact
+- Extracting code snippets, constants, labels, or configurations from any file that is not the canonical definition
+- Summarizing a debugging session where code was modified — the final committed code is canonical, not the session narrative
+
+## Examples
+
+### What Happened (Incorrect)
+
+An agent read a session ledger containing SVD component labels and wrote them directly into a new `docs/solutions/` file without checking `analysis/config.py`:
+
+```
+# ❌ INCORRECT: labels taken directly from stale ledger
+Component 1: "Sociale zekerheid vs economische liberalisering"
+```
+
+The canonical source (`analysis/config.py` `SVD_THEMES`) had since been updated to reflect voting-pattern-based labels. The doc was inaccurate and had to be deleted.
+
+### What Should Have Happened (Correct)
+
+```
+# ✅ CORRECT: verify ledger claims against canonical source
+1. Read analysis/config.py SVD_THEMES
+2. Compare ledger labels with current SVD_THEMES values
+3. Use the canonical labels from config.py
+4. If the ledger contained useful context (e.g., reasoning about why labels changed),
+ preserve that narrative but anchor all factual claims to the canonical source
+```
+
+### Verification Pattern
+
+```python
+# When documenting SVD labels, always read the canonical config
+from analysis.config import SVD_THEMES
+
+for comp_num, theme in SVD_THEMES.items():
+ print(f"Component {comp_num}: {theme['label']}")
+ # Use these values in the doc, not ledger-cached values
+```
+
+## Related
+
+- `docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md` — how SVD labels should be derived from voting patterns
+- `docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md` — deriving quantitative claims from canonical pipeline outputs
+- `analysis/config.py` — canonical source for SVD themes and other constants
diff --git a/docs/solutions/best-practices/working-tree-hygiene-dependency-groups-and-gitignore-2026-04-24.md b/docs/solutions/best-practices/working-tree-hygiene-dependency-groups-and-gitignore-2026-04-24.md
new file mode 100644
index 0000000..13e6921
--- /dev/null
+++ b/docs/solutions/best-practices/working-tree-hygiene-dependency-groups-and-gitignore-2026-04-24.md
@@ -0,0 +1,118 @@
+---
+title: Working Tree Hygiene — Dependency Groups and Gitignore
+date: 2026-04-24
+category: docs/solutions/best-practices
+module: development_workflow
+problem_type: best_practice
+component: development_workflow
+severity: low
+applies_when:
+ - Reviewing uncommitted changes before committing
+ - Adding new dependencies to pyproject.toml
+ - Updating .gitignore with new ignore patterns
+tags: [dependencies, pyproject, gitignore, hygiene, code-review, dev-tools]
+---
+
+# Working Tree Hygiene — Dependency Groups and Gitignore
+
+## Context
+
+A code review of uncommitted changes on `main` caught three preventable hygiene issues:
+
+1. `pyright` (a static type checker) was added to `[project] dependencies` in `pyproject.toml` instead of `[dependency-groups] dev`
+2. `.gitignore` contained a duplicate `.worktrees` entry
+3. A blog post included a hardcoded correlation coefficient with no reproducible source (documented separately in `blog-numbers-from-pipeline-outputs`)
+
+All three were caught before commit, but they illustrate a pattern: small working tree cleanups accumulate friction when not reviewed systematically.
+
+## Guidance
+
+### Dependency classification
+
+When adding a package to `pyproject.toml`, ask: **does this run in production?**
+
+| If... | Put it in... |
+|-------|-------------|
+| The app imports it at runtime | `[project] dependencies` |
+| It is a type checker, test runner, linter, or dev server | `[dependency-groups] dev` |
+| It is only used in build scripts or CI | `[dependency-groups] dev` |
+
+**Concrete check:** search the codebase for `import ` or `from `. If it only appears in `tests/`, `scripts/`, or type stubs, it belongs in `dev`.
+
+### Gitignore hygiene
+
+Before committing a `.gitignore` change, run:
+
+```bash
+sort .gitignore | uniq -d
+```
+
+If anything prints, you have duplicates. Remove them.
+
+Also check that your new entry does not overlap with an existing pattern:
+- `.worktrees/` and `.worktrees` are redundant — keep the slash form for directories
+- `data/*.json` already covers `data/motions.json` — do not add the specific file
+
+### Pre-commit audit checklist
+
+For every set of uncommitted changes:
+
+1. **Dependencies**: Any new packages in the right group?
+2. **Gitignore**: Any duplicates or redundant patterns?
+3. **Blog/docs**: Any hardcoded numbers without canonical sources? (see `blog-numbers-from-pipeline-outputs`)
+4. **Config**: Any secrets or local paths committed by accident?
+
+## Why This Matters
+
+These issues are individually trivial, but together they create a "broken windows" effect. A `pyproject.toml` with dev tools in runtime dependencies signals that the project does not distinguish between production and development concerns. Duplicate `.gitignore` entries suggest the file is append-only and never reviewed. Small hygiene lapses compound into larger maintainability debt.
+
+The fix is cheap: a 30-second scan of the diff before committing prevents all of them.
+
+## When to Apply
+
+- Before every commit that touches `pyproject.toml`, `.gitignore`, or `uv.lock`
+- When onboarding a new dependency
+- During code review of any PR that adds build tools, test frameworks, or local config
+
+## Examples
+
+**Dependency misclassification:**
+
+```toml
+# ❌ Before
+[project]
+dependencies = [
+ "duckdb>=1.3.2",
+ "pyright>=1.1.408", # dev tool in runtime deps
+]
+
+# ✅ After
+[project]
+dependencies = [
+ "duckdb>=1.3.2",
+]
+
+[dependency-groups]
+dev = [
+ "pytest>=9.0.2",
+ "pyright>=1.1.408",
+]
+```
+
+**Gitignore duplicate:**
+
+```diff
+ # Worktrees
+ .worktrees/
+
+ # Generated analysis files
+ thoughts/explorer/*.json
+-
+- # Stray temp files
+- .worktrees # ← duplicate, remove
+```
+
+## Related
+
+- `docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md` — companion guidance on keeping quantitative claims reproducible
+- `docs/solutions/workflow-issues/verify-session-artifacts-against-canonical-sources-2026-04-24.md` — same verification principle applied to session artifacts
diff --git a/docs/solutions/logic-errors/svd-theme-divergence-from-party-positions.md b/docs/solutions/logic-errors/svd-theme-divergence-from-party-positions.md
index 4f323fb..533ad8e 100644
--- a/docs/solutions/logic-errors/svd-theme-divergence-from-party-positions.md
+++ b/docs/solutions/logic-errors/svd-theme-divergence-from-party-positions.md
@@ -17,8 +17,8 @@ SVD axis themes in `analysis/config.py` can drift from actual party positions in
## Symptoms
- Axis 4 theme said "Mainstreampartijen versus FVD/DENK-oppositie" but actual party positions showed NSC (-24.47) and BBB (-4.58) on the left extreme, D66 (10.53)/CDA (10.11)/JA21 (9.90) on the right extreme, and FVD/DENK in the middle
-- Pole labels (`left_pole`/`right_pole`) described parties that weren't actually on those sides after flip
- The flip mechanism (`compute_flip_direction`) worked correctly, but theme text was stale
+- **NOTE (2026-04-12): The `left_pole`/`right_pole` static fields added here caused the same bug — when runtime flip differed from static config flip, labels pointed to wrong sides. These fields were removed. See `docs/solutions/ui-bugs/svd-axis-pole-labels-incorrect-after-flip.md` for the corrected approach.**
## Root Cause
@@ -46,9 +46,9 @@ Updated `analysis/config.py` component 4:
"right_pole": "D66, CDA, JA21 — moties met brede steun",
```
-### 2. Added semantic left_pole/right_pole labels
+### 2. Added semantic left_pole/right_pole labels — SUPERSEDED (2026-04-12)
-Added `left_pole` and `right_pole` fields to all 10 SVD_THEMES entries. These describe what's on the left and right sides AFTER flip, decoupling label text from raw SVD math. Updated 4 rendering locations in `explorer.py` to use these semantic labels with backward compat fallback.
+**This approach caused the same bug.** The static `left_pole`/`right_pole` fields assumed a fixed flip direction, but `compute_flip_direction` determines flip at runtime. When runtime flip differed from static config, labels pointed to wrong sides. These fields were removed. See `docs/solutions/ui-bugs/svd-axis-pole-labels-incorrect-after-flip.md` for the corrected approach.
### 3. Created validation hook
@@ -69,15 +69,15 @@ Returns exit code 1 if any divergence found — suitable for CI integration.
The flip mechanism (`compute_flip_direction`) correctly positions canonical right parties on the right side by comparing mean scores. The validation hook uses the same function with full average vectors to verify post-flip positions. Theme pole labels are now pre-computed semantic descriptions that match the flipped orientation, not raw SVD positive/negative poles.
## Prevention
-
- Run `scripts/validate_svd_themes.py` after any SVD recomputation
- Add to CI pipeline: `uv run python scripts/validate_svd_themes.py --db data/motions.db`
- When updating themes, verify against actual party positions from `svd_vectors`, not just motion sponsors
-- Consider automating theme generation from party positions + motion analysis
+- **NEVER add static `left_pole`/`right_pole` fields** — derive labels at render time from runtime flip (see corrected approach in `svd-axis-pole-labels-incorrect-after-flip.md`)
+- Run `tests/test_svd_axis_alignment.py` to validate alignment after SVD recomputation
## Related Files
-
-- `analysis/config.py` — SVD_THEMES with left_pole/right_pole fields
-- `explorer.py` — rendering functions using semantic pole labels
+- `analysis/config.py` — SVD_THEMES (no `left_pole`/`right_pole`)
+- `explorer.py` — label derivation and component 3-10 scoring
- `analysis/svd_labels.py` — compute_flip_direction() function
- `scripts/validate_svd_themes.py` — validation hook
+- `tests/test_svd_axis_alignment.py` — alignment tests (added 2026-04-12)
diff --git a/docs/solutions/workflow-issues/verify-session-artifacts-against-canonical-sources-2026-04-24.md b/docs/solutions/workflow-issues/verify-session-artifacts-against-canonical-sources-2026-04-24.md
new file mode 100644
index 0000000..9edbead
--- /dev/null
+++ b/docs/solutions/workflow-issues/verify-session-artifacts-against-canonical-sources-2026-04-24.md
@@ -0,0 +1,71 @@
+---
+title: Verify Session Artifacts Against Canonical Sources Before Creating Compounding Docs
+date: 2026-04-24
+category: docs/solutions/workflow-issues
+module: ce-compound
+problem_type: workflow_issue
+component: documentation
+severity: high
+applies_when:
+ - Merging session ledgers into docs/solutions
+ - Creating compounding documentation from transient artifacts
+ - Extracting labels, config values, or data points from session files
+symptoms:
+ - Documentation contains outdated information
+ - Agent creates docs without cross-checking canonical sources
+ - Inaccurate labels propagated to durable documentation
+root_cause: missing_workflow_step
+resolution_type: workflow_improvement
+related_components:
+ - ce-compound
+ - analysis
+tags: [ce-compound, session-ledgers, canonical-sources, verification, documentation-quality, svd-labels]
+---
+
+# Verify Session Artifacts Against Canonical Sources Before Creating Compounding Docs
+
+## Context
+
+During a `ce-compound` ledger-to-docs merge, an agent read an old session ledger (`ses_2b9f`) from `thoughts/ledgers/` and extracted SVD component labels. These labels were written into a new `docs/solutions/` file as authoritative documentation. However, the labels in the ledger were stale — the canonical source (`analysis/config.py` `SVD_THEMES`) had since been updated. The user caught the discrepancy before the doc was committed and flagged it for correction.
+
+Session ledgers are generated at capture time and may become stale as the codebase evolves. They are snapshots, not authorities — using their content directly risks propagating outdated information into durable docs.
+
+## Guidance
+
+When merging session artifacts into compounding documentation:
+
+1. **Identify the canonical source** for every data point extracted from a session file. If the information exists in the codebase (config, database schema, function output), that is the canonical source, not the ledger.
+
+2. **Cross-check all extracted values** against the canonical source before writing. For SVD labels, verify against `analysis/config.py` `SVD_THEMES`. For quantitative claims, run the pipeline function that produces them. For schema details, check the model or migration.
+
+3. **When in doubt, ask the user** which source to use. Do not assume a ledger file is current unless you have confirmed it.
+
+4. **Tag the doc with relevant components** (e.g., `analysis`) so future sweeps can detect drift.
+
+5. **If the canonical source has changed since the ledger was captured**, update the doc to reflect the current state, not the ledger state.
+
+## Why This Matters
+
+Session ledgers are transient artifacts. They capture what was true at a point in time, not what is true now. Treating them as authoritative introduces stale data into the durable documentation layer, which erodes trust and requires expensive corrections later. This is the same class of problem as hardcoding blog numbers from memory — the fix is to route every data point through its canonical source.
+
+Unverified documentation is worse than no documentation because it misleads with apparent authority.
+
+## When to Apply
+
+- When `ce-compound` extracts labels, values, or claims from a session ledger
+- When creating any `docs/solutions/` doc whose content depends on codebase state (config values, function outputs, schema)
+- When a session file references code or config that has been modified since the session was recorded
+
+## Examples
+
+**Actual incident — outdated SVD labels:**
+
+A ledger from an old session contained SVD component labels that described motion patterns. These labels had been revised in `analysis/config.py` (the `SVD_THEMES` dict) as the voting analysis matured.
+
+- ❌ What happened: Agent extracted the labels from the ledger and created `docs/solutions/insights/svd-voting-patterns-by-component-2026-04-04.md` using them
+- ✅ What should have happened: Agent verified each label against `analysis/config.py` `SVD_THEMES`, found that the canonical source had updated values, and used the current values instead (or flagged the discrepancy to the user)
+
+## Related
+
+- `docs/solutions/best-practices/blog-numbers-from-pipeline-outputs-2026-04-16.md` — same principle applied to blog copy: always derive data from canonical pipeline functions, not memory or artifacts
+- `docs/solutions/workflow-issues/trajectories-diagnostic-false-alarm-2026-03-31.md` — another instance of trusting an intermediary artifact (diagnostic JSON) without verifying against the canonical database state
diff --git a/pyproject.toml b/pyproject.toml
index 1cff67b..a13178f 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -24,4 +24,5 @@ dependencies = [
[dependency-groups]
dev = [
"pytest>=9.0.2",
+ "pyright>=1.1.408",
]
diff --git a/reports/drift/axis_stability.png b/reports/drift/axis_stability.png
index 8b091a1..9c080de 100644
Binary files a/reports/drift/axis_stability.png and b/reports/drift/axis_stability.png differ
diff --git a/reports/drift/semantic_drift.png b/reports/drift/semantic_drift.png
index ee8bd03..144d435 100644
Binary files a/reports/drift/semantic_drift.png and b/reports/drift/semantic_drift.png differ
diff --git a/thoughts/ledgers/CONTINUITY_continuity-ledger.md b/thoughts/ledgers/CONTINUITY_continuity-ledger.md
deleted file mode 100644
index c274a36..0000000
--- a/thoughts/ledgers/CONTINUITY_continuity-ledger.md
+++ /dev/null
@@ -1,55 +0,0 @@
-# format: ##|
-# use refs exactly as shown in hashline edit/patch tools
-#HL REV:C4181A89
-#HL 1#AD2#963|# Session: continuity-ledger
-#HL 2#625#EA0|Updated: 2026-03-31T12:00:00Z
-#HL 3#DA3#29F|
-#HL 4#3B8#9B2|## Goal
-#HL 5#49D#054|Preserve the essential session context and state for the stemwijzer project so work can resume seamlessly after context clears.
-#HL 6#DA3#B25|
-#HL 7#3CD#7E4|## Constraints
-#HL 8#343#88A|- Keep the ledger concise; only essential information is recorded.
-#HL 9#C8A#AD0|- Focus on WHAT and WHY, not HOW.
-#HL 10#7DD#B90|- Mark uncertain information explicitly as UNCONFIRMED.
-#HL 11#04E#272|- Include current git branch and key file paths.
-#HL 12#CCD#F02|- Never store secrets or values from .env files.
-#HL 13#DA3#A4D|
-#HL 14#E5A#9FA|## Progress
-#HL 15#E30#F0C|### Done
-#HL 16#829#1C2|- [x] Determine need for a continuity ledger and file location.
-#HL 17#906#394|- [x] Create and add this continuity ledger file to the repository (this file). UNCONFIRMED: whether committed/pushed to remote.
-#HL 18#B2A#001|- [x] Monitor and merge subsequent ledger updates when provided (inspected other CONTINUITY_* ledgers on 2026-03-31T12:00:00Z). (UNCONFIRMED: whether merged/committed)
-#HL 19#DA3#387|
-#HL 20#AC7#256|### In Progress
-#HL 21#405#F17|- [ ] Short QA: sample similarity lookups (N=20-50) to validate fused vectors (see CONTINUITY_stemwijzer.md). Estimated effort: 30–60 minutes. (UNCONFIRMED assignment)
-#HL 22#DA3#77C|
-#HL 23#8B6#828|### Blocked
-#HL 24#2A1#2DC|- None
-#HL 25#DA3#C2F|
-#HL 26#7A9#773|## Key Decisions
-#HL 27#20F#D99|- **Store concise session state in thoughts/ledgers/**: keeps context portable and easy to merge.
-#HL 28#4B6#2BB|- **Minimal fields only (goal, constraints, progress, decisions, next steps, file ops, context)**: reduces noise and maintenance.
-#HL 29#DA3#F5B|
-#HL 30#62A#B91|## Next Steps
-#HL 31#22B#0CD|1. Provide previous ledger content on subsequent updates so merges preserve full history.
-#HL 32#E49#DA8|2. Use this ledger as the single source for resuming interrupted sessions; update "In Progress" items as work proceeds.
-#HL 33#4B7#4A5|3. Coordinate short QA on recent fusion/similarity run (see CONTINUITY_stemwijzer.md) in a separate session if needed.
-#HL 34#DA3#1D0|
-#HL 35#1CA#DCD|## File Operations
-#HL 36#0F3#F62|### Read
-#HL 37#256#5B3|- `README.md`
-#HL 38#A0D#268|- `thoughts/ledgers/CONTINUITY_stemwijzer.md` (INSPECTED)
-#HL 39#AC9#FE0|- `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md` (INSPECTED)
-#HL 40#DA3#081|
-#HL 41#455#EBF|### Modified
-#HL 42#3F4#1DD|- `thoughts/ledgers/CONTINUITY_continuity-ledger.md` (this file)
-#HL 43#DA3#C78|
-#HL 44#2BA#352|## Critical Context
-#HL 45#112#C18|- Repository root: /home/sgeboers/Projects/stemwijzer
-#HL 46#9CD#0EE|- Current git branch: `main` (UNCONFIRMED: local workspace branch)
-#HL 47#DEF#90F|- Other existing continuity ledgers: `CONTINUITY_stemwijzer.md`, `CONTINUITY_fusion_similarity_run.md`
-#HL 48#2D0#620|- UNCONFIRMED: whether this file has been committed/pushed to remote.
-#HL 49#DA3#373|
-#HL 50#7C4#A51|## Working Set
-#HL 51#381#266|- Branch: `main`
-#HL 52#BD8#51B|- Key files: `README.md`, `thoughts/ledgers/CONTINUITY_continuity-ledger.md`, `thoughts/ledgers/CONTINUITY_stemwijzer.md`, `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md`
diff --git a/thoughts/ledgers/CONTINUITY_fusion_similarity_run.md b/thoughts/ledgers/CONTINUITY_fusion_similarity_run.md
deleted file mode 100644
index 0563bee..0000000
--- a/thoughts/ledgers/CONTINUITY_fusion_similarity_run.md
+++ /dev/null
@@ -1,50 +0,0 @@
-# Session: fusion_similarity_run
-Updated: 2026-03-23T16:47:04Z
-
-## Goal
-Record outcomes and metrics from the completed fusion+similarity run so work can resume and a short QA can be executed.
-
-## Constraints
-- Keep summary minimal and machine-readable where detailed counts live in the attached JSON.
-- Do not expose secrets.
-
-## Progress
-### Done
-- [x] Fusion + similarity run completed and core results captured (totals recorded below).
-
-### In Progress
-- [ ] Short QA: sample similarity lookups (recommended)
-
-### Blocked
-- None blocking; QA recommended to validate results and sampling.
-
-## Key Decisions
-- **Pad vectors where necessary**: Several windows had inconsistent vector dimensions; vectors were padded to a common dimension to allow fusion/similarity processing. Rationale: maintain pipeline progress and maximize data retention; warnings were logged for padded windows.
-
-## Next Steps
-1. Run a short QA session: perform sample similarity lookups across N=20-50 items to validate fused vectors and detect anomalies.
-2. Inspect windows flagged in the summary JSON for inconsistent dims and consider source fixes.
-3. If QA passes, promote results to downstream consumers; otherwise, re-run fusion for affected windows after fixing source dims.
-
-## File Operations
-### Read
-- `N/A` (per-window details are in the summary JSON attached below)
-
-### Modified
-- `thoughts/ledgers/fusion_similarity_summary.json`
-- `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md`
-
-- ## Critical Context
-- Start timestamp: 2026-03-23T15:30:00Z
-- End timestamp: 2026-03-23T16:47:04Z
-- Total duration: 1h17m4s (4624 seconds)
-- Totals:
- - embeddings: 28172
- - fused_embeddings: 40524
- - similarity_rows: 405216
-- Per-window inserted counts and any per-window errors are recorded in: `thoughts/ledgers/fusion_similarity_summary.json` (JSON summary attached to repo). This file contains an array of windows with inserted counts and error/warning flags.
-- Note: padding occurred due to inconsistent vector dims in several windows — warnings were logged alongside the affected windows in the JSON summary.
-
-## Working Set
-- Branch: `main`
-- Key files: `thoughts/ledgers/fusion_similarity_summary.json`, `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md`
diff --git a/thoughts/ledgers/CONTINUITY_ses_2a6e.md b/thoughts/ledgers/CONTINUITY_ses_2a6e.md
deleted file mode 100644
index f84ea22..0000000
--- a/thoughts/ledgers/CONTINUITY_ses_2a6e.md
+++ /dev/null
@@ -1,85 +0,0 @@
----
-session: ses_2a6e
-updated: 2026-04-04T15:34:15.344Z
----
-
-# Session Summary
-
-## Goal
-Analyze and document how the most important motions are defined and ranked in the Stemwijzer codebase, focusing on importance criteria, selection mechanisms, metadata, key files, and user interaction patterns.
-
-## Constraints & Preferences
-- Provide detailed findings with file paths and line numbers
-- Focus on code analysis without making changes
-- Document the complete motion ranking and display system
-
-## Progress
-### Done
-- [x] Analyzed motion importance criteria (controversy_score, SVD scores, entropy-based discrimination)
-- [x] Documented motion selection mechanisms for SVD display, Political Compass, quiz, and similarity search
-- [x] Mapped database schema for motions, mp_votes, svd_vectors, similarity_cache tables
-- [x] Identified key files and their roles in motion handling
-- [x] Documented user interaction flows for SVD components tab, MP quiz, and motion browser
-- [x] Cataloged SVD_THEMES dictionary with all 10 component labels and explanations
-
-### In Progress
-- (none - analysis complete)
-
-### Blocked
-- (none)
-
-## Key Decisions
-- **Analysis-only session**: No code modifications were requested or made; this was purely investigative work to understand the existing motion ranking system.
-
-## Next Steps
-1. Awaiting further instructions from user on what to do with the analysis (e.g., implement changes, add features, optimize)
-
-## Critical Context
-### Motion Importance Metrics
-1. **Controversy Score**: `1 - winning_margin` (0.5 = even split, higher = more controversial)
-2. **SVD Component Scores**: Absolute projection on each SVD component axis
-3. **Entropy Score**: Shannon entropy of vote distribution (for quiz discrimination)
-
-### Motion Selection Strategies
-- **SVD Display**: Top 10 per component (5 positive pole, 5 negative pole)
-- **Political Compass**: Top 5 at each pole for axis labeling
-- **Quiz Seed**: Top 8 controversial motions with individual MP votes
-- **Quiz Discriminating**: Entropy-ranked motions that best split candidate MPs
-
-### Database Schema
-```sql
-motions: id, title, description, date, policy_area, voting_results (JSON),
- winning_margin, controversy_score, layman_explanation, body_text, url
-mp_votes: motion_id, mp_name, party, vote, date
-svd_vectors: window_id, entity_type, entity_id, vector (JSON 50-dim)
-similarity_cache: source_motion_id, target_motion_id, score, vector_type, window_id
-```
-
-### Key Functions
-| Function | Location | Purpose |
-|----------|----------|---------|
-| `get_motions_with_individual_votes()` | database.py:660-692 | Get controversial motions with MP votes |
-| `choose_discriminating_motions()` | database.py:817-903 | Entropy-based motion selection |
-| `_top_motion_ids()` | axis_classifier.py:274-295 | Top N motions per axis pole |
-| `build_svd_components_tab()` | explorer.py:3081-3497 | UI for SVD motion display |
-| `build_mp_quiz_tab()` | explorer.py:3499-3724 | MP quiz with adaptive motion selection |
-
-### SVD Themes Location
-`explorer.py:432-762` - Dictionary `SVD_THEMES` contains labels, explanations, and party poles for components 1-10.
-
-## File Operations
-### Read
-- `/home/sgeboers/Projects/stemwijzer/analysis/axis_classifier.py`
-- `/home/sgeboers/Projects/stemwijzer/database.py`
-- `/home/sgeboers/Projects/stemwijzer/explorer.py` (partial reads at offsets 1860, 3050, 3400)
-- `/home/sgeboers/Projects/stemwijzer/pages/1_Stemwijzer.py`
-- `/home/sgeboers/Projects/stemwijzer/scripts/generate_svd_json.py`
-- `/home/sgeboers/Projects/stemwijzer/similarity/lookup.py`
-- `/home/sgeboers/Projects/stemwijzer/src/types/motion_types.py`
-- `/home/sgeboers/Projects/stemwijzer/migrations/2026_03_21__create_mp_metadata.sql`
-- `/home/sgeboers/Projects/stemwijzer/migrations/2026_03_21__create_mp_votes.sql`
-- `/home/sgeboers/Projects/stemwijzer/migrations/2026_03_21__create_svd_vectors.sql`
-- `/home/sgeboers/Projects/stemwijzer/migrations/2026-03-22-add-similarity-cache.sql`
-
-### Modified
-- (none)
diff --git a/thoughts/ledgers/CONTINUITY_ses_2b07.md b/thoughts/ledgers/CONTINUITY_ses_2b07.md
deleted file mode 100644
index 776dc7a..0000000
--- a/thoughts/ledgers/CONTINUITY_ses_2b07.md
+++ /dev/null
@@ -1,56 +0,0 @@
----
-session: ses_2b07
-updated: 2026-04-02T19:01:27.654Z
----
-
-# Session Summary
-
-## Goal
-Execute Task 2 from the SVD Label Unification implementation plan: refactor explorer.py to export SVD_THEMES at module level and update analysis/svd_labels.py to import it properly.
-
-## Constraints & Preferences
-- Follow TDD principles: run tests before/after changes
-- Make minimal changes to accomplish the task
-- Preserve all existing SVD_THEMES data (10 components with labels, explanations, poles, flip settings)
-- Ensure no circular import issues between explorer.py and analysis/svd_labels.py
-
-## Progress
-### Done
-- [x] Ran baseline tests (4 tests passed in tests/test_svd_labels.py)
-- [x] Moved SVD_THEMES dict from inside `build_svd_components_tab` function (line ~2639) to module level in explorer.py (after PARTY_COLOURS, line 434)
-- [x] Removed duplicate SVD_THEMES definition from inside `build_svd_components_tab` function
-- [x] Updated `_get_svd_themes()` function in analysis/svd_labels.py to import directly from explorer module instead of using complex importlib.util fallback
-- [x] Verified all 4 tests still pass after changes
-- [x] Confirmed SVD_THEMES is now accessible at module level in explorer.py for external import
-
-### In Progress
-- [ ] Commit the changes (changes staged but not yet committed)
-
-### Blocked
-- (none)
-
-## Key Decisions
-- **Import method**: Use direct `import explorer` and access `explorer.SVD_THEMES` instead of importlib.util machinery. Rationale: Now that SVD_THEMES is at module level, the direct import is clean and the lazy runtime import in `_get_svd_themes()` prevents circular dependencies at module load time.
-- **Module placement**: Placed SVD_THEMES after PARTY_COLOURS (line 434) to keep constants together near the top of the file. Rationale: This keeps the canonical source of truth visible and maintains logical grouping with other module-level constants.
-
-## Next Steps
-1. Run full test suite to verify no regressions: `uv run pytest tests/ -v`
-2. Commit the changes: `git add explorer.py analysis/svd_labels.py && git commit -m "refactor: move SVD_THEMES to module level for import"`
-3. Proceed to Task 3: Update axis_classifier.py to use svd_labels module
-
-## Critical Context
-- SVD_THEMES now defined at explorer.py line 434 with full type annotation `dict[int, dict[str, str]]`
-- SVD_THEMES contains 10 components (1-indexed) with keys: label, explanation, positive_pole, negative_pole, flip
-- Function `_get_svd_themes()` in analysis/svd_labels.py now uses simple import pattern with global cache `_svd_themes_cache`
-- The function references in explorer.py at lines 2691 and 2719 (`SVD_THEMES.get()`) continue to work unchanged since they now reference the module-level variable
-- All 4 tests in tests/test_svd_labels.py pass, including label retrieval and flip direction computation
-
-## File Operations
-### Read
-- `/home/sgeboers/Projects/stemwijzer/docs/superpowers/plans/2026-04-02-svd-label-unification.md`
-- `/home/sgeboers/Projects/stemwijzer/explorer.py` (lines 1-2000, 2450-2649, 2600-2859, 2810-2859)
-- `/home/sgeboers/Projects/stemwijzer/analysis/svd_labels.py`
-
-### Modified
-- `/home/sgeboers/Projects/stemwijzer/explorer.py`: Added SVD_THEMES at module level (line 434), removed local definition from `build_svd_components_tab()` function
-- `/home/sgeboers/Projects/stemwijzer/analysis/svd_labels.py`: Simplified `_get_svd_themes()` to use direct import from explorer instead of importlib.util fallback
diff --git a/thoughts/ledgers/CONTINUITY_ses_2b4f.md b/thoughts/ledgers/CONTINUITY_ses_2b4f.md
deleted file mode 100644
index e70907b..0000000
--- a/thoughts/ledgers/CONTINUITY_ses_2b4f.md
+++ /dev/null
@@ -1,55 +0,0 @@
----
-session: ses_2b4f
-updated: 2026-04-01T21:57:48.280Z
----
-
-# Session Summary
-
-## Goal
-Analyze how the SVD Components tab in `explorer.py` computes party positions, focusing on: data loading flow, window_size default, X/Y coordinate computation, and whether positions are individual MPs or party centroids.
-
-## Constraints & Preferences
-- Provide exact file:line references for all code paths
-- Trace data flow through multiple files and functions
-- Answer 4 specific questions about the SVD Components tab implementation
-
-## Progress
-### Done
-- [x] Analyzed `build_svd_components_tab()` at `explorer.py:2449-2867`
-- [x] Traced `load_positions()` at `explorer.py:603-656` — default window_size is "quarterly"
-- [x] Traced `load_party_axis_scores()` at `explorer.py:836-853`
-- [x] Traced `_load_mp_vectors_by_party()` at `explorer.py:778-832`
-- [x] Analyzed `compute_2d_axes()` at `analysis/political_axis.py:131-476`
-- [x] Analyzed `compute_party_bootstrap_cis()` at `analysis/political_axis.py:624-695`
-- [x] Analyzed `compute_party_centroids()` at `explorer_helpers.py:246-317`
-- [x] Documented complete data flow with file:line references
-
-### In Progress
-- (none — analysis complete)
-
-### Blocked
-- (none)
-
-## Key Decisions
-- **Window size**: The SVD Components tab uses `"quarterly"` as the default window_size (via `load_positions()` at line 604)
-- **Position type for components 1-2**: Party centroids computed as mean(x), mean(y) from individual MP PCA projections (line 2747)
-- **Position type for components 3-10**: Mean SVD vector per party, with component value extracted by index
-- **Data source**: `svd_vectors` table filtered by `entity_type='mp'` and `window_id='current_parliament'`
-
-## Next Steps
-1. (No pending work — analysis was completed)
-
-## Critical Context
-- **For components 1 and 2**: Party positions come from `load_positions()` which performs PCA on Procrustes-aligned MP SVD vectors, then computes party centroids by averaging x/y coordinates of all MPs in that party
-- **For components 3-10**: Party positions come from `load_party_axis_scores()` which computes mean SVD vector per party from `window='current_parliament'`
-- **Bootstrap CIs**: Computed via `_cached_bootstrap_cis()` at `explorer.py:873-880` using `compute_party_bootstrap_cis()` from `analysis/political_axis.py`
-- **MP→Party mapping**: Via `mp_metadata` table, normalized using `_PARTY_NORMALIZE` dict at `explorer.py:471-480`
-
-## File Operations
-### Read
-- `/home/sgeboers/Projects/stemwijzer/explorer.py` (full file: lines 1-3094)
-- `/home/sgeboers/Projects/stemwijzer/analysis/political_axis.py` (full file: lines 1-695)
-- `/home/sgeboers/Projects/stemwijzer/explorer_helpers.py` (full file: lines 1-317)
-
-### Modified
-- (none)
diff --git a/thoughts/ledgers/CONTINUITY_ses_2b9f.md b/thoughts/ledgers/CONTINUITY_ses_2b9f.md
deleted file mode 100644
index a97f9c4..0000000
--- a/thoughts/ledgers/CONTINUITY_ses_2b9f.md
+++ /dev/null
@@ -1,78 +0,0 @@
----
-session: ses_2b9f
-updated: 2026-04-04T16:29:25.695Z
----
-
-# Session Summary
-
-## Goal
-Improve SVD component axis labels to accurately reflect actual motion content and voting patterns, ensuring the explorer UI and JSON exports are consistent.
-
-## Constraints & Preferences
-- Right-wing parties must appear on RIGHT side of all axes
-- Labels should match what motions actually discuss AND how parties vote
-- Each motion should appear on only one component (exclusive assignment)
-- Report files saved to `thoughts/explorer/` directory
-- Maintain backward compatibility with `--no-exclusive` flag
-
-## Progress
-### Done
-- [x] **Updated SVD_THEMES labels** for Components 1-10 based on deep analysis
-- [x] **Fixed JSON/report mismatch bug** - report was using `scored[:30]` instead of positive/negative separation
-- [x] **Discovered "29 389" issue** - This is Tweede Kamer document identifier in body_text, NOT a motion ID
-- [x] **Identified Component 1 root cause** - Captures coalition vs opposition voting unity, not semantic content
-- [x] **Analyzed voting patterns** across all 10 components using `mp_votes` table
-- [x] **Updated Components 2, 4, 5, 6** based on voting pattern analysis
-- [x] **Regenerated reports** with new labels
-
-### In Progress
-- [ ] Commit the Component 2, 4, 5, 6 label updates
-
-### Blocked
-- (none)
-
-## Key Decisions
-- **SVD captures voting patterns, not semantics**: A component can include defense motions (right votes for) AND social care motions (left votes for) because they're on opposite sides of coalition-opposition divide
-- **Component 1 is coalition-opposition dimension**: 9 coalition parties vs 6 opposition parties voting on opposite sides
-- **Component 3 is TRUE welfare dimension**: PVV votes WITH left (SP, GL-PvdA, PvdD, Volt, DENK) against BBB, CDA, VVD, D66 - cross-block alignment
-- **Component 4 is FVD/DENK isolation**: Only 2 parties vote negatively while 15 vote positively - these parties are outside the mainstream
-
-## Next Steps
-1. **Commit Component 2, 4, 5, 6 label updates**
-2. **Test the explorer** to verify labels render correctly in UI
-3. **Review Component 3** - current label "Verzorgingsstaat vs bezuinigingen" is accurate (cross-block welfare voting)
-4. **Consider Components 7-10** - keep as "(indicatief)" since voting patterns are diverse/unclear
-
-## Critical Context
-
-### Voting Pattern Analysis Results
-
-| Component | Label | Pos Parties | Neg Parties | Interpretation |
-|-----------|-------|------------|------------|----------------|
-| 1 | Rechts kabinetsbeleid vs links oppositiebeleid | 9 coalition+center | 6 opposition | Pure coalition-opposition |
-| 2 | PVV/FVD-populisme vs mainstream | PVV, FVD only | 14 others | Populist isolation |
-| 3 | Verzorgingsstaat vs bezuinigingen | SP, FVD, PVV, GL-PvdA, Volt, DENK, PvdD | BBB, CDA, ChristenUnie, NSC, D66, VVD, SGP, JA21 | TRUE welfare dimension |
-| 4 | Mainstreampartijen vs FVD/DENK | 15 parties | FVD, DENK only | Opposition outsiders |
-| 5 | Christelijk-sociaal vs progressieve individuele rechten | SGP, CDA, ChristenUnie, NSC + others | SP, VVD, GL-PvdA, PvdD, Volt | Christian-democratic values |
-| 6 | Migratie/cultuur vs klimaat/inclusie | PVV, JA21, BBB, CDA, ChristenUnie, VVD, SGP, FVD, DENK | SP, PvdD, D66, GL-PvdA, Volt | Migration/cultural dimension |
-
-### File Operations
-### Read
-- `/home/sgeboers/Projects/stemwijzer/explorer.py` (SVD_THEMES at lines 434-611)
-- `/home/sgeboers/Projects/stemwijzer/scripts/generate_svd_json.py`
-- `/home/sgeboers/Projects/stemwijzer/thoughts/explorer/top_svd_top_motions.json`
-- `/home/sgeboers/Projects/stemwijzer/thoughts/explorer/top_svd_top_motions_report.md`
-
-### Modified
-- `/home/sgeboers/Projects/stemwijzer/explorer.py` - Updated SVD_THEMES labels (Components 1, 2, 4, 5, 6)
-- `/home/sgeboers/Projects/stemwijzer/scripts/generate_svd_json.py` - Fixed positive/negative separation bug
-
-### Created/Regenerated
-- `thoughts/explorer/top_svd_top_motions_report.md` (with updated labels)
-- `thoughts/explorer/top_svd_top_motions.json` (84 rows, 10 components)
-
-### Commits
-- `33edb33` - feat: implement exclusive SVD motion assignment with label review report
-- `e77f0ec` - fix: update SVD_THEMES labels to match actual motion content
-- `bfe37c6` - fix: align report generation with JSON output for positive/negative separation
-- `f7fc908` - fix: update Component 1 label to coalition-opposition reality
diff --git a/thoughts/ledgers/CONTINUITY_ses_2bed.md b/thoughts/ledgers/CONTINUITY_ses_2bed.md
deleted file mode 100644
index 34f33f4..0000000
--- a/thoughts/ledgers/CONTINUITY_ses_2bed.md
+++ /dev/null
@@ -1,72 +0,0 @@
----
-session: ses_2bed
-updated: 2026-03-31T00:07:06.270Z
----
-
-# Session Summary
-
-## Goal
-Generate and update the project's mindmodel, then debug why the trajectories plot is not showing in the Explorer app.
-
-## Constraints & Preferences
-- Keep changes minimal and reversible
-- Diagnostics must be opt-in (EXPLORER_DEBUG_TRAJECTORIES env var)
-- Helpers must be import-safe and pure
-- Use `uv` for local/CI runs (not pip directly)
-- Follow existing project conventions (snake_case, PascalCase for classes, dataclass Config)
-
-## Progress
-### Done
-- [x] **Generated mindmodel** via `mm-constraint-writer` agent → wrote 9 files to `.mindmodel/`:
- - `manifest.yaml`, `stack/stack.yaml`, `architecture/architecture.yaml`, `conventions/conventions.yaml`, `domain/domain-glossary.yaml`, `patterns/patterns.yaml`, `anti-patterns/anti-patterns.yaml`, `dependencies/dependencies.yaml`, `constraints/README.md`
- - Top anti-pattern: `explorer_helpers.py:compute_party_coords` party_map key/value mismatch hypothesis (later invalidated)
-- [x] **Ran 7 parallel analysis agents** covering: stack detection, dependency mapping, convention extraction, domain extraction, code clustering, pattern discovery, anti-pattern detection
-- [x] **Investigated the trajectories "not showing" bug** systematically:
- - Read `explorer.py` (2948 lines), `explorer_helpers.py` (297 lines), `analysis/political_axis.py` (695 lines), `analysis/trajectory.py` (297 lines), `analysis/visualize.py`, `scripts/diagnose_trajectories_cli.py`
- - Ran DB queries confirming: `svd_vectors` has entity_type values `mp` and `motion` only (NO `party` rows), `entity_type='party'` count = 0
- - Ran `diagnose_trajectories_cli.py` — all 4 scenarios produced `party_map_count: 0`
- - **CRITICAL FINDING**: The diagnostic script was artificially passing `load_party_map_ret={}` (empty dict) in ALL scenarios, creating a false alarm
- - Tested with **real data** confirming `party_map` has **1036 entries** (NOT empty)
- - Confirmed party centroids ARE computed correctly: CDA, D66, VVD, PVV, SP, GroenLinks-PvdA etc. all produce traces
- - Annual view (12 windows): CDA, D66, VVD traces visible
- - Quarterly view (33 windows): 6 party traces
- - `select_trajectory_plot_data` returns `trace_count=6` with real data (not 0)
-- [x] **Identified root cause of the diagnostic JSON confusion**: The `2026-03-31-trajectories-diagnostics.json` was created by `diagnose_trajectories_cli.py` which passes `load_party_map_ret={}` artificially, not reflecting real production behavior
-
-## Key Decisions
-- **The trajectories plot DOES work correctly**: The diagnostic JSON showing `party_map_count: 0` was caused by the diagnostic script itself (passing empty party_map), NOT a production bug
-- **No production code changes needed for the core trajectories functionality** — it's working as designed
-- **The diagnostic script `scripts/diagnose_trajectories_cli.py` needs fixing** to use real data paths instead of mocking everything to empty
-- **The anti-pattern detected** (`compute_party_coords` party_map mismatch) was a false alarm — entity_ids in `svd_vectors` are ALL MP names, never party names (no `entity_type='party'` rows exist)
-
-## Next Steps
-1. **Fix `scripts/diagnose_trajectories_cli.py`** to use real data paths (`data/motions.db`) and real `load_party_map` / `load_positions` calls instead of mocking everything to empty
-2. **Re-run the fixed diagnostic script** to produce a correct `trajectories-diagnostics.json` artifact
-3. **Update the mindmodel manifest** to reflect that trajectories work correctly (remove the incorrect anti-pattern about party_map mismatch — it doesn't apply since no party-level entity_ids exist in svd_vectors)
-4. **Consider writing an integration test** that calls `select_trajectory_plot_data` with real DB data and asserts `trace_count > 0` (as the design doc planned but wasn't implemented)
-5. **Decide what to do with `EXPLORER_FORCE_SHOW_TRAJECTORIES=1`** — currently a no-op because party centroids always compute; could be useful for debugging or removed as dead code
-
-## Critical Context
-- **Project type**: Dutch political voting compass (Stemwijzer), Python ≥3.13, Streamlit, DuckDB
-- **DB state**: `mp_metadata` has 798 rows with party info; `svd_vectors` has 73,165 rows with entity_type `mp` (8,219) and `motion` (65,000+), **zero** `entity_type='party'` rows
-- **Window IDs**: 41 windows (annual + quarterly), `get_uniform_dim_windows` returns 33 that pass the dim≥25 AND cnt≥10 filter
-- **`run_app()` hardcodes `window_size = "annual"`** (not quarterly) — so the default view uses 12 windows with 3 default party traces (CDA, D66, VVD)
-- **Mismatch between mp_metadata names and svd_vectors entity_ids**: ~6 MPs in annual view have name variants that don't match party_map (e.g., `De Pater-Postma, W.L.` vs `Pater-Postma de, W.L.`), but this is minor (6 out of 612 = ~1%)
-- **Existing diagnostic JSON** at `thoughts/shared/diagnostics/2026-03-31-trajectories-diagnostics.json` shows `party_map_count: 0` — this is a red herring from the diagnostic script, NOT real production behavior
-
-## File Operations
-
-### Read
-- `/home/sgeboers/Projects/stemwijzer/.mindmodel/manifest.yaml`
-- `/home/sgeboers/Projects/stemwijzer/explorer.py` (2948 lines, lines 1–50, 210–329, 414–443, 486–535, 584–643, 641–720, 1297–1315, 1601–1800, 1800–1919, 1919–1998, 1998–2057, 210–329, 2868–2947)
-- `/home/sgeboers/Projects/stemwijzer/explorer_helpers.py` (full, 297 lines)
-- `/home/sgeboers/Projects/stemwijzer/analysis/political_axis.py` (full, 695 lines)
-- `/home/sgeboers/Projects/stemwijzer/analysis/trajectory.py` (full, 297 lines)
-- `/home/sgeboers/Projects/stemwijzer/analysis/visualize.py` (lines 30–109 for `_load_party_map`)
-- `/home/sgeboers/Projects/stemwijzer/scripts/diagnose_trajectories_cli.py` (full, 118 lines)
-- `/home/sgeboers/Projects/stemwijzer/tests/test_build_trajectories_tab_fallback.py` (full, 61 lines)
-- `/home/sgeboers/Projects/stemwijzer/thoughts/shared/designs/2026-03-31-diagnose-no-plot-trajectories-design.md`
-- `/home/sgeboers/Projects/stemwijzer/thoughts/shared/plans/2026-03-30-diagnose-no-plot-trajectories.md`
-
-### Modified
-- (none yet — verified trajectories work correctly via DB queries and Python tests)
diff --git a/thoughts/ledgers/CONTINUITY_stemwijzer.md b/thoughts/ledgers/CONTINUITY_stemwijzer.md
deleted file mode 100644
index 77d1323..0000000
--- a/thoughts/ledgers/CONTINUITY_stemwijzer.md
+++ /dev/null
@@ -1,131 +0,0 @@
-# Session: stemwijzer
-Updated: 2026-03-31T12:40:00Z
-
-## Goal
-2D political compass + motion similarity search from parliamentary votes + motion text. Full historical coverage 2016–2026, precomputed similarity cache, fused (SVD + text) embeddings.
-
-## Constraints
-- DuckDB only (`data/motions.db`); open/close `duckdb.connect(self.db_path)` per method
-- Vectors stored as JSON text (no external vector DB)
-- Logging via `logging.getLogger(__name__)`; no `print()` in library modules
-- Tests run offline (network monkeypatched) — use `.venv/bin/python -m pytest -q`
-- Do NOT modify `app.py` or `scheduler.py`
-- Use `.venv/bin/python` (Arch Linux system Python is externally managed)
-
-## Current DB State (verified 2026-03-22 ~16:00; additional run summary 2026-03-23)
-
-| Table | Rows |
-|---|---|
-| motions | 10,613 |
-| embeddings | 10,753 |
-| svd_vectors | 24,528 |
-| fused_embeddings | **10,613** (1:1 with motions, 0 duplicates) — per-run fusion summary reported larger aggregate inserts (see Critical Context) (UNCONFIRMED mapping)
-| similarity_cache | **212,206** (top_k=20, all annual windows) — fusion+similarity run produced a larger set of inserted rows (see Critical Context) (UNCONFIRMED mapping)
-| mp_votes | 199,967 |
-| mp_metadata | 798 |
-
-## Annual Window Coverage
-
-| Year | Motions | Fused | Similarity |
-|---|---|---|---|
-| 2016 | 132 | 132 | 2,640 |
-| 2017 | 30 | 30 | 600 |
-| 2018 | 100 | 100 | 2,000 |
-| 2019 | 3 | 3 | 6 |
-| 2020 | 0 | 0 | 0 (no data) |
-| 2021 | 0 | 0 | 0 (no data) |
-| 2022 | 4,116 | 4,116 | 82,320 |
-| 2023 | 621 | 621 | 12,420 |
-| 2024 | 948 | 948 | 18,960 |
-| 2025 | 3,715 | 3,715 | 74,300 |
-| 2026 | 948 | 948 | 18,960 |
-
-## Completed This Session
-- [x] Text embeddings: ran with real OpenRouter API at batch_size=200 → 10,753 embedding rows
-- [x] Re-ran `extract_mp_votes` on all motions → 111,978 new rows (party-level votes backfilled)
-- [x] SVD re-run (annual 2016–2026) with full vote data → 24,528 svd_vector rows
-- [x] Fixed `store_fused_embedding` double-counting bug: added DELETE before INSERT
-- [x] Cleaned and re-ran fusion → 10,613 fused rows, zero duplicates
-- [x] Re-ran similarity cache top_k=20 for all 9 active windows → 212,206 rows
-- [x] Test suite: **34 passed, 2 skipped** ✅
-- [x] Rerun embeddings (scripts/rerun_embeddings.py) completed: embeddings stored = **28,172** (final) — recorded in fusion+similarity run summary (UNCONFIRMED mapping to `embeddings` table)
-- [x] Fusion + similarity run completed (per-window processing) — aggregate inserts recorded in `thoughts/ledgers/fusion_similarity_summary.json`
-
-## Key Decisions
-- `store_fused_embedding` (database.py line 686): Now does DELETE+INSERT instead of plain INSERT to prevent duplicates on re-runs.
-- Annual windows chosen for historical political compass (2016–2026).
-- top_k=20 for similarity cache.
-- Party-level votes (e.g. `{"PVV": "voor"}`) handled in `extract_mp_votes` — actor without comma → `party=actor_name`.
-
-## Open Items (not blocking, data coverage gaps)
-1. **2020–2021 data gap**: No motions in DB at all. Need to run downloader with `--start-date 2019-01-01 --end-date 2021-12-31` if data exists in API.
-2. **2024 gap ~3,020 motions**: OData API has ~3,968 2024 motions, only 948 in DB. Root cause unclear — needs investigation of URL-based dedup in `insert_motion`.
-3. **"Verworpen." dedup**: Short-text motions (title="Verworpen.") get spurious similarity=1.0. UI/query layer should filter `score < 0.999 OR title != 'Verworpen.'`.
-4. **svd_vectors has duplicates**: 2025 has 7,430 rows for 3,715 motions (2x). Doesn't affect fused_embeddings (DELETE+INSERT handles it) but wastes space. Low priority.
-
-## Key File Paths
-- DB: `data/motions.db`
-- Venv: `.venv/bin/python`
-- Pipeline entry: `pipeline/run_pipeline.py`
-- Fusion: `pipeline/fusion.py`
-- SVD: `pipeline/svd_pipeline.py`
-- Text embeddings: `pipeline/text_pipeline.py`
-- MP votes extraction: `pipeline/extract_mp_votes.py`
-- Database layer: `database.py`
-- Similarity compute: `similarity/compute.py`
-- Similarity lookup: `similarity/lookup.py`
-- Tests: `tests/` (pytest, offline)
-
-## Branch
-`main`
-
-## Progress
-### Done
-- [x] All items listed under "Completed This Session" above
-
-### In Progress
-- [ ] Short QA: sample similarity lookups and sanity checks (N=20-50) against `fused_embeddings`/similarity results
- - Purpose: validate fused vectors, detect padding/anomalies, and confirm similarity rows are sensible
- - Estimated effort: 30–60 minutes
-- [ ] Trajectories tab: chart not rendering — root cause found (silent exception in `st.plotly_chart`)
- - Fix applied: commit 72d1c20 — shows st.error + diagnostics when rendering fails
- - Pending: user to verify fix by running Explorer with EXPLORER_DEBUG_TRAJECTORIES=1
-
-### Blocked
-- None blocking for QA; earlier provider failures affected embedding rerun but rerun was completed per fusion run summary (UNCONFIRMED)
-
-## Key Decisions
-- **Retry strategy on provider failure**: On repeated provider failures, retry embedding batches with smaller batch_size (e.g. 50 -> 20) or switch provider. Rationale: smaller batches reduce per-request risk and increase chance of partial success; switching provider if persistent. (UNCONFIRMED)
-
-## Next Steps
-1. Run Short QA: perform sample similarity lookups across N=20-50 items and validate fused vectors
-2. Inspect `thoughts/ledgers/fusion_similarity_summary.json` for windows with padded vectors or warnings; decide whether to re-run fusion for affected windows
-3. If QA passes, promote results to downstream consumers and update DB count fields (mark as confirmed)
-4. If anomalies found, re-run fusion for affected windows and re-compute similarity for those windows
-5. Archive list of any failed motion IDs from embedding run and consider retry with smaller batch_size or alternate provider (if any failures remain) (UNCONFIRMED)
-
-## File Operations
-### Read
-- `data/motions.db`
-- `scripts/rerun_embeddings.py` (invoked)
-- `thoughts/ledgers/fusion_similarity_summary.json` (run summary)
-
-### Modified
-- `thoughts/ledgers/CONTINUITY_stemwijzer.md` (this file)
-- `thoughts/ledgers/fusion_similarity_summary.json` (aggregate per-window results from fusion+similarity run)
-- `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md`
-
-## Critical Context
-- Rerun embeddings started 2026-03-23T01:42Z; final embedding count recorded by fusion run = **28,172** (see `thoughts/ledgers/fusion_similarity_summary.json`) (UNCONFIRMED mapping to `embeddings` table)
-- Fusion + similarity run (2026-03-23T15:30:00Z → 2026-03-23T16:47:04Z) produced aggregate inserts recorded in the summary JSON:
- - embeddings: 28,172
- - fused_embeddings (aggregate inserts across windows): 40,524
- - similarity_rows (aggregate): 405,216
-- Note: the fused_embeddings and similarity_rows totals are aggregate per-window insert counts (may double-count motions appearing in multiple windows) — mapping to unique table counts is UNCONFIRMED.
-- Per-window inserted counts and any per-window errors/warnings are recorded in: `thoughts/ledgers/fusion_similarity_summary.json`.
-- Padding occurred for windows with inconsistent vector dims; warnings logged per-window (see summary JSON). Decision to pad preserved pipeline progress but should be reviewed (see Key Decisions / Next Steps).
-- Earlier provider error: Batch 951..1000 failed with provider error {'error': {'message': 'No successful provider responses.', 'code': 404}} — these batches were retried/covered in the rerun captured by the fusion run (UNCONFIRMED; check failed IDs in summary JSON).
-
-## Working Set
-- Branch: `main`
-- Key files: `data/motions.db`, `scripts/rerun_embeddings.py`, `thoughts/ledgers/CONTINUITY_stemwijzer.md`, `thoughts/ledgers/fusion_similarity_summary.json`, `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md`
diff --git a/thoughts/ledgers/CONTINUITY_svd_axis_consistency_fix.md b/thoughts/ledgers/CONTINUITY_svd_axis_consistency_fix.md
deleted file mode 100644
index a536992..0000000
--- a/thoughts/ledgers/CONTINUITY_svd_axis_consistency_fix.md
+++ /dev/null
@@ -1,56 +0,0 @@
-# Session: svd_axis_consistency_fix
-Updated: 2026-04-13T23:08:19Z
-
-## Goal
-Ensure SVD components tab and compass show consistent party positions by using aligned PCA positions for components 1-2.
-
-## Constraints
-- Right-wing parties (PVV, FVD, JA21, SGP) must appear on RIGHT side of all axes in both visualizations
-- SVD labels should reflect voting patterns, not semantic content
-- Components 1-2 use aligned PCA; Components 3-10 use raw SVD values
-
-## Progress
-### Done
-- [x] Fix SVD axis label alignment (removed static left_pole/right_pole, derive from runtime flip)
-- [x] Fix score mismatch in tijdtraject view (components 3-10 use per-window scores, not Procrustes-aligned)
-- [x] Fix PCA alignment consistency between compass and SVD components tab
-- [x] Update all 10 component labels based on motion analysis
-- [x] Add pool-based motion assignment (10 motions per component)
-- [x] Add SVD axis alignment and label consistency tests
-
-### In Progress
-- (none)
-
-### Blocked
-- (none)
-
-## Key Decisions
-- **Components 1-2 use aligned PCA positions**: Consistent with compass visualization, derived from `load_positions()`
-- **Components 3-10 use raw SVD scores**: Per-window flip handles orientation, Procrustes not needed
-- **New helper `_get_aligned_party_coords()`**: Converts aligned MP positions to party centroids for components 1-2
-
-## Next Steps
-1. Run visual verification to confirm compass and SVD tab show consistent party orderings
-2. Consider adding tests for the new `_get_aligned_party_coords()` helper
-3. Update any documentation that references the old behavior
-
-## File Operations
-### Read
-- `explorer.py` (components tab, load_positions, trajectory rendering)
-- `analysis/political_axis.py` (PCA alignment, compute_party_centroids)
-- `analysis/config.py` (SVD_THEMES)
-- `analysis/svd_labels.py` (label derivation)
-
-### Modified
-- `explorer.py` - Added `_get_aligned_party_coords()`, updated component 1-2 to use aligned positions
-
-## Critical Context
-- **Commit 823df6f**: Removed static left_pole/right_pole, fixed tijdtraject score mismatch
-- **Commit 12936c5**: Use aligned PCA for components 1-2 (consistent with compass)
-- **Commit 036c3f9**: Extended aligned PCA to all SVD components 1-10
-- **Commit 3a67100**: Use aligned PCA scores for time trajectory view
-- **Related docs**: `docs/solutions/ui-bugs/svd-axis-pole-labels-incorrect-after-flip.md`
-
-## Working Set
-- Branch: `main`
-- Key files: `explorer.py`, `analysis/config.py`, `analysis/svd_labels.py`, `tests/test_svd_axis_alignment.py`
diff --git a/thoughts/ledgers/audit_events.json b/thoughts/ledgers/audit_events.json
deleted file mode 100644
index aca542d..0000000
--- a/thoughts/ledgers/audit_events.json
+++ /dev/null
@@ -1,1324 +0,0 @@
-[
- {
- "id": "91d3a66a-5542-4325-8fc0-f0715b570e5b",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:01:06.889693Z"
- },
- {
- "id": "1e1bcee0-5f2a-4337-b57f-ca83ac93da7e",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:01:07.327717Z"
- },
- {
- "id": "51fec578-84ae-4d69-85df-b415a2b6c752",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:01:07.827925Z"
- },
- {
- "id": "38c9ba12-5829-410c-ac38-b992a9b22652",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:01:11.681524Z"
- },
- {
- "id": "1e69741a-d0ff-41c9-b5e7-e4e2e8475836",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:01:12.061577Z"
- },
- {
- "id": "c2fa7d58-958b-4efb-b670-d044de0db357",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:01:12.104491Z"
- },
- {
- "id": "4bd245c7-9e6c-41dc-bce0-f36d3b675ef8",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:03:14.657886Z"
- },
- {
- "id": "e885a2da-48f3-4130-b62f-413ae2670b9c",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:03:14.997464Z"
- },
- {
- "id": "29949f88-0739-4029-964a-d1a9be3d1030",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:03:15.057155Z"
- },
- {
- "id": "b08b870b-1923-4cc7-8384-6a4fd5a5ae63",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:08:18.540282Z"
- },
- {
- "id": "acb1d1ef-1a2f-4256-b23c-dc7272e6cda8",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:08:18.992755Z"
- },
- {
- "id": "a2e7f741-ce46-4533-a4a1-98d202ad5ba9",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:16:44.733143Z"
- },
- {
- "id": "208fd3d6-dcaa-408a-b7ef-054703083756",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:16:45.983301Z"
- },
- {
- "id": "9112851e-ba85-4498-8aa9-4f71aa91d6ec",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:16:46.060827Z"
- },
- {
- "id": "2a5bffe4-c75b-46f2-baf1-164ac87953d6",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:18:02.707023Z"
- },
- {
- "id": "834a7419-6d1b-48e0-825c-08c3ea780c94",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:18:04.158612Z"
- },
- {
- "id": "9d27c575-c186-4cc6-b202-1e7d44600983",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:18:04.213958Z"
- },
- {
- "id": "b03f2ce4-6ac0-41a5-8f32-36dc86db4048",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:19:07.187178Z"
- },
- {
- "id": "4589cfd1-16c9-4743-b2e9-15be42e121e7",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:19:07.560463Z"
- },
- {
- "id": "b2e8fa30-b3ca-42f1-9138-c188b2683723",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:19:07.631447Z"
- },
- {
- "id": "492bd375-b002-446b-b424-4dc3cf40ea44",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:20:12.734568Z"
- },
- {
- "id": "e9349d9b-7962-489b-8a06-1753f1606048",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:20:13.089162Z"
- },
- {
- "id": "a8af51f1-2126-4b57-87e3-6913ada4643b",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:20:13.152945Z"
- },
- {
- "id": "999cccc1-dcbe-48c7-80f0-229351780823",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:20:26.037664Z"
- },
- {
- "id": "03acdcb0-da76-4ac7-bc7d-ea5a314734b8",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:20:27.339457Z"
- },
- {
- "id": "964943f7-c431-484a-a2a3-070327287d90",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:20:27.382680Z"
- },
- {
- "id": "4d4ffec0-83c9-48e6-8724-df04de5cf741",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:25:42.534010Z"
- },
- {
- "id": "56511987-38b1-4286-a8ac-9771c297051f",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:25:42.861164Z"
- },
- {
- "id": "89e29b19-8926-47c8-8a52-922a65da4189",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:25:44.542652Z"
- },
- {
- "id": "a430eb8d-c36e-4ad1-b5c3-01c1fc5f3be5",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:30:11.148013Z"
- },
- {
- "id": "2b108662-6d4b-48b6-88ba-9f3111491217",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:30:12.361529Z"
- },
- {
- "id": "18aad6fc-567f-4d7d-8569-28357c6c301d",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:30:12.429871Z"
- },
- {
- "id": "730467ab-2c1d-47ec-a29e-9d02803c8b1f",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T20:31:09.659557Z"
- },
- {
- "id": "e87170fe-c03b-42a6-bd96-d45a26717359",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T20:31:10.991569Z"
- },
- {
- "id": "cf5c8957-c893-478d-924b-ce77d9e53a41",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T20:31:11.039826Z"
- },
- {
- "id": "d0aa2441-6fc8-4df9-bf37-621f8d9e5351",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T21:37:36.823732Z"
- },
- {
- "id": "e55be18c-0fb5-4576-aa51-24559148916e",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T21:37:38.061523Z"
- },
- {
- "id": "b8f7ee6c-4ce5-4415-97bc-e00e02b2c851",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T21:37:38.105562Z"
- },
- {
- "id": "75e76c14-28e5-4f5b-a2d4-52c9dba5efc4",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T21:53:11.948658Z"
- },
- {
- "id": "0e515c39-ef8a-4abf-87fb-1c96e669e7f5",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T21:53:13.215965Z"
- },
- {
- "id": "836ba937-d26f-4869-9661-36d5744f649a",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T21:53:13.259045Z"
- },
- {
- "id": "95606fed-c023-4fb1-98a4-5c5707e95056",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T21:54:11.170757Z"
- },
- {
- "id": "e188e203-932c-4f79-bed9-af1ec1336102",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T21:54:12.444776Z"
- },
- {
- "id": "591eb30e-a91f-448b-b9a0-9a3d75a35790",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T21:54:12.491990Z"
- },
- {
- "id": "75d70cad-41f4-49f8-ae8a-08b1bafb02b4",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T21:58:08.120685Z"
- },
- {
- "id": "a0cc28c1-28f6-409e-ad5c-8a974ff28e00",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T21:58:08.847188Z"
- },
- {
- "id": "84e5699d-64d9-4179-99e2-f79cb9b27b33",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T21:58:08.888009Z"
- },
- {
- "id": "f0e47b0a-4791-4475-9036-9e87a5e00be8",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T22:31:18.698163Z"
- },
- {
- "id": "50006ac9-d6b5-4198-9cca-c937dee60eab",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T22:31:19.938465Z"
- },
- {
- "id": "d44d09e5-fa20-40ea-8829-dfa044addf57",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T22:31:19.997516Z"
- },
- {
- "id": "c1876f8b-79bd-4fd0-9d0f-68eb8dd3321d",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-23T22:52:46.531868Z"
- },
- {
- "id": "39fb4426-e348-455c-b4f7-0e77c6a72dc7",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-23T22:52:47.772599Z"
- },
- {
- "id": "fc04bbb8-1378-460c-9914-c923cd1a45f8",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-23T22:52:47.836920Z"
- },
- {
- "id": "de3394a0-8c8e-4282-8369-f53aa957fd46",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-24T19:08:06.647810Z"
- },
- {
- "id": "8491ed90-9314-41a9-9d02-092a5d0bebd5",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-24T19:08:08.085618Z"
- },
- {
- "id": "ae7c88e5-ba28-4012-8991-c58fea9c0778",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-24T19:08:08.131631Z"
- },
- {
- "id": "b73e6bf8-2b66-43bf-ad9c-e92d34ae38db",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-24T19:18:02.854710Z"
- },
- {
- "id": "3a6bf0e0-9f07-477d-9079-715d8c0f39c4",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-24T19:18:05.512388Z"
- },
- {
- "id": "75d9e229-78e6-439e-8095-c01ba7830de9",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-24T19:18:05.557773Z"
- },
- {
- "id": "d45fc116-47be-4486-ba5c-ab2edd7f7e76",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-24T19:28:43.867346Z"
- },
- {
- "id": "b4ead1cd-58b1-4ff6-aa73-c77ab09ba063",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-24T19:28:45.051895Z"
- },
- {
- "id": "463bfa1b-59fe-4fd3-a8dd-b39674948656",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-24T19:28:45.097703Z"
- },
- {
- "id": "8efe6c13-e152-48f9-be1b-b5ca3542b316",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-29T21:09:02.760202Z"
- },
- {
- "id": "221e4c30-9d29-457e-82ee-77865aa8f916",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-29T21:09:05.540431Z"
- },
- {
- "id": "f8ebd6fc-c92b-4d57-9ef4-a2f3e8d54943",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-29T21:09:05.589179Z"
- },
- {
- "id": "11a13431-5d46-486d-b46c-c2adc84b6217",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-29T21:44:22.514902Z"
- },
- {
- "id": "909e0f8b-6292-4299-b142-1bd523f7b10b",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-29T21:44:23.768305Z"
- },
- {
- "id": "748b042c-4505-41df-a883-d21fe28540ad",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-29T21:44:23.815343Z"
- },
- {
- "id": "6b7982cb-5404-4b21-b347-ba915d62a0d9",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-29T21:59:12.684721Z"
- },
- {
- "id": "e2c06f76-c88c-4ed7-907a-ccbfb1964940",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-29T21:59:13.959568Z"
- },
- {
- "id": "cf6010f3-e194-464c-af06-92ff879351bc",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-29T21:59:14.005849Z"
- },
- {
- "id": "a75af459-ae06-4e30-b903-83a6f4d6e2ca",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-29T22:13:10.039634Z"
- },
- {
- "id": "855b86dd-21ff-477a-bcd5-4038aa168d72",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-29T22:13:10.684676Z"
- },
- {
- "id": "3062100d-838d-493b-9e5d-24a1a1c2fb5b",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-29T22:13:10.708575Z"
- },
- {
- "id": "81cd3e04-7a80-48fe-b672-4a01c43cc34f",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-30T16:31:17.877408Z"
- },
- {
- "id": "16d3c270-d1d6-4d99-8621-cdc6c250dcc7",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-30T16:31:19.567603Z"
- },
- {
- "id": "90676f40-fe78-48ed-96ac-e9ffbef8ba8e",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-30T16:31:19.631440Z"
- },
- {
- "id": "431e737e-16e1-48f6-83cb-5ba1d027e469",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-30T18:26:00.255862Z"
- },
- {
- "id": "105d745b-dfef-4159-89b6-1a928feefa8f",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-30T18:26:00.601609Z"
- },
- {
- "id": "70b74aa7-03c9-4144-8a51-428ff79a4ca7",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-30T18:26:00.642695Z"
- },
- {
- "id": "ecef4c40-bdbb-44f4-a8ec-a027d1634933",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-30T18:31:04.409375Z"
- },
- {
- "id": "1331e65d-863a-4c5f-a036-5e0f59694788",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-30T18:31:04.812686Z"
- },
- {
- "id": "206b7610-7b47-4b23-8ef8-f2042429d036",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-30T18:31:04.851848Z"
- },
- {
- "id": "5e9b74b8-0377-4497-99e1-25aec5e55082",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-30T18:32:23.601326Z"
- },
- {
- "id": "74ec8b6c-7fab-426c-aa48-3fabf016c7e9",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-30T18:32:24.029561Z"
- },
- {
- "id": "a119058b-a4b1-42aa-8921-201e24e0808e",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-30T18:32:24.079920Z"
- },
- {
- "id": "f3d6b7a2-3d3c-41f6-872e-49fd635f0d41",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-03-30T18:52:42.235576Z"
- },
- {
- "id": "18620b56-3082-4f42-bda4-a3eb4de6b611",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-03-30T18:52:43.583303Z"
- },
- {
- "id": "738a557b-0952-45b4-b86c-fda53fae2aa1",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-03-30T18:52:43.630069Z"
- },
- {
- "id": "35963a0d-c63f-4afe-95f6-5b5208e57d29",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-01T23:46:34.584639Z"
- },
- {
- "id": "e68dfe7f-e0d9-4384-914d-4007f89b8e29",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-01T23:46:37.561361Z"
- },
- {
- "id": "e0a791fd-108d-49eb-bb09-8d0428e52e69",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-01T23:46:37.604922Z"
- },
- {
- "id": "f276f1ad-dc32-46a3-a74d-aa6e8a372fca",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-01T23:55:13.967080Z"
- },
- {
- "id": "7dc7b020-6c39-43c3-8aa3-174d344f91a7",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-01T23:55:14.671187Z"
- },
- {
- "id": "ac7e74cc-3b49-48dd-b9f0-d112df40d5ef",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-01T23:55:14.699780Z"
- },
- {
- "id": "fa59ab8a-2bb7-4b71-a48c-138278966dfb",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:01:42.827310Z"
- },
- {
- "id": "009b293a-fec3-49de-8492-3af93d597af0",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:01:44.222780Z"
- },
- {
- "id": "2c31b5a9-5332-4bec-abc7-bce8a8c1f91a",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:01:44.267873Z"
- },
- {
- "id": "fcb47926-785a-49a8-b492-6ed0f837f91e",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:02:21.461896Z"
- },
- {
- "id": "c20946d2-153e-44c7-9b87-a75974c05b8f",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:02:22.721758Z"
- },
- {
- "id": "6f943c69-e340-40c9-8df5-9a69dc0176d4",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:02:22.781638Z"
- },
- {
- "id": "0b0c6aa3-777f-4d13-b445-0334ca79abbc",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:05:29.897839Z"
- },
- {
- "id": "3fb54286-4495-4b42-8819-92fb05036d9b",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:05:31.206251Z"
- },
- {
- "id": "93ebb7a9-4734-43a7-8e32-7e4531e8393f",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:05:31.267085Z"
- },
- {
- "id": "d2a9f12e-9dfd-4879-b50f-1fc7db6f78f6",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:05:45.130369Z"
- },
- {
- "id": "5d5d756e-85aa-485e-8961-f2f6d495ebd6",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:05:46.352702Z"
- },
- {
- "id": "c4c61d53-22b5-4669-9ff3-457c0c0d0367",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:05:46.424142Z"
- },
- {
- "id": "b2599219-407b-4ea8-865c-41f61119b76b",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:07:57.967977Z"
- },
- {
- "id": "553c5da0-3fd6-4751-8990-5b2598f5e356",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:07:59.154098Z"
- },
- {
- "id": "73dd2e11-4ec5-4aa2-9434-1511988ee2ae",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:07:59.196193Z"
- },
- {
- "id": "e3bab538-e5e4-49be-9cb9-80a2fca75658",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:08:15.084851Z"
- },
- {
- "id": "2688a143-6744-41b2-975f-c55fd7594df5",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:08:16.326930Z"
- },
- {
- "id": "b4249645-67ef-4902-908b-f4e4aa17a7a2",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:08:16.371816Z"
- },
- {
- "id": "c4d307fe-8363-4de7-86be-65b28b44105c",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:14:50.773665Z"
- },
- {
- "id": "aaa28b70-3af9-4b0b-bb5a-203367206a9f",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:14:52.080492Z"
- },
- {
- "id": "19dbbc73-2954-4045-a0d3-21189380e097",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:14:52.152262Z"
- },
- {
- "id": "025319c5-5fc0-46e3-8230-216a70b40ac6",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:22:21.697203Z"
- },
- {
- "id": "0a2623fb-66be-4b62-8a1c-33c7f248bbd9",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:22:22.950861Z"
- },
- {
- "id": "7efd721a-ca00-48c4-95e5-d45a609404f4",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:22:22.994655Z"
- },
- {
- "id": "d9506033-2fc0-4317-8dd8-a388655e087b",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:23:15.774659Z"
- },
- {
- "id": "9fda605d-8c05-45b1-a9d6-b27c0cccbc7c",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:23:16.993127Z"
- },
- {
- "id": "b0d9da81-4528-4d79-97ef-107ec4f207dc",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:23:17.038391Z"
- },
- {
- "id": "4378866d-023b-4bfb-a1d1-c012bb7f2f09",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:24:31.074547Z"
- },
- {
- "id": "b0b5c909-305d-4947-af1e-a3495aba0c39",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:24:32.344948Z"
- },
- {
- "id": "c408f4c6-c188-4d20-8f06-ae293efce45e",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:24:32.397774Z"
- },
- {
- "id": "e4f83fc7-12f1-4b08-8f65-48dc4335f2dd",
- "actor_id": null,
- "action": "embedding_failed",
- "target_type": "motion",
- "target_id": "99",
- "metadata": {
- "error": "RuntimeError(\"Simulated embedding failure for index 0: 'failing motion'\")"
- },
- "created_at": "2026-04-02T19:25:52.766542Z"
- },
- {
- "id": "3705c35f-f94f-4333-9c09-318e57ec6bc3",
- "actor_id": null,
- "action": "test_action",
- "target_type": "unit",
- "target_id": "u1",
- "metadata": {
- "k": 1
- },
- "created_at": "2026-04-02T19:25:54.130128Z"
- },
- {
- "id": "c83ce165-d7fa-437a-a99d-ba5ef50083ab",
- "actor_id": null,
- "action": "another_action",
- "target_type": "motion",
- "target_id": null,
- "metadata": {},
- "created_at": "2026-04-02T19:25:54.178825Z"
- }
-]
\ No newline at end of file
diff --git a/thoughts/ledgers/fusion_similarity_summary.json b/thoughts/ledgers/fusion_similarity_summary.json
deleted file mode 100644
index 7a7658b..0000000
--- a/thoughts/ledgers/fusion_similarity_summary.json
+++ /dev/null
@@ -1,22 +0,0 @@
-{
- "session": "fusion_similarity_run",
- "start_timestamp": "2026-03-23T15:30:00Z",
- "end_timestamp": "2026-03-23T16:47:04Z",
- "duration_seconds": 4624,
- "totals": {
- "embeddings": 28172,
- "fused_embeddings": 40524,
- "similarity_rows": 405216
- },
- "windows": [
- {"window_id": "win-001", "inserted": 1024, "errors": 0, "warnings": 0},
- {"window_id": "win-002", "inserted": 2048, "errors": 0, "warnings": 1, "warning_message": "padded vectors due to dim mismatch"},
- {"window_id": "win-003", "inserted": 4096, "errors": 0, "warnings": 2, "warning_message": "padded vectors due to dim mismatch"},
- {"window_id": "win-004", "inserted": 8192, "errors": 0, "warnings": 0},
- {"window_id": "win-005", "inserted": 15344, "errors": 0, "warnings": 3, "warning_message": "padded vectors due to dim mismatch"}
- ],
- "notes": [
- "Padding occurred for several windows where vector dimensions were inconsistent. Warnings logged per-window.",
- "Recommend short QA: sample similarity lookups (20-50 items) to validate fused vectors."
- ]
-}
diff --git a/thoughts/ledgers/qa_similarity_20260323T194335Z.json b/thoughts/ledgers/qa_similarity_20260323T194335Z.json
deleted file mode 100644
index d91c8fb..0000000
--- a/thoughts/ledgers/qa_similarity_20260323T194335Z.json
+++ /dev/null
@@ -1,29 +0,0 @@
-{
- "timestamp": "2026-03-23T19:43:35.098568Z",
- "sample_size": 2,
- "top_k": 3,
- "results": [
- {
- "motion_id": 1,
- "top_k": 3,
- "suspicious": 1
- },
- {
- "motion_id": 2,
- "top_k": 3,
- "suspicious": 1
- }
- ],
- "motions": {
- "1": {
- "motion_id": 1,
- "top_k": 3,
- "suspicious": 1
- },
- "2": {
- "motion_id": 2,
- "top_k": 3,
- "suspicious": 1
- }
- }
-}
\ No newline at end of file
diff --git a/thoughts/shared/designs/2026-03-19-stemwijzer-design.md b/thoughts/shared/designs/2026-03-19-stemwijzer-design.md
deleted file mode 100644
index 6873f67..0000000
--- a/thoughts/shared/designs/2026-03-19-stemwijzer-design.md
+++ /dev/null
@@ -1,98 +0,0 @@
----
-date: 2026-03-19
-topic: "Stemwijzer AI & DB design"
-status: draft
----
-
-## Problem Statement
-
-We need a clear, low-risk design to improve AI usage and query ergonomics in this repository. The codebase currently ingests motions, stores them in DuckDB, and generates AI-driven layman summaries via an OpenRouter/OpenAI client. There are a few maintenance issues (e.g., missing config keys, a broken reset script) and no embedding/search infrastructure.
-
-**Goal:**
-- Centralize AI/LLM usage behind a provider abstraction so we can swap or prefer providers later.
-- Introduce minimal embeddings storage and search so we can add semantic features without heavy infra.
-- Prefer ibis for read/query paths where that improves clarity and maintainability (the repo already imports ibis in read.py).
-
-
-## Constraints
-
-- Work must be incremental and non-disruptive: keep existing DuckDB schema and write paths where possible.
-- Do not add external services (vector DB) in the first iteration — store embeddings in DuckDB as JSON for now.
-- Secrets must remain environment-driven (no checked-in secrets). Add env var defaults only.
-- Keep changes small and well-tested; make it easy to roll back.
-
-
-## Approach (chosen)
-
-I'll introduce two small layers:
-- **ai_provider**: a thin adapter that exposes get_embedding(text) and chat_completion(messages). It will use the existing OpenRouter/OpenAI path by default and can be extended to prefer other providers if/when desired. Prefer QWEN via OpenRouter and the OPENROUTER_API_KEY environment variable, falling back to OPENAI_API_KEY where appropriate.
-- **query_dal**: read-focused utilities implemented with ibis to replace direct SQL reads in the app and other read-heavy paths. Writes (insert_motion, update_user_vote) stay in database.py initially.
-
-This gives the benefits of abstraction and pythonic query composition while keeping risk low.
-
-
-## Architecture
-
-High level components (repo root):
-- api_client.py — fetches motion data from Tweede Kamer OData (unchanged)
-- scraper.py — optional HTML scraping fallback (unchanged)
-- database.py — current writes, schema initialization (add small embeddings table)
-- summarizer.py — generate layman summaries (refactor to use ai_provider)
-- app.py — Streamlit UI (switch read paths to query_dal)
-- scheduler.py — orchestrates ingestion and triggers summarization (unchanged)
-
-Additions:
-- ai_provider.py — single place for LLM/embedding calls and retries
-- query_dal.py — ibis-based read helpers (get_filtered_motions, calculate_party_matches)
-- minimal embeddings table in DuckDB (motion_id, model, vector JSON, created_at)
-
-
-## Components and responsibilities
-
-- **ai_provider**: choose provider, handle retries/backoff, return plain Python objects (list[float] embeddings, str completions). Keep error classes small and testable.
-- **database (existing)**: add store_embedding and search_similar helpers (naive in-Python cosine scan). Keep insert_motion/update_user_vote unchanged to minimize risk.
-- **query_dal**: use ibis for read queries used by Streamlit paths (get_filtered_motions, session lookups). Return parsed JSON fields.
-- **summarizer**: call ai_provider.chat_completion to get summary; update motions.layman_explanation; optionally compute embedding via ai_provider.get_embedding and store via database.store_embedding.
-- **app.py**: replace direct duckdb selects with query_dal functions.
-
-
-## Data Flow
-
-1. Ingest: scheduler / scraper / api_client fetch motions and call database.insert_motion(motion).
-2. Summarize: summarizer calls ai_provider.chat_completion(summary prompt) → writes layman_explanation to motions table. Optionally computes embedding and writes to embeddings table.
-3. Query: Streamlit app calls query_dal.get_filtered_motions (ibis) to load motions for sessions and query_dal.calculate_party_matches for results.
-4. Semantic search (future): query_dal or app can call database.search_similar by providing an embedding computed with ai_provider.get_embedding.
-
-
-## Error Handling
-
-- ai_provider: retries with exponential backoff for transient errors; raises a ProviderError for terminal failures so callers can decide retry semantics.
-- Summarizer: non-fatal on AI failures — store an empty/fallback summary and log the failure; surface a user-facing message in Streamlit if generating summaries fails interactively.
-- DB functions: existing try/except patterns retained; ensure connections are closed on error.
-
-
-## Testing Strategy
-
-- Unit tests for ai_provider using mocks for HTTP/openai responses.
-- DB tests using temporary DuckDB files to verify store_embedding and search_similar behavior.
-- query_dal tests using ibis against a temporary DB file; ensure JSON fields parse correctly.
-- Summarizer tests mock ai_provider to assert DB writes happen.
-
-
-## Open Questions
-
-- Store embeddings inside motions table vs separate embeddings table? Recommendation: separate embeddings table for clarity and easier upserts.
-- Do we want to prefer other providers (Copilot) automatically? This repo currently references OPENROUTER. If user wants Copilot preference, we can add env vars and selection logic later.
-
-
-## Next steps (short)
-
-1. Add ai_provider.py (adapter) and tests.
-2. Add embeddings table and store/search helpers in database.py and tests.
-3. Add query_dal.py with ibis reads and tests.
-4. Refactor summarizer.py to use ai_provider and optionally store embeddings.
-5. Update Streamlit app read paths to use query_dal.
-6. Fix housekeeping bugs: reset.py references reset_database(), scraper uses undefined SCRAPING_DELAY — address these small fixes in a separate patch.
-
-
-I'm proceeding to save this design to thoughts/shared/designs/2026-03-19-stemwijzer-design.md and will spawn the planner to create a detailed implementation plan. Interrupt if you want changes to the design text above.
diff --git a/thoughts/shared/designs/2026-03-21-motions-guided-explorer-design.md b/thoughts/shared/designs/2026-03-21-motions-guided-explorer-design.md
deleted file mode 100644
index e0b2d48..0000000
--- a/thoughts/shared/designs/2026-03-21-motions-guided-explorer-design.md
+++ /dev/null
@@ -1,116 +0,0 @@
----
-date: 2026-03-21
-topic: "Reuse motions as a guided policy explorer"
-status: draft
----
-
-## Problem Statement
-
-We want to repurpose existing "motions" data so it becomes a lightweight, discovery-driven way for users to explore policy positions and discover related content. This is not a full proposal system; it's a guided exploration and bookmarking flow that leverages our existing ingestion, summarization, embeddings, and session voting work.
-
-**Why now:** We already ingest motions, generate layman explanations, compute embeddings, and store per-session votes. Reusing those building blocks gives high user value with modest effort.
-
-## Constraints
-
-**Non-negotiables and technical limits:**
-- Use the existing database schema where possible (motions table, embeddings table, user_sessions). Do not require a new external vector DB for MVP.
-- Keep the Streamlit UI model (app.py) and session-based votes intact for the initial rollout.
-- Avoid breaking migrations: rely on existing migrations and add new ones when necessary (no forced drops).
-- Respect current error-handling posture: network calls can fail; system must degrade gracefully.
-
-## Chosen Approach
-
-I'm choosing a "Guided Policy Explorer" approach because it reuses thehighest-value existing pieces (summaries, embeddings, session voting) and delivers a clear UX that fits the current codebase. This gives immediate product value with low risk.
-
-**Core idea:** present curated short sessions and motion detail pages that combine the existing layman explanation, party-match results, and semantic "related motions" powered by stored embeddings.
-
-Alternatives considered:
-- "Motion-as-Proposal platform": full lifecycle (draft → comment → vote). Rejected for MVP due to high complexity and data model changes.
-- "Motion Digest / Research Assistant": read-only pages and newsletters. Lower effort, but less interactive and reuses fewer of our current session features.
-
-## Architecture
-
-High-level view (existing pieces in bold):
-- Ingest: **api_client.py** + **scraper.py** gather motions and create motion records in the DB.
-- Persist: **database.py** stores motions, embeddings, and user_sessions.
-- Enrichment: **summarizer.py** + **ai_provider.py** generate layman explanations and embeddings.
-- Background jobs: **scheduler.py** runs ingest, summarization, and periodic clustering.
-- UI: **app.py** current Streamlit session flow — extend with "Explore" and "Motion detail" pages.
-- New: small **clusterer / similarity API** to compute and cache related-motion lists per motion.
-
-## Key Components & Responsibilities
-
-- Motion Ingest (existing): keep ingest as-is; add metadata flags (e.g., curated, candidate).
-- Motion Store (existing): motions table + embeddings table; add an **events/audit** table for user actions and important state transitions.
-- Summarizer / Embedding Worker (existing): scheduled job that ensures motions have layman_explanation and embeddings; add retry/backoff and logging.
-- Similarity service (new): computes nearest neighbors using stored vectors in-process for MVP and caches results in a small table. Swap to a vector index later if needed.
-- Session & Voting (existing): continue using user_sessions JSON blob for individual sessions; add optional event log entries for each vote.
-- UI (update): add "Explore" landing, motion detail view with layman text, party-match snapshot, related motions, and bookmark/flag actions. Reuse Streamlit components.
-- Admin tooling (new): migration scripts, a CLI to recompute embeddings/similarity, and an audit query helper.
-
-## Data Flow
-
-1. Ingest job (api_client/scraper) produces motion records and calls db.insert_motion.
-2. Summarizer worker picks up motions without layman_explanation or embeddings, calls ai_provider, and writes layman_explanation + embeddings.
-3. Clusterer/similarity job computes related-motion lists using stored embeddings and writes them to a cache table.
-4. UI "Explore" shows curated motion lists; "Motion detail" reads motion, layman_explanation, party-match snapshot, and cached related motions.
-5. User vote actions update user_sessions and also append an event to the audit table for traceability.
-6. Background analytics (optional) reuses user_events and embeddings for offline insights.
-
-## Error Handling Strategy
-
-- External calls: add retries with exponential backoff for AI provider and external APIs. Failures set a marker (e.g., summary_missing) and the system continues.
-- Missing embeddings: UI gracefully disables "related motions" and offers "compute on demand".
-- Idempotency: make insert_motion idempotent by URL/external id check at DB layer; use optimistic handling for duplicates.
-- Concurrency: avoid read-modify-write races by writing user events (append-only) and deriving session state from events when race-prone updates are detected.
-- Observability: replace prints with structured logging (module-level logger) and add basic metrics for worker errors, API failures, and queue lags.
-
-## Testing Strategy
-
-- Unit tests: DB helpers (insert_motion, store_embedding, similarity cache), summarizer functions (mock ai_provider), and session vote logic.
-- Migration tests: follow the existing pattern of applying migration SQL in a temp DB and asserting schema.
-- Integration tests: end-to-end ingest → summarize → embedding → similarity → UI-read path in CI (use monkeypatch for AI calls).
-- Load tests: simulate a few thousand embeddings search calls against the in-process search to validate performance assumptions for MVP.
-- Acceptance: confirm UX flows: Explore session, Motion detail, Vote -> party match, Related motions populated.
-
-## High-level Plan & Estimates
-
-Assumptions: one full-stack engineer (Python + Streamlit) and one part-time reviewer. All estimates are rough.
-
-Milestone 0 — Validate & quick discovery (1 day)
-- Locate user's added markdown plan and extract exact requirements. (I'm assuming the file exists in thoughts/shared; if not, we validated by searching.)
-
-Milestone 1 — MVP (8–12 engineer days)
-- Add similarity cache table and migration.
-- Summarizer: make embedding generation robust with retries and store vectors.
-- Clusterer job: compute and cache related motions.
-- UI: Explore landing, Motion detail page, related motion UI, bookmark/flag button.
-- Add event/audit table and write events on user votes and bookmarks.
-
-Milestone 2 — Hardening & instrumentation (3–5 engineer days)
-- Replace prints with structured logging across touched modules.
-- Add migration tests and CI integration tests (mock AI).
-- Add health metrics & basic alerting for worker failures.
-
-Milestone 3 — Polish & UX feedback (3–5 engineer days)
-- UX tweaks, performance tuning, compute on-demand fallback for embeddings, documentation, admin CLI.
-
-Total MVP + polish: ~2–3 weeks of focused work.
-
-## Risks & Mitigations
-
-- Risk: Naive in-process embedding search will not scale. Mitigation: cache nearest neighbors per motion and plan a migration path to a vector index.
-- Risk: AI provider flakiness. Mitigation: retries, timeouts, and clear UI fallback. Tests must mock provider in CI.
-- Risk: Race conditions on session votes. Mitigation: append-only event log and derive authoritative session view from events when needed.
-- Risk: Schema drift and missing migrations. Mitigation: add migration tests and document required migrations in repo.
-
-## Open Questions
-
-- Which exact user journeys do we want first (single-session discover vs. persistent account/bookmarking)?
-- Do we want bookmarks persisted globally or per-session only? (Privacy implications.)
-- What's acceptable latency for "related motions" — precomputed nightly vs. near-real-time?
-- Any policy/legal ban on storing full body_text or on long-term retention of user votes?
-
----
-
-I'm proceeding to create the design doc file at thoughts/shared/designs/2026-03-21-motions-guided-explorer-design.md and will spawn the implementation planner next. Interrupt if you want changes to the approach or scope now.
diff --git a/thoughts/shared/designs/2026-03-21-parliamentary-embedding-pipeline-design.md b/thoughts/shared/designs/2026-03-21-parliamentary-embedding-pipeline-design.md
deleted file mode 100644
index 8230cff..0000000
--- a/thoughts/shared/designs/2026-03-21-parliamentary-embedding-pipeline-design.md
+++ /dev/null
@@ -1,184 +0,0 @@
----
-date: 2026-03-21
-topic: "Parliamentary Embedding Pipeline (Late Fusion)"
-status: validated
----
-
-## Problem Statement
-
-We want to implement the late-fusion embedding system described in EMBEDDING_ANALYSIS.md: track how MPs shift politically over time and map motions onto a meaningful ideological axis. The primary blocker is data structure — individual MP votes already arrive from the OData API and are stored inside `motions.voting_results` as a mixed JSON blob (party names + MP names together). We need to extract these into a proper relational structure before the SVD pipeline can be built.
-
-**Why this is the right next step:** We already have motion text, layman explanations, text embeddings infrastructure (Qwen3 via ai_provider), and DuckDB. The missing pieces are (1) first-class MP vote rows, (2) MP metadata (party affiliation, tenure dates), and (3) the SVD + Procrustes + fusion compute pipeline.
-
-## Constraints
-
-- **DuckDB only** — no pgvector, no external vector store. In-Python compute (scipy) is correct.
-- **voting_results already has MP names** — extraction is a parsing pass over existing data, not a new API call. Individual MP names are identified by the presence of a comma in the key (already handled in `calculate_party_matches`, `database.py:264`).
-- **Existing embeddings table is keyed to motion_id** — we must not break the current schema. SVD and fused vectors go into new tables.
-- **ai_provider.get_embedding already works** — use it as-is for text embeddings; no model changes needed for MVP.
-- **ibis/DuckDB preferred** over raw SQL for analysis queries (per project preferences).
-- **uv** for dependency management; add `scipy`, `umap-learn`, `plotly`, `sentence-transformers` (or use existing ai_provider for embeddings).
-
-## Approach
-
-**Late-fusion pipeline in four phases:**
-
-1. **Extract** — parse MP-level votes out of `voting_results` JSON into an `mp_votes` table; fetch MP metadata from OData into `mp_metadata`.
-2. **Compute SVD** — per time window, build sparse MP × motion matrix → SVD → Procrustes-align windows sequentially.
-3. **Text embeddings** — ensure every motion has a text embedding (existing path; just fill gaps).
-4. **Fuse** — concatenate aligned SVD motion vector + text embedding → store in `fused_embeddings` table.
-
-Alternatives considered:
-- **Pure text embeddings only**: easier but loses the behavioral (voting) signal entirely. Rejected because the whole point of the plan is the fused representation.
-- **Store aligned SVD + rotation matrices separately**: more flexible for recomputing, but adds complexity. MVP will store aligned vectors directly; rotation matrices are logged for debugging but not persisted.
-
-## Architecture
-
-```
-Data layer (DB):
- motions (existing)
- embeddings (existing — text vectors keyed to motion_id)
- mp_votes (NEW — one row per MP per motion)
- mp_metadata (NEW — MP name, party, entry/exit dates)
- svd_vectors (NEW — per window, per entity: MP or motion)
- fused_embeddings (NEW — per motion, per window: SVD + text concatenated)
-
-Pipeline modules (new, in pipeline/):
- extract_mp_votes.py — JSON blob → mp_votes rows
- fetch_mp_metadata.py — OData /Kamerlid → mp_metadata rows
- svd_pipeline.py — time windows → SVD → Procrustes alignment → svd_vectors
- text_pipeline.py — ensure embeddings coverage, delegates to existing summarizer
- fusion.py — join svd_vectors + embeddings → fused_embeddings
-
-Analysis modules (new, in analysis/):
- political_axis.py — first SVD component / anchor-party axis
- trajectory.py — MP drift across aligned windows
- clustering.py — UMAP on fused motion embeddings, thematic clusters
- visualize.py — Plotly interactive trajectory and cluster plots
-
-CLI entry points (new):
- pipeline/run_pipeline.py — orchestrate all phases with flags
-```
-
-## Key Components & Responsibilities
-
-**mp_votes table**
-- Schema: `(id, motion_id, mp_name, party, vote ENUM(voor/tegen/afwezig), date, created_at)`
-- Populated by `extract_mp_votes.py` doing a one-time parse of `motions.voting_results` JSON.
-- Idempotent: skip motion_id if already extracted (upsert or EXISTS check).
-- `party` field is left NULL initially; backfilled from `mp_metadata` after that table is populated.
-
-**mp_metadata table**
-- Schema: `(mp_name, party, entry_date, exit_date, source_id)`
-- Fetched from OData `/Kamerlid` endpoint (needs verification — see Open Questions).
-- Fallback: derive approximate party affiliation from `mp_votes` rows (majority-party heuristic) if OData metadata is unavailable.
-
-**svd_vectors table**
-- Schema: `(window_id, entity_type ENUM(mp/motion), entity_id, vector JSON, model TEXT, created_at)`
-- Stores both MP and motion SVD vectors per time window, after Procrustes alignment.
-- `window_id` is a string like `2024-Q1`.
-
-**fused_embeddings table**
-- Schema: `(motion_id, window_id, vector JSON, svd_dims INT, text_dims INT, created_at)`
-- Separate from the existing `embeddings` table to avoid schema conflicts.
-- Vector is the concatenation of the SVD motion vector and the text embedding.
-
-**svd_pipeline.py**
-- Groups motions by time window (quarterly default).
-- Builds a sparse `scipy.sparse.csr_matrix` (MPs as rows, motions as columns, vote values encoded as +1/−1/0).
-- Calls `scipy.sparse.linalg.svds(matrix, k=dims)` — `k` is configurable (default 50).
-- Applies Procrustes alignment between consecutive windows using overlapping MPs as anchors.
-- Logs Procrustes disparity score per transition; flags high disparity (election transitions).
-
-**extract_mp_votes.py**
-- Reads all motions with `voting_results` JSON, parses keys: if comma in key → individual MP name, else → party/fraction name.
-- Writes MP-level rows to `mp_votes`; party-level rows are ignored here (they're already used by the existing `calculate_party_matches` flow).
-- Handles the three vote values: `voor` (+1), `tegen` (−1), `afwezig` (0).
-
-**fusion.py**
-- For each motion in a window: lookup SVD motion vector from `svd_vectors`; lookup text embedding from `embeddings`.
-- Concatenates vectors (simple `list + list`); stores in `fused_embeddings`.
-- Skips motion if either vector is missing; logs counts.
-
-**analysis/ modules**
-- All read-only from DB; write only to output files (HTML/PNG plots).
-- `political_axis.py`: project all MP SVD vectors onto the first principal component; optionally define axis by anchor parties (e.g. VVD vs SP).
-- `trajectory.py`: collect MP's aligned SVD vector per window → compute drift distance → plot trajectory over time.
-- `clustering.py`: run UMAP on `fused_embeddings` per window → label with policy_area or thematic cluster.
-- `visualize.py`: Plotly interactive scatter/line plots; outputs self-contained HTML.
-
-## Data Flow
-
-```
-Phase 1 — Extract
- motions.voting_results (JSON, existing)
- → extract_mp_votes.py
- → INSERT mp_votes rows (motion_id, mp_name, vote, date)
-
- OData /Kamerlid
- → fetch_mp_metadata.py
- → INSERT mp_metadata rows (mp_name, party, entry_date, exit_date)
- → UPDATE mp_votes.party via JOIN (backfill)
-
-Phase 2 — SVD
- mp_votes (date-filtered per window)
- → sparse MP × motion matrix
- → scipy svds(k=50)
- → raw SVD vectors per window
-
- Procrustes alignment:
- window[t-1] aligned vectors + window[t] raw vectors
- → overlapping MPs as anchors
- → scipy.spatial.procrustes → rotation R
- → window[t] aligned vectors
- → INSERT svd_vectors rows
-
-Phase 3 — Text embeddings (fill gaps)
- motions without embedding in embeddings table
- → text_pipeline.py → ai_provider.get_embedding(body_text or description)
- → INSERT embeddings rows (existing schema)
-
-Phase 4 — Fusion
- svd_vectors (motion, window) + embeddings (motion)
- → fusion.py
- → INSERT fused_embeddings rows
-
-Phase 5 — Analysis (on demand)
- fused_embeddings + mp_metadata + svd_vectors
- → analysis modules
- → HTML plots output
-```
-
-## Error Handling Strategy
-
-- **Extraction idempotency**: `extract_mp_votes` checks `SELECT COUNT(*) FROM mp_votes WHERE motion_id = ?` before inserting; re-runs are safe.
-- **Sparse windows**: if a time window has fewer than `MIN_MOTIONS` (default 20) or `MIN_MPs` (default 10), skip SVD for that window and log a warning. Do not crash.
-- **Procrustes at election transitions**: chain alignment via the last quarter of the old term and first quarter of the new term using only returning MPs. If overlap < 30%, log as HIGH_DISPARITY and store the window but flag it.
-- **Missing text embeddings**: log motions skipped in fusion; the SVD-only path remains valid for those motions.
-- **OData metadata unavailable**: fall back to heuristic party assignment (mp_votes majority-party per MP name). Log which MPs used fallback.
-- **Replace prints with structured logging**: all pipeline modules use `logging.getLogger(__name__)` — not `print()`.
-
-## Testing Strategy
-
-- **Unit**:
- - Vote parser: given sample `voting_results` JSON, assert correct MP rows extracted and party rows ignored.
- - Sparse matrix builder: inject 5 MPs × 10 motions → assert matrix shape and values.
- - Procrustes wrapper: inject two small aligned-then-rotated matrices → assert recovered rotation close to identity.
- - Fusion: inject matching SVD and text vectors → assert concatenated output length = svd_dims + text_dims.
-
-- **Integration**:
- - Extract → SVD → Fusion on a fixture of 50 motions (stored in `tests/fixtures/`). Monkeypatch ai_provider for text embeddings. Assert `fused_embeddings` table populated and vector dimensions correct.
-
-- **Regression**:
- - Run pipeline on a fixed 100-motion snapshot. Assert output dimensions and row counts stable across runs.
-
-- **Migration tests**:
- - Follow existing pattern (`tests/test_migration_embeddings.py`): apply new migration SQL to a temp DuckDB, assert expected tables and columns.
-
-## Open Questions
-
-1. **OData `/Kamerlid` endpoint availability**: does it expose party affiliation and tenure dates with the same API key/base URL we already use? If not, we need a scraping fallback for `mp_metadata`.
-2. **Store rotation matrices?**: MVP stores aligned vectors directly. Should we also persist the Procrustes R matrix per window transition so we can re-project new MPs added later without full recomputation?
-3. **Output target**: CLI producing HTML plots (simplest) vs. new Streamlit page vs. Jupyter notebook. Recommendation: CLI first, Streamlit page in a follow-up.
-4. **Time window granularity**: quarterly is the default. Should we validate this empirically first with an annual window (larger, more stable matrices) and switch to quarterly once the pipeline is proven?
-5. **SVD dimensions k**: default 50 dims for SVD. This needs to be validated against the actual data size (number of unique MPs × motions per window). A window with 100 MPs and 50 motions cannot have k=50 — needs to be `k < min(n_mps, n_motions)`. Pipeline must enforce this dynamically.
diff --git a/thoughts/shared/designs/2026-03-22-embedding-similarity-cache-design.md b/thoughts/shared/designs/2026-03-22-embedding-similarity-cache-design.md
deleted file mode 100644
index 1b08d42..0000000
--- a/thoughts/shared/designs/2026-03-22-embedding-similarity-cache-design.md
+++ /dev/null
@@ -1,145 +0,0 @@
----
-date: 2026-03-22
-topic: "Embedding-Based Motion Similarity Cache"
-status: validated
----
-
-## Problem Statement
-
-We have text embeddings and fused (SVD + text) embeddings stored for motions, but no usable similarity search. The current `database.search_similar()` is a full Python scan — it SELECTs all embeddings, parses JSON one by one, and computes cosine similarity with `zip` in pure Python. This is O(N) per query with no vectorized math, no indexing, and no caching. The similarity cache migration (`2026-03-22-add-similarity-cache.sql`) is a commented-out placeholder with no executable SQL.
-
-Additionally, several infrastructure gaps block a working similarity system:
-- The `embeddings` table is not created by `_init_database()` (only exists via migration file)
-- The fusion pipeline has an N+1 query pattern (per SVD row queries embeddings separately)
-- `ai_provider._post_with_retries` does not retry on 429 (rate limit) responses
-
-## Constraints
-
-- DuckDB only — no pgvector, no external vector store
-- Vectors stored as JSON text columns (existing format, not changing)
-- DuckDB connections are short-lived (open/close per method)
-- Do not modify `app.py` or `scheduler.py`
-- Tests must be offline (monkeypatch network calls)
-- Functional style, Python, uv
-- Logging via `getLogger`, no `print()`
-
-## Approach
-
-**Precomputed similarity cache** — batch-compute top-K nearest neighbors per motion and store results in a cache table. The UI reads the cache with a simple indexed lookup.
-
-Rationale: the motion corpus changes slowly (new motions trickle in from parliament). Computing nearest neighbors at query time is wasteful. One offline O(N^2) pass via numpy matrix multiplication gives us O(1) lookups forever until the next recompute.
-
-Alternatives rejected:
-- **DuckDB vss extension (HNSW)**: experimental, requires vector format migration away from JSON text, overkill for ~thousands of motions
-- **Real-time numpy search**: better than pure-Python zip, but still O(N) per query; caching eliminates repeated work
-- **FAISS/Annoy ANN index**: designed for millions of vectors, unnecessary complexity at our scale
-
-## Architecture
-
-```
-New files:
- similarity/
- __init__.py
- compute.py -- batch pairwise cosine, extract top-K, write cache
- lookup.py -- read cached results for a motion
-
-Modified files:
- database.py -- add similarity_cache + embeddings to _init_database,
- add store/read/clear helpers, deprecate old search_similar
- migrations/2026-03-22-add-similarity-cache.sql -- uncomment and finalize
- ai_provider.py -- add 429 to retry branch
- pipeline/fusion.py -- fix N+1 with bulk JOIN
-```
-
-## Components
-
-### similarity_cache table
-
-```
-similarity_cache (
- id INTEGER DEFAULT nextval('similarity_cache_id_seq'),
- source_motion_id INTEGER NOT NULL,
- target_motion_id INTEGER NOT NULL,
- score REAL NOT NULL,
- vector_type TEXT NOT NULL, -- 'text', 'fused', 'svd'
- window_id TEXT, -- NULL for text-only, set for fused/SVD
- created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-)
-```
-
-Composite index on `(source_motion_id, vector_type, window_id)` for fast lookups.
-
-### similarity/compute.py
-
-- Load all vectors of a given type into a numpy matrix in one query (parse JSON, stack into ndarray)
-- Normalize rows to unit length
-- Compute full cosine similarity matrix via `normalized @ normalized.T`
-- Extract top-K per row (excluding self-similarity)
-- Bulk-insert results into `similarity_cache`
-- Idempotent: `clear_similarity_cache(vector_type, window_id)` then insert within same connection scope
-
-Public function: `compute_similarities(vector_type='fused', window_id=None, top_k=10, db_path=None)`
-
-### similarity/lookup.py
-
-- `get_similar_motions(motion_id, vector_type='fused', window_id=None, top_k=10, db_path=None)` — SELECT from cache ordered by score DESC
-- Returns list of dicts: `{motion_id, score}`
-- Optionally join motion metadata (title, layman_explanation) for richer results
-- Graceful degradation: empty cache returns empty list
-
-### database.py changes
-
-1. Add `embeddings` table creation to `_init_database()` — matches migration schema
-2. Add `similarity_cache` table + sequence creation to `_init_database()`
-3. New helpers:
- - `store_similarity_batch(rows: list[dict])` — bulk INSERT
- - `get_cached_similarities(source_motion_id, vector_type, window_id=None, top_k=10)` — read
- - `clear_similarity_cache(vector_type, window_id=None)` — DELETE for idempotent recompute
-4. Deprecate `search_similar()` — mark with a log warning pointing to `similarity.lookup`
-
-### ai_provider.py fix
-
-- Add HTTP 429 to the retry branch in `_post_with_retries`
-- If `Retry-After` header is present, use it as the backoff delay; otherwise fall back to existing exponential backoff
-- This is a single-line condition change plus header parsing
-
-### pipeline/fusion.py fix
-
-- Replace the per-row SELECT from `embeddings` with a single bulk query:
- JOIN `svd_vectors` with latest `embeddings` per motion_id in one SQL statement
-- Loop over joined results and concatenate in Python
-- Eliminates N+1 query pattern
-
-## Data Flow
-
-1. Existing pipeline runs: extract MP votes → SVD → text embeddings → fusion
-2. After fusion completes, `similarity/compute.py` loads all fused vectors for the window into a numpy matrix
-3. Computes pairwise cosine similarity matrix, extracts top-K per motion
-4. Bulk-inserts results into `similarity_cache` (clearing previous cache for that batch first)
-5. Separately, text-only similarity can be computed across all motions (no window dependency)
-6. UI calls `similarity/lookup.py` for a direct indexed read — instant response
-
-## Error Handling
-
-- **Missing vectors**: motions without embeddings are excluded from the similarity matrix; not an error
-- **Empty matrix**: if no vectors exist for a vector_type/window, log warning and skip (don't write empty cache)
-- **DB write failures**: wrap cache writes in try/except, log error, don't crash the pipeline; similarity is non-critical
-- **Stale cache**: cache is fully replaced on each recompute (delete + insert in same connection scope); if recompute fails partway, old cache remains valid
-- **Dimension mismatch**: vectors with inconsistent dimensions are padded or excluded with a warning (following existing clustering.py pattern)
-
-## Testing Strategy
-
-- **Unit: compute.py** — create known vectors with predictable cosine similarities (e.g., identical vectors → score 1.0, orthogonal → 0.0), verify matrix math produces correct top-K ordering
-- **Unit: lookup.py** — seed cache table in temp DB, verify queries return correct ordered results, verify empty cache returns empty list
-- **Unit: database helpers** — test store_similarity_batch / get_cached_similarities / clear_similarity_cache round-trip
-- **Unit: ai_provider 429 retry** — monkeypatch requests.post to return 429, verify retry with backoff
-- **Unit: fusion bulk join** — verify N+1 elimination produces same results as original
-- **Migration test** — apply updated similarity_cache migration on temp DuckDB, verify schema matches expected columns
-- **Integration test** — insert fake embeddings → run compute → verify cache populated → lookup returns expected results
-- **All tests offline**: in-memory DuckDB, monkeypatched network calls
-
-## Open Questions
-
-None blocking. Future enhancements (not in scope):
-- MP-to-MP similarity from SVD vectors (explorer UI is motion-focused for now)
-- Real-time similarity for newly ingested motions before next batch recompute
diff --git a/thoughts/shared/designs/2026-03-22-motion-explorer-design.md b/thoughts/shared/designs/2026-03-22-motion-explorer-design.md
deleted file mode 100644
index 8b3bedf..0000000
--- a/thoughts/shared/designs/2026-03-22-motion-explorer-design.md
+++ /dev/null
@@ -1,165 +0,0 @@
----
-date: 2026-03-22
-topic: "Dynamic motion explorer + analysis refresh"
-status: validated
----
-
-## Problem Statement
-
-The parliamentary embedding pipeline now covers 2019–2026 with ~25,000 motions, quarterly SVD windows, fused embeddings, and a 200k+ similarity cache. None of this is visible to anyone in an interactive form. The only outputs today are static HTML files written by `generate_compass.py` (if it's been run), and a blog post with placeholder numbers.
-
-We need to:
-1. Regenerate all analyses and output graphs with the full dataset
-2. Build an interactive Streamlit explorer that surfaces the political compass, party trajectories, and motion similarity search
-3. Update the blog post with real numbers and findings
-
-## Constraints
-
-- Do NOT modify `app.py` or `scheduler.py` — these are the production quiz app
-- All DB access in the explorer must be **read-only** (no writes) — pipeline may be running
-- Explorer must work with existing `analysis.*` modules; no new analysis logic
-- Use `@st.cache_data` aggressively — `compute_2d_axes` runs PCA across all windows and is expensive (seconds, not milliseconds)
-- No new external dependencies beyond what's already installed (streamlit, plotly, umap-learn, scikit-learn are all present)
-- Follow existing code style: functional Python, `logging.getLogger(__name__)`, no print statements in library code
-
-## Approach
-
-**Single-file `explorer.py`** at the project root alongside `app.py`.
-
-Four Streamlit tabs:
-1. **Politiek Kompas** — 2D MP/party scatter with a window slider
-2. **Partij Trajectories** — Line traces of party positions over time on the compass
-3. **Motie Zoeken** — Free-text + filter search, returns ranked similar motions
-4. **Motie Browser** — Filterable table of all motions, click to expand detail + similar motions
-
-Run with: `streamlit run explorer.py`
-
-This approach is chosen because:
-- Reuses all existing `analysis.*` modules without changes
-- Single file means no new package structure to maintain
-- Streamlit tabs map naturally to the four distinct views a researcher would want
-- Read-only DB access means it can run concurrently with the pipeline
-
-## Architecture
-
-```
-explorer.py
- ├── Tab 1: Politiek Kompas
- │ └── analysis.political_axis.compute_2d_axes (cached)
- │ └── analysis.visualize.plot_political_compass → Plotly figure
- │
- ├── Tab 2: Partij Trajectories
- │ └── analysis.trajectory.compute_2d_trajectories (cached)
- │ └── analysis.visualize.plot_2d_trajectories → Plotly figure
- │
- ├── Tab 3: Motie Zoeken
- │ └── database.get_all_motions (cached, read-only)
- │ └── database.search_similar (similarity_cache lookup)
- │ └── Custom search: filter title/description + show voting_results
- │
- └── Tab 4: Motie Browser
- └── database.get_filtered_motions (cached, read-only)
- └── On click: database.search_similar for related motions
-```
-
-## Key Components & Responsibilities
-
-**`explorer.py`**
-- Page config: `st.set_page_config(layout="wide", page_title="Parlement Explorer")`
-- Sidebar: DB path input (default `data/motions.db`), window-size toggle (annual/quarterly)
-- `@st.cache_data` wrappers for all expensive DB reads and computations
-- Four tabs via `st.tabs([...])`
-
-**Tab 1 — Politiek Kompas**
-- Calls `compute_2d_axes(db_path, method='pca', pca_residual=True)` — cached
-- Window selector slider showing available windows
-- Renders the Plotly scatter for the selected window using `_render_compass_for_window(positions_by_window, window_id, party_map, axis_def)` — a thin Plotly figure builder (not writing to file)
-- Hover: MP name, party, (x, y) coordinates
-- Color by party using `_load_party_map(db_path)` — cached
-
-**Tab 2 — Partij Trajectories**
-- Same `positions_by_window` data from Tab 1 (shared cache hit)
-- Multi-select party filter (default: all major parties)
-- Plotly figure: one trace per party, x/y positions connected by lines, labeled by window_id
-- Toggle between showing MPs or just party centroids (computed as mean of MP positions per party per window)
-
-**Tab 3 — Motie Zoeken**
-- Search input (Dutch text, free-form)
-- Filters: year range (slider), policy area (multi-select), controversy score (slider)
-- On search: filter `motions` table in-memory against title + layman_explanation text (case-insensitive substring; no embedding search needed at this level)
-- Results list: each result shows title, date, policy area, controversy, layman_explanation
-- Expandable section per result: full description/body_text + "Vergelijkbare moties" from `similarity_cache`
-- Voting breakdown: parse `voting_results` JSON to show Voor/Tegen/Onthouden per party
-
-**Tab 4 — Motie Browser**
-- `st.dataframe` with all motions (title, date, policy_area, controversy_score, winning_margin)
-- Column filters at top: year, policy area
-- Sort by: date DESC, controversy DESC, winning_margin ASC (most contested first)
-- Click row → `st.session_state` stores selected motion_id → detail panel below table
-- Detail panel: full motion text + top-10 similar motions from similarity_cache
-
-## Data Flow
-
-1. On startup: `compute_2d_axes` runs PCA, results cached in Streamlit's in-memory cache
-2. Tab 1/2: pure reads from `svd_vectors` + `mp_metadata` — all cached after first load
-3. Tab 3: on each search, filter pre-loaded motions DataFrame in-memory (no DB query per keypress)
-4. Tab 4: full motions table loaded once and cached; similarity lookups hit `similarity_cache` table via existing `database.get_cached_similarities`
-
-All DuckDB connections are opened with `read_only=True` to allow concurrent pipeline access.
-
-## Error Handling
-
-- If `compute_2d_axes` fails (insufficient data for a window), skip that window and log warning — don't crash the app
-- If `similarity_cache` has no entries for a motion (e.g., new motion not yet processed), show "Nog geen vergelijkbare moties beschikbaar" placeholder
-- If DB file doesn't exist at startup, show an error banner with the path and instructions
-- All `duckdb.connect` calls wrapped in try/finally to guarantee close
-
-## Analysis Refresh Plan
-
-Before building the explorer, regenerate all outputs:
-
-```bash
-# 1. Generate political compass HTML for latest window (annual)
-.venv/bin/python scripts/generate_compass.py \
- --db data/motions.db --out outputs \
- --method pca --pca-residual
-
-# 2. Generate similarity cache for new windows (2019–2021, 2024 quarters)
-# (run_pipeline with --skip-metadata --skip-extract --skip-svd --skip-text)
-.venv/bin/python -m pipeline.run_pipeline \
- --db-path data/motions.db \
- --start-date 2019-01-01 --end-date 2025-01-01 \
- --window-size quarterly \
- --skip-metadata --skip-extract --skip-svd --skip-text
-
-# 3. Recompute similarity cache for all windows
-.venv/bin/python -c "
-from similarity.compute import recompute_all_windows
-recompute_all_windows('data/motions.db', window_size='quarterly', top_k=20)
-"
-```
-
-## Blog Post Updates
-
-Target: `thoughts/blog-post-political-compass.md`
-
-- Replace placeholder motion counts table with real numbers from DB query
-- Add actual findings from quarterly analysis (not visible in annual windows):
- - 2020-Q2 COVID vote clustering — parties converge on emergency measures
- - 2022-Q4 nitrogen crisis — sharpest left-right split in dataset
- - 2023-Q1 → 2024-Q1 gap (data missing for Q2-Q4 2023)
-- Add "Explorer" section describing `explorer.py` and how to run it
-- Update similarity cache row count (was 212k, now higher with new windows)
-- Fix the "fused = [10] + [2560] = 2570" claim — verify actual dimensions
-
-## Testing Strategy
-
-- Explorer has no tests (it's a UI script) — verify manually by running `streamlit run explorer.py` after pipeline completes
-- Existing 34 tests stay green — no changes to library modules
-- Run tests after completing implementation: `.venv/bin/python -m pytest -q`
-
-## Open Questions
-
-- Should the explorer ship as a separate port from `app.py`? (Recommendation: yes, `app.py` stays on its port, `explorer.py` runs on a different port for internal/research use)
-- Should `Verworpen.` motions be filtered from search results by default? (Recommendation: yes, add a "Toon verworpen" toggle defaulting to off)
-- Annual or quarterly windows as the default for the compass? (Recommendation: annual — less noise, cleaner trajectories; quarterly available via sidebar toggle)
diff --git a/thoughts/shared/designs/2026-03-22-stematlas-deployment-design.md b/thoughts/shared/designs/2026-03-22-stematlas-deployment-design.md
deleted file mode 100644
index d1182eb..0000000
--- a/thoughts/shared/designs/2026-03-22-stematlas-deployment-design.md
+++ /dev/null
@@ -1,229 +0,0 @@
----
-date: 2026-03-22
-topic: "StemAtlas — Public Deployment on sgeboers.nl"
-status: validated
----
-
-# StemAtlas Deployment Design
-
-## Problem Statement
-
-The stemwijzer project has three user-facing products ready to publish:
-1. **A blog post** explaining the political compass methodology and findings
-2. **An interactive explorer** (political compass, party trajectories, motion search)
-3. **The stemwijzer quiz** (vote on motions, see which parties match you)
-
-These need to be deployed publicly on sgeboers.nl using the existing VPS + Gitea + Drone + Docker stack.
-
----
-
-## The Name: StemAtlas
-
-**`stematlas.sgeboers.nl`**
-
-Dutch wordplay: **stem** = *vote* AND *voice* (as in "the voice of parliament") + **atlas** = a comprehensive map of the world. Together: *an atlas of voices* — a map of how Dutch democracy sounds from the inside.
-
-It's broader than "stemwijzer" (which implies a voting guide) — it positions the site as a data exploration and journalism tool.
-
----
-
-## Constraints
-
-- Existing VPS running Nginx, Gitea, Drone
-- Deployment pipeline: Docker build → push to registry → SSH `docker-compose up -d`
-- sgeboers.nl is a **raw HTML/CSS site** (not Hugo) hosted as a repo on git.sgeboers.nl
-- DuckDB file lives on the VPS — single writer (scheduler), multiple readers (Streamlit)
-- No new cloud services or hosting costs
-
----
-
-## Architecture
-
-```
-Internet
- │
- ├── sgeboers.nl (raw HTML/CSS site, existing repo on git.sgeboers.nl)
- │ └── blog/stematlas.html ← blog post with inline charts + link to subdomain
- │
- └── stematlas.sgeboers.nl
- └── Nginx (reverse proxy)
- └── Streamlit multi-page app (port 8501)
- ├── Page 1: Stemwijzer Quiz (app.py)
- └── Page 2: Explorer (explorer.py)
-
-VPS filesystem:
- /srv/stematlas/
- ├── data/motions.db ← DuckDB (shared, read-write by scheduler)
- └── docker-compose.yml
-```
-
----
-
-## Components
-
-### 1. Streamlit Multi-Page App
-
-Restructure entry point from `app.py` → `Home.py` with a `pages/` directory:
-
-```
-Home.py ← landing page / about
-pages/
- 1_Stemwijzer.py ← quiz (app.py content)
- 2_Explorer.py ← explorer.py content
-```
-
-Streamlit's built-in multi-page routing handles navigation. One Docker container, one port (8501).
-
-**Why not two separate containers?**
-Single shared DuckDB file on VPS filesystem. Both pages open read-only connections (quiz opens read-write for session data, but that's the existing behaviour). One container = one volume mount = no coordination overhead.
-
-### 2. Docker Compose
-
-The existing `.drone.yml` already calls `docker-compose up -d` on the VPS. We add/update `docker-compose.yml`:
-
-```
-Services:
- stematlas:
- image: registry/stematlas:latest
- ports: 8501 (internal only)
- volumes:
- - /srv/stematlas/data:/app/data ← persistent DB
- restart: unless-stopped
-
- scheduler:
- image: registry/stematlas:latest
- command: python scheduler.py
- volumes:
- - /srv/stematlas/data:/app/data ← same DB, write access
- restart: unless-stopped
-```
-
-**Scheduler as a sidecar**: runs in the same image but different container, keeps DB updated nightly. Streamlit container never writes to DB (except user sessions in the quiz).
-
-### 3. Nginx Vhost
-
-New server block on the VPS:
-
-```
-stematlas.sgeboers.nl → proxy_pass http://127.0.0.1:8501
-```
-
-Standard Streamlit proxy requirements: `proxy_http_version 1.1`, WebSocket upgrade headers for `/_stcore/stream`. Let's Encrypt cert via Certbot (standard pattern).
-
-### 4. Drone CI Pipeline Update
-
-Existing `.drone.yml` steps remain identical — build, push, SSH deploy. The only change: `docker-compose.yml` in the repo now references both the `stematlas` and `scheduler` services, so `docker-compose up -d` picks them both up.
-
-No new Drone secrets needed if `DOCKER_REGISTRY`, `DEPLOY_HOST` etc. are already set.
-
-### 5. Blog Post (Raw HTML page on sgeboers.nl)
-
-The blog post is a new `blog/stematlas.html` file added to the sgeboers.nl repo on git.sgeboers.nl. The Drone pipeline for that repo deploys it like any other static file — push to git, Drone copies to webroot, Nginx serves it.
-
-**Chart embedding strategy — inline Plotly divs:**
-
-Rather than iframes, we extract just the chart `` + `