Populated the right_wing_motions.category column (previously 100% NULL across
3,030 motions) via parallel subagent classification — 80 agents derived a
10-category taxonomy and classified all motions in minutes.
Adds to the Overton QMD report:
- Plotly dropdown filter on Chart 1 to toggle between policy categories
- Chart 7: category delta bar chart (pre/post centrist support per domain)
- Chart 8: quarterly domain trajectories for the 5 largest categories
- Domain Decomposition narrative section
Also fixes a Streamlit tab crash (m.text -> m.body_text) and adds TDD tests.
CRIT-1: Stylistic extremity direction reversed — both dimensions declined
(stijl 1.875→1.744, not increased +0.097). Holistic moderation, not divergence.
CRIT-2: Masking rate corrected from 36.1% to 9.7% (S≤2, M≥4 on full dataset).
Original 36.8% was from 117-motion manual audit, not extrapolatable.
CRIT-3: Material impact values harmonized to motion-level means (2.79→2.45).
Old values (2.78→2.43) used undisclosed mean-of-yearly-means aggregation.
HIGH-1: Migration domain provenance caveat — category column is NULL,
analysis relies on title keyword matching (approximate boundaries).
HIGH-2: 2026-Q2 bounce caveat — n=44, bimodal distribution, sensitive to
composition (many consensus defense motions among CS=1.0 items).
HIGH-3: Non-right-wing control group corrected — CS rose 58%→62% (+3.5pp),
not 'flat at 49%'. Surge was disproportionate for right-wing content.
Also: fixed 6-party centrist definition (line 30) to 4-party,
removed 'did not shift rightward' phrasing, added Phase 4 synthesis
reminder to build_all_reports.py.
Key finding from 2026 data:
- 2026-Q2 CS bounced back to 0.523 (from 0.334 in Q1)
- Migration CS (0.395) now EXCEEDS non-migration (0.368) for first time
- Multiple 2026-Q2 migration motions got unanimous centrist support (CS=1.00)
Updated verdict across synthesis, QMD, and HTML:
- Non-migration acceptance was a temporary electoral shock response
- Migration acceptance is durable and growing as debate intensifies
- The Overton shift is domain-specific, not uniformly temporary
Also fixed hashline formatting corruption in synthesis file.
- Verdict now says 'window widened' (not 'did not shift') — centrist
support surged for right-wing motions while staying flat for left-wing
- Migration reframed from 'one exception' to 'gateway domain' — where
acceptance expanded most genuinely and right-wing parties learned
frames they applied elsewhere
- Explorer Overton tab: added migration gateway section with pre/post
metrics, full motion text (no truncation), 100-motion browser
- Explorer Kompas tab: updated Overton context to lead with the shift
- Explorer Trajectories tab: Dutch-language Overton annotation
- Synthesis, QMD, HTML report, STATUS.md all updated consistently
- Fetched 276 new motions from Tweede Kamer API (2026-04-23 to 2026-05-31)
- Fixed classify_motions.py: DROP TABLE → CREATE TABLE IF NOT EXISTS
- Restored derived columns (centrist_support_strict, category, etc.) via migration
- Scored 180 missing motions in extremity_scores_2d (now 3,049 total, 0 missing)
- Re-ran temporal trajectory with updated data (inflection: 2024-Q2)
U1: JA21 drives moderation effect (+0.203 CS shift, only party with volume+support gains)
U2: Coalition coding split at July 2024 — opposition effect confirmed (d=0.85 vs 0.87)
U3: Voting margin (ρ=0.812 with centrist support) is far superior to pass rate
U4: SVD trajectory confirms spatial divergence — centrists moved left (Δx=-0.30), right stationary
U5: Mechanism classification Cohen's κ=0.41 (moderate) — taxonomy needs revision
U6: Predictive model AUC-ROC=0.81 — submitter party and category are strongest predictors
Material impact declined post-2024 (2.78→2.43, M>=4 share 23.7%→11.3%).
Right-wing strategic moderation: more motions, milder content, better
framing. The Overton window did not expand — right-wing proposals
shifted into the existing window. 'Acceptance through moderation'
replaces 'acceptance without conversion.'
SVD axes capture agreement structure — centrists 'moving left' means
voting patterns diverged from right-wing, not that parties changed
ideology. 'Acceptance without conversion' is a behavioral claim.
Documented as best-practice learning.
- Classified 24 post-2024 right-wing motions with CS>=0.5
- Dominant mechanisms: consensus framing (33%), institutional (21%), welfare (17%)
- Only 1 targeted restriction, zero system dismantling
- Right-wing gains centrist support through repackaging, not conversion
- Confirms acceptance-without-conversion dynamic at the mechanism level
- Pearson r=0.45 between stylistic and material impact (separable)
- Material impact averages 0.85 points above stylistic
- 36.8% of motions mask high-impact policy behind restrained language
- Original single-score conflates language vs substance
- Mark U4 mechanism analysis as in progress
- Project-local skill .opencode/skills/score-extremity/ for subagent dispatch
- Orchestrator extremity_rescore_2d.py with load_skill/sample/format/validate/store
- 16 TDD tests covering all orchestrator functions
- 117 motions scored by deepseek v4 flash subagents (12 parallel batches)
- Pearson r=0.45 between stylistic and material dimensions — separable
- Key finding: 36.8% of motions use restrained language for consequential policies
- 2d_extremity_correlation_report.md documents distribution, divergence patterns,
and implications for the Overton acceptance-without-conversion narrative
Captures the 7-step methodology developed through multiple analysis
iterations: strict centrist definition, Procrustes-aligned SVD with
anchor-party sign validation, centrist support fraction over pass rate,
acceptance-without-conversion test, opposition-only filter as coalition
control, extremity-stratified tolerance analysis, and LLM score auditing.
- Reclassified centrist to {D66, CDA, CU, NSC} — removing VVD/BBB
which are center-right coalition partners
- Added centrist_support_strict (0.251→0.507, d=+0.65), center_right_support,
and left_support_mp columns via migration script
- Figure 1 now shows center-right (VVD/BBB) support as orange dashed line
- New Figure 3: bar chart of left-party support for right-wing motions
(0.268→0.202, left opposition hardened)
- New report Section 6 covering left-wing support trends
- All analysis now uses strict centrist definition throughout
Extremity Scorer (U4 enhanced):
- Now scores BOTH original motion text AND layman explanation separately
- Schema: text_score, text_explanation, layman_score, layman_explanation
- Text scores: 1→7, 2→33, 3→5, 4→5 (mild-to-moderate)
- Layman scores: 1→12, 2→20, 3→17, 4→1 (slightly milder)
Sentiment Analysis (U5 enhanced):
- Now scores BOTH original motion text AND layman explanation separately
- Schema: text_score, text_explanation, layman_score, layman_explanation
- Text sentiment avg: 0.294 (slightly positive)
- Layman sentiment avg: 0.416 (more positive - summaries tone down hostility)
Category Derivation (new):
- Two-phase LLM approach: derive taxonomy from sample, then apply to all
- Discovered 7 categories from 30-motion sample:
veiligheid/justitie, corona/pandemie, economie/belasting, klimaat/milieu,
defensie/buitenland, asiel/vreemdelingen, overig
- Applied to 50 motions with distribution shown in DB
- Adds category + category_explanation columns to right_wing_motions
Implements U5: sentiment_analysis.py uses LLM batch calls (fallback when no
local Dutch sentiment model is available) to score motion sentiment on [-1, 1]
scale.
Design:
- Prompt asks for sentiment from -1 (hostile/aggressive) to 1 (constructive)
- JSON schema enforces numeric score + Dutch explanation
- Batch size 10, max_workers 5 for parallel API calls
- Stores results in table
- Updates with avg_sentiment, sentiment_std,
pct_strongly_negative per year
Sample validation (50 motions): good variance across [-0.9, 1.0] range.
Implements U3: temporal_analysis.py computes yearly_summary from the
right_wing_motions table (U2 output).
Metrics per year:
- total_right_wing, pct_of_total, total_motions
- avg_right_support, avg_left_opposition, centrist_support
- avg_right_keyword_matches, extremity_index (U4 placeholder)
- yoy_right_wing_delta, yoy_pct_delta
Key finding: right-wing motions grew from ~4% (2018) to ~12% (2024-2025)
of all motions, with rising centrist support over time.
Removes embedding-heavy search and generic browser tabs from the Explorer.
The project does not currently use embeddings meaningfully, so the similarity
search and browser features were dead weight.
Changes:
- explorer.py: Remove search and browser tabs, keep compass/trajectories/SVD
- explorer.py: Remove fused_embeddings and similarity_cache stats from sidebar
- Home.py: Update Explorer description to match new focused layout
- analysis/tabs/__init__.py: Remove search and browser exports
- tests: Update decomposition and import tests for new tab set
Result: Explorer now has 3 focused analytical tabs instead of 5.
- compute_svd_for_window now computes explained variance ratio (s²/sum(s²))
and appends it as a metadata row (entity_type='metadata',
entity_id='explained_variance') to motion_rows
- load_scree_data reads this metadata row from svd_vectors instead of
querying the non-existent sv_metadata column
- run_svd_for_window counts only entity_type='motion' rows in stored_motion
so metadata rows don't inflate the count
- Added 5 TDD tests covering load, compute, store, and round-trip
All 227 tests pass.
- load_scree_data: return [] with TODO until schema stores EVR metadata
- load_party_axis_scores: compute from vectors instead of missing table
- load_party_axis_scores_for_window: same vector-based fallback
- load_party_scores_all_windows[_aligned]: check table existence,
fall back to computing from load_positions when absent
All functions predated decomposition (5afbad1, 2026-04-05) and relied on
party_axis_scores / sv_metadata columns that were never created.
- Add verify-lint-rule-scope-before-relying-on-it: guidance on
confirming lint rule coverage before trusting it for enforcement.
Documents the P2-002 incident where ruff BLE only catches bare
not .
- Update working-tree-hygiene: add dev-tool-in-venv check and
ruff dependency example.
- Add pytest-benchmark to dev dependencies
- Benchmark SVD decomposition on synthetic vote matrix
- Benchmark cosine similarity at small/medium/large scales
P5-003: Benchmark suite
- Wrap duckdb.connect() in try/except with specific duckdb.Error
- Replace bare with in _init_database
- Replace broad with for ALTER TABLE
- Add ruff BLE (blind except) lint rule to prevent regressions
- Add tests verifying graceful error handling for connect, insert, query
P2-002: Fix broad exception handling
- Add 2026-04-24 ROADMAP with 5 phases / 17 items
- Add detailed implementation plans for P1-001 through P4-005
- Add research artifacts and solution docs from ledger merge
- Add test for SVD component 1 compass alignment
- Enable T20 (flake8-print) for all Python files
- Exclude scripts/, tools/, and .mindmodel/examples/ where
stdout output is acceptable
Completes U6 of P2-001 (replace print with logging)
Replace all print() calls with logger.info/logger.error using
lazy % formatting. Add test verifying error path emits ERROR log.
U2 of P2-001 (replace print with logging)
- Create logging_config.py with configure_logging() helper
- Add tests for level setting, format, idempotency, and inheritance
U1 of P2-001 (replace print with logging)
- Fix mindmodel-schedule.yml to use uv and Python 3.13
- Add pytest.yml for push/PR test gate
- Remove broken scheduler service from docker-compose.yml
- Consolidate config.py into analysis/config.py with backward-compat shim
- Rewrite README.md with quickstart and project overview
- Update pre-commit-config.yaml to enable black, ruff, isort hooks
- Add pyright type-check job (continue-on-error until baseline fixed)
- Update AGENTS.md with Gitea infrastructure note
- Remove stale thoughts/ledgers/ and thoughts/shared/ artifacts
- Fix .gitignore duplicate .worktrees entry
- Move pyright to [dependency-groups] dev
- Replace hardcoded blog correlation with reproducible metric reference
- Add docs: verify-session-artifacts, fusion-vector-dimensions,
working-tree-hygiene
- Update blog-numbers-from-pipeline-outputs with correlation example
_get_aligned_party_scores and _get_aligned_trajectory_scores both called
compute_nd_axes() with no window_ids, which defaulted to _load_window_ids()
returning ALL windows including quarterly. This caused the SVD component 1
bar chart to disagree with the compass (which correctly used annual-only
windows via get_uniform_dim_windows). D66 appeared between GL-PvdA and PvdD
in component 1 because quarterly windows contaminated the PCA basis.
- analysis/explorer_data.py: add AND window_id NOT LIKE '%-Q%' to
_UNIFORM_DIM_SQL so quarterly windows are filtered at the source
- explorer.py: remove stale comment justifying quarterly inclusion;
remove redundant '-Q' guard in SVD tab trajectory view
- scripts/recompute_svd.py: replace quarter_bounds() with year_bounds()
that handles annual window IDs like '2024'; filter window list to
annual-only before recomputing SVD
Investigation of GroenLinks-PvdA merger dynamics in SVD space:
- Finding 1: GL-PvdA were 2.8-10.5% of avg inter-party distance apart pre-merger
- Finding 2: Merged party started most cohesive (#1 in 2023) but now 55% above avg spread
- Finding 3: Converged to 4.5% by Q3 2023, essentially indistinguishable
- Finding 4: GL/PvdA were most stable parties (10-25% drift) while VVD/D66 moved 70-177%
Revise SVD_THEMES labels based on TF-IDF analysis of top 50 motions
per component (pool size: current_parliament). Manual review of motion
titles ensures labels reflect actual parliamentary content rather than
party position semantics.
Key corrections:
- Axis 1: fiscal/economic policy vs social welfare + international rights
- Axis 4: active international engagement vs restraint
- Axis 5: pragmatic financial support vs progressive individual rights
- Axis 6: fossil fuels/financial incentives vs climate/intl rights
- Axis 7: practical-administrative vs idealistico-procedural (kept)
- Axis 8: European defense cooperation vs domestic socioeconomic policy
- Axis 9: concrete-administrative vs systemic reform
- Axis 10: citizen protection vs government regulation
Subagent analysis caught that axes 5 and 6 are NOT the same
(Nationale soevereiniteit) — manual motion review confirms distinct
content for each. Axes 1, 5, 6 had completely wrong labels.
Refs: thoughts/explorer/svd_label_review.md
See also: docs/brainstorms/2026-04-13-topic-derived-svd-labels-requirements.md
- Add _get_aligned_trajectory_scores() helper for multi-window aligned scores
- Update trajectory call to use compute_nd_axes instead of raw SVD scores
- Simplify _render_svd_time_trajectory by removing per-window flip computation
- Add compute_nd_axes() for N-component PCA with Procrustes alignment
- Add _get_aligned_party_scores() helper in explorer.py
- Update build_svd_components_tab to use aligned scores for all components
- Compute flip direction from aligned score centroids using CANONICAL_LEFT/RIGHT
Previously the SVD components tab used raw SVD scores while the compass
used Procrustes-aligned PCA positions. This caused party orderings to
differ between the two visualizations.
Changes:
- Components 1-2 now use aligned positions from load_positions()
(same as compass) for consistent party ordering
- Components 3-10 continue to use raw SVD scores
- Added _get_aligned_party_coords() helper to convert aligned MP
positions to party centroids
Previously the compass (political_axis.py) used hardcoded party sets that
excluded Volt and PvdD, while the SVD components tab (svd_labels.py) used
CANONICAL_LEFT/RIGHT which includes them. This caused inconsistencies in
axis orientation where Volt appeared most left on the compass but PvdD
appeared most left in the SVD components visualization.
Changes:
- Import CANONICAL_LEFT/RIGHT from config in political_axis.py
- Replace hardcoded party sets with CANONICAL_LEFT/RIGHT for axis orientation
- Update tests to match new SVD_THEMES labels
Redo theme analysis after pool-based motion assignment change.
New labels reflect actual motion content per component:
1. Economische sectorbelangen versus sociale welvaart
2. Nationalistische versus multilateralistische oriëntatie
3. Verzorgingsstaat versus defensie en nationale veiligheid
4. Internationale instituties en multilateralisme versus nationale soevereiniteit
5. Gemeenschapszin versus individuele rechten
6. Ecologische transitie versus economische conservatie
7. Praktisch-bestuurlijk versus idealistisch-proceduraal
8. Internationale samenwerking versus nationale soevereiniteit
9. Pragmatische probleemoplossing versus regulering
10. Minder overheidsbemoeienis versus meer handhaving
- Added --pool-size argument (default 50) to control pool size
- Pool mode is now default; use --no-exclusive for old behavior
- Algorithm: for each component, claim top 5 positive + 5 negative from pool
- All 10 SVD components now have exactly 10 representative motions
Also removes tests that require missing dependencies (sklearn, plotly) or
missing files (.mindmodel/manifest.yaml):
- tests/mindmodel/ (2 files)
- tests/test_diagnose_no_plot_trajectories.py
- tests/test_explorer_chart.py
- tests/test_motion_drift.py
- tests/test_trajectories_pipeline_integration.py
- tests/test_trajectory_*.py (4 files)
Refs: thoughts/shared/plans/2026-04-12-svd-axis-label-alignment.md
Previously, components 1-2 in the SVD tab used Procrustes-aligned PCA
coordinates (from load_positions), which meant the SVD tab showed PCA
dimensions of the 50D aligned space rather than the actual raw SVD
components. This was a fundamental inconsistency — the SVD tab's component 2
showed completely different party ordering than the raw SVD component 2.
Changes:
- explorer.py: Unified all components 1-10 to use raw SVD values via
load_party_axis_scores_for_window(). Removed the separate
load_positions() path for components 1-2. Now all components use the
same data source (50D vectors from svd_vectors table).
- explorer.py: Updated flip computation to cover ALL components 1-10
(was range 3-11 for components 3-10 only). The compute_flip_direction
function correctly determines sign for each component.
- explorer.py: Unified rendering to always use _render_party_axis_chart_1d
(was _render_party_axis_chart for components 1-2 using 2D coords).
- explorer.py: Unified trajectory to always use load_party_scores_all_windows.
- analysis/config.py: Updated component 1 label (simplified explanation,
removed coalition-specific policy references).
- analysis/config.py: Updated component 2 label to "Nationalistisch versus
kosmopolitisch" matching raw SVD data (PVV/FVD at positive extreme,
Volt/DENK/GL-PvdA at negative extreme).
- tests: Updated test assertions to match new labels.
- scripts/validate_svd_themes.py: Verified all components pass right-wing
alignment check, config flip consistency, and theme pole consistency.
Fixes the core inconsistency: SVD tab component 2 now uses the same raw
SVD data as components 3-10, with consistent party ordering and labels.
The compass remains a separate PCA-based visualization.
Remove the '🔍 Wat bepaalt deze assen?' expander that showed individual
motion titles (➕/➖) with axis labels and variance explanation. Only the
Stemdiscipline analyse (Rice index) section remains. Also removes the
now-unused _render_axis_motions() helper function.
Delete 17 malformed YAML constraint files and 10 stale numbered
constraint files. Convert domain glossary, patterns, stack, and
anti-patterns to markdown format. Update manifest.yaml to reference
new markdown files.
Add four test files covering:
- test_config.py: SVD_THEMES structure validation
- test_explorer_labels.py: label derivation from positive/negative poles and flip
- test_svd_axis_alignment.py: right-wing centroid on RIGHT side for all axes
- test_validate_svd_themes.py: theme validation script tests
Allow analysis modules to be imported in lightweight test environments
without duckdb installed. Modules that need duckdb for actual queries
still require it at runtime, but import-time failures are handled gracefully.
Two related bugs fixed:
1. Label alignment: Removed static left_pole/right_pole from SVD_THEMES
entries. These labels assumed a fixed flip direction but could mismatch
with runtime flip computation, causing right-wing parties to appear on
the wrong side. Labels are now always derived from positive_pole,
negative_pole, and the runtime flip direction.
2. Score mismatch: Changed tijdtraject view for components 3-10 from
load_party_scores_all_windows_aligned() to load_party_scores_all_windows().
Procrustes alignment rotates the full 50-dim vector space to align
components 1-2, but this also transforms components 3-10, making their
scores incomparable with the single-window view. Per-window flip
computation already handles orientation alignment for these components.
Also updated svd_labels.py to prefer analysis.config as the canonical
source for SVD_THEMES, falling back to explorer only when config is
unavailable.
Key findings:
- Coalition started losing votes structurally from 2019
- Not that 'right' won, but that government lost
- Added government_win_rate.png visualization
- Updated analysis with party vote counts
- Add polarization_analysis.png: spread over time for all axes
- Add axis1_deep_dive.png: focus on Axis 1 (coalition vs opposition)
- Add Dutch blog post on parliamentary polarization findings
- Add script to find motions closest to semantic gravity per axis/window
- Document Axis 1 semantic shift: from administrative law (2016)
to migration/asylum policy (2026)
- Shows that 'coalition' votes on different topics over time
- Implement SVD axis stability using Lasso regression on fused embeddings
- Add overtone shift analysis to detect semantic content changes
- Implement semantic drift tracking for motion content over time
- Add party voting analysis with cross-ideological voting patterns
- Generate markdown report with visualizations
- Add comprehensive test suite with 12 passing tests
See reports/drift/report.md for analysis results.
- Add compute_overtone_shift(): tracks semantic gravity movement across windows
even when party ordering stays the same
- Update _generate_report() with overtone shift section including dimension-level
analysis and inflection point detection
- Update methodology section to reflect new metrics
- All 12 tests pass
Key finding: no axes exceed 0.7 stability threshold — semantic features
defining each SVD axis shift significantly across windows (0.06-0.51 range)
- Replace Procrustes-based stability with Ridge regression on fused embeddings
- For each SVD axis, fit Ridge: SVD_score ~ fused_embedding per window
- Compare weight vectors via max(cosine similarity, Jaccard top-100)
- Add --regression-alpha CLI argument (default 1.0)
- Keep party-based fallback for windows with < 50 motions
- Update tests for new regression-based approach
Key finding: regression weights show moderate stability (0.06-0.51)
but no axes exceed 0.7 threshold — semantic features defining each
axis shift significantly across windows
- Add scripts/motion_drift.py: analyzes SVD axis stability, semantic drift,
and cross-ideological voting patterns across annual windows
- Add analysis/motion_drift.py: core analysis functions with Procrustes
alignment fallback using party-based sign consistency
- Add matplotlib dependency for static chart generation
- Add tests/test_motion_drift.py: 12 tests covering all analysis functions
- Report output: markdown with embedded PNG charts
Key findings from real data:
- No axes are fully stable (>0.7) across 2019-2026
- All axes show moderate consistency (0.40-0.47) — stable within periods
but flip between cabinet periods (2019/2022/2026 vs 2023/2024/2025)
- Party voting analysis detects cross-ideological voting patterns
sgeboers