docs(solutions): document Overton window shift analysis methodology

Captures the 7-step methodology developed through multiple analysis
iterations: strict centrist definition, Procrustes-aligned SVD with
anchor-party sign validation, centrist support fraction over pass rate,
acceptance-without-conversion test, opposition-only filter as coalition
control, extremity-stratified tolerance analysis, and LLM score auditing.

docs/solutions/best-practices/overton-window-shift-methodology-2026-05-24.md

@@ -0,0 +1,146 @@
+---
+title: Measuring Overton window shifts with parliamentary voting data
+date: 2026-05-24
+category: best-practices/
+module: analysis/right_wing
+problem_type: best_practice
+component: development_workflow
+severity: medium
+applies_when:
+  - "Analyzing longitudinal shifts in political acceptability of motions"
+  - "Testing whether centrist parties moved toward policy extremes"
+  - "Distinguishing coalition-driven voting from genuine ideological repositioning"
+tags:
+  - overton-window
+  - parliamentary-voting
+  - centrist-classification
+  - svd-drift
+  - acceptance-without-conversion
+  - dutch-parliament
+  - political-analysis
+  - extremity-scoring
+---
+
+# Measuring Overton Window Shifts with Parliamentary Voting Data
+
+## Context
+
+Dutch Tweede Kamer voting data over 2016–2026 provides a natural experiment for testing whether the political Overton window — the range of politically acceptable policy proposals — has shifted rightward. The central question: did centrist parties become more accepting of right-wing policy, and what objective indicators support or refute a genuine shift?
+
+Initial analysis efforts ran into three structural problems before converging on a reliable methodology. (session history)
+
+## Guidance
+
+### 1. Define Centrist Parties by Quantitative Position
+
+Do not assume party self-labels or media classifications. Define centrist parties by their SVD position on the primary left-right axis:
+
+- Compute SVD scores per party per window
+- A party qualifies as centrist only if its coordinate falls within ±0.5σ of the parliamentary median over the full period
+- In the Dutch context, this yields **D66, CDA, CU, NSC** as centrist
+- VVD and BBB are center-right, not centrist — including them inflated baseline centrist support by ~0.13 and diluted the Overton shift signal from d=0.85 to d=0.68 (session history)
+
+### 2. Align SVD Axes with Procrustes + Validate Sign via Anchor Parties
+
+Raw per-window SVD axes are independently computed and not comparable across years. The stability gate (Spearman correlation of party rankings) failed for 9/10 consecutive window pairs.
+
+**Correct approach:**
+- Use chained Procrustes alignment to the first chronological window
+- Zero-pad vectors to max dimension across all windows
+- Run global PCA on the stacked aligned matrix
+- Apply flip correction per axis using canonical right/left party centroids
+- **Critically:** after flip correction, axis sign may differ from raw SVD_THEMES labels. Always validate orientation against 2+ known-party anchors (e.g., PVV vs Volt on axis-2) before interpreting direction
+
+For the Dutch parliament after flip correction:
+- Axis 1 (economic): positive = pro-market, negative = welfare
+- Axis 2 (cultural): **negative = nationalist** (PVV at -0.56, FVD at -0.36), **positive = kosmopolitisch** (Volt at +0.27, GL-PvdA at +0.21) — this is the **opposite** of raw `SVD_THEMES[2]` labeling
+
+### 3. Replace Pass Rate with Centrist Support Fraction
+
+The Dutch Tweede Kamer passes 96%+ of all motions. Pass rate is structurally useless — no signal survives the ceiling effect.
+
+**Use instead:** `centrist_support = centrist_voor_votes / centrist_total_votes`
+
+In the party-disciplined Dutch context, computing this as fraction-of-parties vs fraction-of-MPs produces near-identical results (r = 0.998, mean difference < 0.001), but MP-level is preferred for international comparability.
+
+### 4. The Overton Test: Rising Support + Widening Ideological Distance
+
+The defining signature of an Overton window widening is **acceptance without conversion**:
+
+- Centrist voting support for right-wing motions rises (Cohen's d = +0.85 for opposition-only motions)
+- **But** SVD axes show centrists moved LEFT on both dimensions (economic: -0.223, cultural: +0.081 toward kosmopolitisch)
+- **And** the cultural distance between centrist and right-wing parties **widened** from 0.282 to 0.428 (+0.146)
+- The combined pattern rules out ideological conversion and confirms window widening: the range of acceptable policy expanded without centrist parties themselves moving right
+
+### 5. Opposition-Only Filter as Coalition Control
+
+Motions submitted by government parties after they enter coalition could reflect coalition discipline, not ideological shift. Filter to motions where the lead submitter party is NOT in the ruling coalition:
+
+- Parse submitter party from motion title prefixes
+- Hardcode coalition composition per year
+- If opposition-only centrist support also rises (as it did: d=+0.85, stronger than overall d=+0.65), the shift is not coalition-driven
+
+**Fragility note:** submitter parsing covers ~87–91% of motions. Missing 10%+ are bills, amendments, and motions with non-standard titles.
+
+### 6. Extremity-Stratified Centrist Support as Tolerance Test
+
+Bucket right-wing motions by LLM-scored extremity (1–2, 2–3, 3–4, 4–5). Compute centrist support per bucket for pre-2024 vs post-2024:
+
+- If high-extremity motions gained disproportionate support: centrists became more tolerant of extreme content
+- If all buckets rose uniformly: the shift is about quantity, not tolerance
+- If only mild motions rose: right-wing parties filed milder motions, shift is illusory
+
+The persistence of a gradient (centrists still differentiate by extremity post-shift) confirms centrist support is not random — they vote strategically, not blindly.
+
+### 7. Audit LLM Scores with 2D Rescoring
+
+LLM-based extremity scores conflate two distinct dimensions:
+- **Stylistic extremity:** inflammatory language, emotional charge
+- **Material impact:** substantive rights restriction, institutional change
+
+Manual audit of 5 motions per extremity bucket (20 total) found 75% agreement with systematic LLM biases: overrating anti-institutional language and migration-adjacent content. A flat content extremity trend (d=-0.09) may partially reflect these biases rather than genuine stability.
+
+**Mitigation:** deferred two-dimensional rescoring on a stratified sample to measure correlation. If r < 0.6, the dimensions are independent and dual scoring is required.
+
+## Why This Matters
+
+Without this methodology, Overton window analysis produces false negatives or ambiguous results:
+- Loose centrist definitions mask the signal (session history)
+- Pass rate creates a false negative (no detectable change at 96%+ ceiling)
+- Raw SVD mislabels axis direction, producing the wrong interpretation (centrists moving "right" when they actually moved left)
+- Missing opposition control conflates coalition discipline with ideological shift
+
+The "acceptance without conversion" finding has implications beyond Dutch politics: it demonstrates that policy acceptance can expand without ideological convergence, a pattern observable in any party-disciplined legislature with longitudinal voting data.
+
+## When to Apply
+
+- When analyzing longitudinal shifts in policy acceptability from voting records
+- When centrist parties show increased voting alignment with fringe parties
+- When distinguishing between coalition effects and genuine ideological repositioning
+- When working with per-window SVD/PCA embeddings that need cross-temporal comparison
+- When using LLM-scored extremity metrics as independent variables — audit first
+
+## Examples
+
+**Before (what didn't work):**
+- Raw per-window SVD positions compared without alignment → 9/10 stability failures, uninterpretable
+- Pass rate tracked as primary Overton metric → flatline at 96%+, no signal
+- Loose centrist definition (VVD, D66, CDA, NSC, BBB, CU) → inflated baseline, diluted d=0.68
+- Static SVD_THEMES pole labels assumed without verifying against anchor parties → wrong axis direction
+
+**After (what works):**
+- Procrustes-aligned PCA with anchor-party sign validation → interpretable cross-window drift
+- Centrist support fraction as primary metric → clear pre/post shift (d=+0.65 strict, d=+0.85 opposition-only)
+- Strict centrist definition (D66, CDA, CU, NSC) → unmasked the full effect size
+- Runtime flip correction produces correct axis convention → centrists moved LEFT on both axes, diverging from right
+
+## Related
+
+- `docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md` — foundational: SVD captures voting patterns, not semantic content
+- `docs/solutions/ui-bugs/svd-axis-pole-labels-incorrect-after-flip.md` — sign convention discovery via runtime flip
+- `docs/solutions/insights/quantifying-political-extremity.md` — voting extremity vs policy extremity are independent
+- `docs/solutions/insights/svd-stability-vs-overtone-shift.md` — SVD stability and Overton shift are independent phenomena
+- `docs/solutions/logic-errors/svd-component-labels-mismatch.md` — initial semantic-vs-voting labeling confusion
+- `analysis/right_wing/overton_breakpoint_analysis.py` — reference implementation
+- `analysis/right_wing/overton_svd_drift.py` — Procrustes-aligned drift computation
+- `analysis/right_wing/classify_motions.py` — centrist_support computation logic