You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
220 lines
10 KiB
220 lines
10 KiB
# Overton Window Analysis — Status
|
|
|
|
**Last updated:** 2026-06-07
|
|
**Active plan:** `docs/plans/2026-06-06-001-overton-coherent-narrative-plan.md`
|
|
**Working branch:** `feat/right-wing-motion-analysis`
|
|
|
|
---
|
|
|
|
## Context
|
|
|
|
The Overton window analysis is a flagship output of Stemwijzer's **Track 2: Analytical Depth and Transparency** (see `STRATEGY.md`). Stemwijzer is a Dutch parliamentary analysis platform with three tracks:
|
|
|
|
1. **Data pipeline reliability** — robust ingestion of all Tweede Kamer votes
|
|
2. **Analytical depth and transparency** — interpretable political dimensions (this analysis)
|
|
3. **Agent-native architecture** — self-documenting, agent-operable codebase
|
|
|
|
The Overton analysis demonstrates what the platform can do: SVD compass, 29K+ scored motions, 2D extremity scoring, and Procrustes-aligned drift detection — all in service of a real political science question.
|
|
|
|
## Goal
|
|
|
|
A coherent, multi-surface story about whether the Dutch Overton window shifted — accessible as an interactive Quarto article, live in the Streamlit Explorer, and backed by reproducible analysis scripts. The narrative serves dual purpose: a political science finding AND a platform showcase that drives engagement with the Stemwijzer compass and explorer.
|
|
|
|
Three tiers:
|
|
|
|
1. **Narrative spine** — Quarto article (the story, with "About Stemwijzer" section)
|
|
2. **Detailed appendices** — Markdown reports in `reports/overton_window/` (the evidence)
|
|
3. **Live exploration** — Streamlit Explorer Overton tab + existing Kompas/Trajectories tabs (the data)
|
|
|
|
---
|
|
|
|
## Completed
|
|
|
|
### Core Analysis (U1-U5 from plan 001)
|
|
- [x] Right-wing motion classification (2,986 → 3,030 classified)
|
|
- [x] 1D extremity scoring (LLM, 2,986 motions)
|
|
- [x] Sentiment analysis (LLM, 2,986 motions)
|
|
- [x] Category derivation (7+13 categories)
|
|
- [x] Temporal aggregation (yearly trends, 2016-2026)
|
|
|
|
### Overton Window Analysis (plans 002-003)
|
|
- [x] Centrist support breakpoint (strict 4-party: D66/CDA/CU/NSC)
|
|
- [x] Opposition-only filtering (coalition control)
|
|
- [x] Domain decomposition (migration vs non-migration)
|
|
- [x] SVD spatial drift (Procrustes-aligned PCA)
|
|
- [x] Content extremity trends (material impact declined, style rose)
|
|
- [x] "Acceptance without conversion" confirmed
|
|
- [x] Findings report written
|
|
|
|
### 2D Extremity Scoring (plan 004)
|
|
- [x] Project-local skill: `.opencode/skills/score-extremity/SKILL.md`
|
|
- [x] 2D scoring (stijl-extremiteit + materiele impact, 1-5)
|
|
- [x] Pearson r = 0.47 (right-wing), r = 0.43 (all-motion) — dimensions separable
|
|
- [x] All 29,591 motions scored via subagent pipeline
|
|
- [x] 2D temporal decomposition (material fell, style rose — divergence confirmed)
|
|
- [x] Gravity-controlled analysis (M≥4 centrist support shifted +0.263)
|
|
|
|
### Gap Analysis & Extensions (plans 005-006)
|
|
- [x] Quarterly temporal trajectory (33 quarters, inflection at 2024-Q2)
|
|
- [x] Causal timing (electoral jump, not coalition-driven)
|
|
- [x] Left-wing response (18.3× asymmetry, Volt exception)
|
|
- [x] Mechanism classification (consensus framing confirmed, κ=0.41 moderate)
|
|
- [x] Party differentiation (JA21 drives moderation, PVV entered government)
|
|
- [x] Voting margin analysis (ρ=0.812, far superior to pass rate)
|
|
- [x] Predictive model (LR CV AUC-ROC=0.816, RF CV AUC-ROC=0.845, RF test=0.805)
|
|
- [x] Coalition coding fix (2024 split at July 1)
|
|
- [x] All-motion 2D extremity (29,591 motions, stijl=1.36, mat=2.12)
|
|
- [x] HTML report with gravity-controlled charts + example motions
|
|
|
|
### Code Quality
|
|
- [x] Shared helpers extracted to `analysis/right_wing/common.py`
|
|
- [x] requests.Timeout bug fixed
|
|
- [x] p-value walrus operator fixed
|
|
- [x] 35 tests for common.py (TDD)
|
|
- [x] DROP TABLE bug fixed in classify_motions.py
|
|
|
|
### Knowledge Capture
|
|
- [x] Overton methodology documented (7-step, `docs/solutions/best-practices/`)
|
|
- [x] Domain decomposition methodology documented
|
|
- [x] Extended analysis methodology documented
|
|
- [x] Large-scale subagent scoring methodology documented
|
|
- [x] Narrative architecture documented
|
|
|
|
---
|
|
|
|
## In Progress
|
|
|
|
### Plan 007: Coherent Narrative (current)
|
|
- [ ] U1: Clean up stale reports (remove findings_report.md, blog_post.html)
|
|
- [ ] U1: Fix hashline corruption in synthesis report
|
|
- [ ] U1: Add cross-reference headers to all reports
|
|
- [ ] U1: Switch HTML report to strict 4-party centrist definition
|
|
- [ ] U2: Install Quarto CLI
|
|
- [ ] U2: Write Quarto narrative spine (8 sections, interactive Plotly)
|
|
- [ ] U3: Add Overton context panel to Explorer Kompas tab
|
|
- [ ] U3: Add 2024 breakpoint annotation to Trajectories tab
|
|
- [ ] U3: Create new Overton tab (centrist support trend, right-wing motion browser)
|
|
- [ ] U3: Wire Overton tab into Explorer
|
|
- [ ] U4: Write `build_all_reports.py` master script
|
|
- [ ] U5: Write `reports/overton_window/README.md` reading guide
|
|
- [ ] U5: Update project README.md
|
|
|
|
---
|
|
|
|
## Deferred
|
|
|
|
### Analysis Depth
|
|
- [ ] European comparison (AfD, Meloni, Le Pen, Sweden Democrats)
|
|
- [ ] Mechanism taxonomy revision (κ=0.41 → improve agreement)
|
|
- [ ] Forward-looking scenario analysis (permanent vs temporary shift)
|
|
- [ ] Anti-institutional pivot deep-dive (abolition → contestation)
|
|
- [x] Re-populate category column in right_wing_motions (wiped by DROP TABLE)
|
|
|
|
### Presentation
|
|
- [ ] Quarto blog post with interactive charts
|
|
- [ ] Table of contents / reading guide linking all 17 reports
|
|
- [ ] Single-script reproducible build with Quarto render
|
|
|
|
### Infrastructure
|
|
- [ ] Agent-native architecture improvements (pipeline_run_stage, UI integration)
|
|
- [ ] CRUD completeness (delete_motion, mop up entity gaps)
|
|
|
|
---
|
|
|
|
## Report Directory Map
|
|
|
|
```
|
|
reports/overton_window/
|
|
├── STATUS.md ← THIS FILE
|
|
├── README.md ← Reading guide (U5, pending)
|
|
├── overton_window.qmd ← Narrative spine (U2, pending)
|
|
├── _quarto.yml ← Quarto config (U2, pending)
|
|
│
|
|
├── overton_window_synthesis.md ★ Master synthesis (291 lines)
|
|
├── overton_report.html ★ Public HTML dashboard
|
|
│
|
|
├── breakpoint_analysis.md Appendix: Centrist support breakpoint
|
|
├── breakpoint_figure_1.png Fig: Centrist support over time
|
|
├── breakpoint_figure_2.png Fig: Extremity-stratified
|
|
├── breakpoint_figure_3.png Fig: Left-wing support
|
|
├── breakpoint_figure_4.png Fig: Gravity-controlled CS
|
|
│
|
|
├── extremity_2d_temporal.md Appendix: 2D extremity temporal
|
|
├── extremity_2d_temporal_figure.png Fig: 4-panel 2D temporal
|
|
│
|
|
├── temporal_trajectory.md Appendix: Quarterly trajectory
|
|
├── temporal_trajectory_figure.png Fig: 33-quarter trajectory
|
|
│
|
|
├── causal_timing.md Appendix: Causal attribution
|
|
├── causal_timing_figure.png Fig: Pre/post event timing
|
|
│
|
|
├── svd_stability_report.md Appendix: Procrustes SVD drift
|
|
├── svd_drift_chart.png Fig: 2D party compass
|
|
├── svd_trajectory_figure.png Fig: Party trajectories
|
|
│
|
|
├── mechanism_classification.md Appendix: Why motions gain support
|
|
├── mechanism_validation.md Appendix: κ=0.41 validation
|
|
│
|
|
├── party_differentiation.md Appendix: Per-party shifts
|
|
├── party_differentiation_figure.png Fig: JA21/FVD/PVV/SGP comparison
|
|
│
|
|
├── left_wing_response.md Appendix: Left-wing voting
|
|
├── left_wing_response_figure.png Fig: Left party support
|
|
│
|
|
├── voting_margin.md Appendix: Voting margin analysis
|
|
├── voting_margin_figure.png Fig: Margin distribution
|
|
│
|
|
├── predictive_model.md Appendix: ML prediction
|
|
├── predictive_model_figure.png Fig: Feature importance
|
|
│
|
|
├── success_correlation.md Appendix: Pass rate (ceiling)
|
|
│
|
|
├── 2d_extremity_correlation_report.md Appendix: Full 29,591 correlation
|
|
│
|
|
├── findings_report.md ✗ REMOVED (superseded)
|
|
├── blog_post.html ✗ REMOVED (replaced by Quarto)
|
|
```
|
|
|
|
---
|
|
|
|
## Data Sources
|
|
|
|
| Table | Rows | Purpose |
|
|
|-------|------|---------|
|
|
| `right_wing_motions` | 29,588 | Classified right-wing motions with centrist support metrics |
|
|
| `extremity_scores` | 2,986 | Original 1D LLM scores (legacy) |
|
|
| `extremity_scores_2d` | 3,089 | 2D scores for right-wing motions (active) |
|
|
| `extremity_scores_all` | 29,591 | 2D scores for ALL motions (baseline) |
|
|
| `sentiment_scores` | 2,986 | Dutch sentiment scores (legacy) |
|
|
| `motions` | 29,570+ | Main motions table |
|
|
| `mp_votes` | — | Per-MP per-motion vote records |
|
|
| `party_axis_scores` | — | Procrustes-aligned PCA party positions |
|
|
| `overton_svd_center` | 11 | Yearly SVD centrist/right-wing centers |
|
|
|
|
---
|
|
|
|
## Key Numbers (canonical)
|
|
|
|
| Metric | Pre-2024 | Post-2024 | Δ |
|
|
|--------|----------|-----------|---|
|
|
| Centrist support (strict 4-party) | 0.251 | 0.507 | +0.256 |
|
|
| Opposition-only CS | 0.270 | 0.543 | +0.272 |
|
|
| Material impact (right-wing) | 2.79 | 2.45 | −0.34 |
|
|
| M≥4 share (% high-impact) | 23.7% | 11.3% | −12.4 pp |
|
|
| SVD cultural gap | 0.282 | 0.428 | +0.146 |
|
|
| Stylistic extremity | 1.875 | 1.744 | −0.131 |
|
|
| Migration CS (strict 4-party) | 0.134 | 0.342 | +0.208 |
|
|
| All-motion stijl | — | 1.36 | — |
|
|
| All-motion materieel | — | 2.12 | — |
|
|
| Stijl-materieel r (RW) | — | 0.47 | — |
|
|
| Stijl-materieel r (all) | — | 0.43 | — |
|
|
| Voting margin ρ | — | 0.812 | — |
|
|
| Mechanism κ | — | 0.41 | — |
|
|
| AUC-ROC (logistic) | — | 0.81 | — |
|
|
| Inflection quarter | — | 2024-Q2 | — |
|
|
| Peak centrist support | — | 0.648 (2024-Q4) | — |
|
|
| Latest (2026-Q1) | — | 0.334 | — |
|
|
|
|
## Verdict
|
|
|
|
**The Overton window widened: more right-wing positions became politically acceptable after 2024. Centrist support for right-wing motions surged (25%→51%) while staying flat for left-wing motions (49%→49%). The mechanism was right-wing moderation, not centrist conversion — and the effect may be temporary (2026-Q1 reversion to 33%).**
|
|
|