sgeboers
/
motief
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
287 Commits
1 Branch
0 Tags
28 MiB
 
 
Tag: Branch: Tree: 0183bbc8a3
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0183bbc8a3'
${ noResults }
motief/docs/plans/2026-05-08-002-feat-overton...

21 KiB
Raw Blame History

title type status date
Quantify the Overton Window Shift in Dutch Parliament feat active 2026-05-08

Supersedes: docs/plans/2026-05-08-001-feat-visualize-migration-antidemocratic-plan.md

Quantify the Overton Window Shift in Dutch Parliament

Summary

Test the hypothesis that the Overton window has shifted rightward in the Tweede Kamer by analyzing three objective indicators: (1) centrist support for and passage of right-wing motions, with a focus on opposition motions to control for coalition effects, (2) whether content extremity of proposed motions increased over time, and (3) rightward drift of the parliamentary center via SVD positions. The analysis is deliberately descriptive — we report effect sizes and visual patterns, not statistical confirmation from an 8-point time series.

Problem Frame

Direction 3 analysis revealed a dramatic structural break in 2024: centrist support for right-wing motions nearly doubled (0.33 → 0.67), and pass rates jumped from ~33% to ~70%. This is consistent with an Overton window shift — but it could also be a coalition artifact (NSC/VVD supporting PVV government proposals) rather than a genuine ideological repositioning. We need rigorous, multi-indicator evidence to distinguish these explanations.

The existing mechanism-coding plan answers "what kinds of anti-democratic mechanisms exist in extreme motions" — a different question. This plan answers the user's actual question: "Has the Overton window shifted, and what objective indications do we have?"

Requirements

  • R1. Quantify the 2024 breakpoint in centrist support and pass rates with effect sizes, not overclaimed statistical significance.
  • R2. Test whether the shift persists when controlling for coalition status — i.e., do opposition right-wing motions also gain centrist support post-2024?
  • R3. Test whether the shift is domain-specific to migration or broader across right-wing motions.
  • R4. Quantify whether the content extremity of proposed right-wing motions increased over time.
  • R5. Quantify whether the parliamentary center shifted rightward on SVD axes, with axis stability validated before cross-window comparison.
  • R6. A concise report with 3 charts and an honest, uncertainty-aware conclusion.

Scope Boundaries

  • In scope: Descriptive quantitative analysis of vote data, SVD positions, content extremity trends, and coalition status.
  • Out of scope: LLM-based mechanism coding, sentiment analysis refinements, keyword penetration analysis (see Key Technical Decisions), anti-democratic scoring pipeline.
  • No Streamlit UI changes — this is a research artifact.
  • No changes to the classification pipeline or agent_tools.

Deferred to Follow-Up Work

  • Full anti-democratic scoring pipeline (Direction 2)
  • Interactive dashboard for Overton window monitoring
  • Causal inference modeling (diff-in-diff, regression discontinuity) — requires more data or a valid control group
  • Longitudinal keyword adoption study with independently-derived keyword sets (not the differential TF-IDF set from right-wing motions)
  • Two-dimensional extremity validation: Score a stratified sample (50–100 motions) for both stylistic extremity (inflammatory language, emotional charge) and material impact (rights restricted, groups affected, enforcement mechanisms). If correlation is low (r < 0.6), re-score all 2,986 motions with a refined dual-dimension prompt so extremity-stratified analyses can separate "harsh phrasing" from "substantive policy impact." If correlation is high (r > 0.7), the single score is sufficient.

Context & Research

Relevant Code and Patterns

  • analysis/right_wing/direction3_migration_antidemocratic.py — Direction 3 analysis script
  • analysis/explorer_data.py — load_party_scores_all_windows(), get_available_windows(), SVD position computation
  • analysis/right_wing/classify_motions.py — keyword matching logic for right-wing classification
  • analysis/config.py — CANONICAL_RIGHT, CANONICAL_LEFT, CANONICAL_CENTRIST, PARTY_COLOURS
  • scripts/motion_drift.py — matplotlib chart patterns

Data Assets

  • right_wing_motions — classified motions with centrist_support, right_support, left_opposition
  • extremity_scores — LLM-scored extremity per motion
  • mp_votes — individual MP votes per motion (for pass/fail and party breakdown)
  • motions — motion metadata including date, title, voting_results (JSON)
  • svd_vectors — SVD embeddings per MP/party per window
  • party_results — session-level party agreement data

Institutional Learnings

  • docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md — SVD positions reflect voting patterns, not semantic content. This is a feature: if the center shifts on SVD axes, it's because voting behavior changed, not because words changed.

Key Technical Decisions

  • No LLM batching: All indicators are computable from existing structured data. This keeps the analysis deterministic, cheap, and reproducible.
  • Descriptive statistics only: With only 8–11 annual observations (2016–2026), statistical tests like Mann-Kendall are underpowered. We report Cohen's d, pre/post differences, and visual patterns — not p-values claimed as "confirmation."
  • Opposition-only filter as coalition control: Instead of comparing government vs. opposition (which conflates the 2024 coalition change with the Overton shift), we compare opposition right-wing motions pre- and post-2024. If opposition motions also gain centrist support, the shift is not purely coalition-driven.
  • Keyword penetration analysis DROPPED: The right-wing keywords were derived differentially from right-wing motion titles. Finding these keywords in centrist motions does not demonstrate ideological adoption — it demonstrates topic salience. This would require an independently-derived keyword set (e.g., from party manifestos or media corpora). Deferred.
  • SVD cross-window comparison requires stability validation: Axes are re-oriented per window (see political_axis.py). Before comparing centrist positions across years, we validate that the same parties occupy roughly the same relative positions. If instability exceeds a threshold, we report the drift as uninterpretable.
  • Coalition status inferred from submitter party: Use motion title prefix to identify lead MP, map to party via mp_metadata, then tag as "government" if the party is in the governing coalition for that year. Coalition composition is hardcoded per year.

Open Questions

Resolved During Planning

  • Q: How to define "centrist"? Use CANONICAL_CENTRIST = frozenset({"VVD", "D66", "CDA", "NSC", "BBB", "CU"}) from classify_motions.py.
  • Q: How to define "government" vs. "opposition"? Hardcode coalition composition per year based on historical Dutch governments.
  • Q: Why drop keyword penetration? Circular: keywords were derived from right-wing motions; their appearance in centrist motions measures topic prevalence, not ideological adoption.
  • Q: What statistical test? None. With 8–11 points, we report descriptive effect sizes (Cohen's d) and let charts carry the evidence.

Deferred to Implementation

  • Q: Exact coalition composition for 2016–2023 — verify against Wikipedia or Binnenhof records at implementation time.
  • Q: Whether to use quarterly or annual SVD windows — start with annual for simplicity; quarterly if annual resolution is too coarse.
  • Q: Threshold for SVD axis stability — define at implementation (e.g., Spearman correlation of party rankings across windows ≥ 0.8).

Implementation Units

U1. Commit Direction 3 Script

Goal: Preserve the working Direction 3 analysis.

Requirements: R1 (context)

Dependencies: None

Files:

  • Stage/Commit: analysis/right_wing/direction3_migration_antidemocratic.py

Approach: Stage and commit the existing script with a conventional message.

Verification: Script runs without errors.

U2. Descriptive Breakpoint & Opposition-Controlled Analysis

Goal: Quantify the 2024 breakpoint in centrist support, pass rates, and content extremity — with coalition effects controlled via opposition-only filtering, domain decomposition, and a baseline comparison.

Requirements: R1, R2, R3, R4

Dependencies: U1

Files:

  • Create: analysis/right_wing/overton_breakpoint_analysis.py
  • Output: reports/overton_window/breakpoint_analysis.md
  • Output: reports/overton_window/breakpoint_figure_1.png (centrist support + pass rate)
  • Output: reports/overton_window/breakpoint_figure_2.png (extremity + motion count)

Prerequisites:

  • right_wing_motions table must have category column (run derive_categories.py if missing).
  • reports/overton_window/ directory must exist (create if missing).

Approach:

  1. Yearly aggregates (all right-wing motions):

    • Mean centrist_support, mean right_support, mean left_opposition
    • Pass rate (computed from mp_votes or voting_results)
    • Mean extremity_score (from extremity_scores)
    • Count of motions

  2. Baseline comparison (all motions):

    • For each year, compute pass rate across all motions (not just right-wing classified ones)
    • For each year, compute mean centrist support across all motions (% of centrist parties voting "voor")
    • These serve as dashed reference lines to test whether the 2024 shift is specific to right-wing content or a general coalition effect

  3. Pre/post 2024 comparison:

    • Compute difference-in-means for centrist support, pass rate, and extremity
    • Report Cohen's d as effect size
    • Do NOT claim statistical significance — frame as "effect size" only

  4. Opposition-only analysis (coalition control):

    • Hardcode coalition composition per year (see Key Technical Decisions)
    • For each right-wing motion, identify the lead submitter party from the title prefix (or mp_votes majority submitter)
    • Filter to motions where the submitter party is NOT in the ruling coalition
    • Recompute yearly centrist support and pass rate for this opposition-only subset
    • Compare pre-2024 vs post-2024 opposition metrics
    • Interpretation gate: If opposition metrics also rise post-2024, the shift is not purely coalition-driven. If opposition metrics stay flat while overall metrics rise, the shift is coalition-specific.

  5. Domain decomposition (migration vs. non-migration):

    • Split right-wing motions into migration (category = 'asiel/vreemdelingen') and non-migration
    • Compute yearly centrist support, pass rate, and extremity for each
    • Test: is the 2024 shift driven entirely by migration, or is it broader?

  6. Extremity-stratified pass rate (tolerance shift test):

    • Bucket right-wing motions by extremity score: 1–2 (mild), 2–3 (moderate), 3–4 (high), 4–5 (extreme)
    • Compute pass rate per bucket for pre-2024 (2018–2023) and post-2024 (2024–2025)
    • Key test: If high-extremity motions (3–5) went from low pass rate → high pass rate while mild motions stayed flat, centrists are more tolerant of extreme content — direct Overton shift evidence
    • If pass rate rose uniformly across all buckets, the shift is about quantity, not tolerance
    • If only the 1–2 bucket rose, right-wing parties filed milder motions post-2024 and the "shift" is illusory

  7. Visualization (two figures):

    • Figure 1 (2 panels) — core evidence:
      • Panel A: Centrist support over time (all right-wing, opposition-only, migration, non-migration) + dashed baseline (all motions)
      • Panel B: Pass rate over time (same breakdown) + dashed baseline (all motions)
    • Figure 2 (2 panels) — supplementary:
      • Panel C: Mean extremity over time (all right-wing, opposition-only, migration, non-migration)
      • Panel D: Extremity-stratified pass rate — grouped bars showing pass rate per extremity bucket for pre-2024 vs post-2024, with N labels per bar
    • Annotate 2024 with a vertical line on panels A–C

  8. Manual extremity audit (LLM score validation):

    • After computing results, sample 5 motions per extremity bucket (20 total) using stratified random sampling
    • For each sampled motion, display: title, category, LLM extremity score, and a short excerpt from the body text (first 300 chars)
    • As the analyst, read each motion and judge: "Would I assign this to the same extremity bucket (1–2 / 2–3 / 3–4 / 4–5)?" Record agreement per motion
    • Language impact divergence check: For each motion, note whether the LLM score appears driven by stylistic extremity (inflammatory phrasing) or material impact (substantive rights restriction, institutional change). Flag motions where the two diverge (e.g., mild language but severe policy impact, or inflammatory language but trivial mechanism)
    • Report overall agreement rate and note any systematic LLM bias (e.g., "LLM overrates anti-institutional language" or "LLM underrates economic nationalism")
    • If agreement < 70%, flag LLM scoring as unreliable for the stratified analysis and note in the report; still present results but with stronger caveats

  9. Report: One markdown document with sections for each analysis, the manual audit table, embedded charts, and an honest uncertainty statement. Include N per extremity bucket so reader can gauge reliability.

Patterns to follow:

  • scripts/motion_drift.py — matplotlib multi-panel chart saving
  • analysis/right_wing/temporal_analysis.py — yearly aggregation patterns
  • analysis/right_wing/classify_motions.py — title parsing for MP names and party mapping

Test scenarios:

  • Happy path: script produces a markdown report with all analyses, manual audit table, and two charts.
  • Edge case: missing data for a year → skip that year with a warning.
  • Edge case: no opposition motions in a year → report NaN with a note.
  • Edge case: coalition composition ambiguous for a year → document assumption and continue.
  • Edge case: parsed submitter party has 0 "voor" votes in voting_results → flag as anomalous and skip for coalition tagging.
  • Edge case: <5 motions in an extremity bucket → sample what's available and note reduced sample size.
  • Edge case: manual audit agreement < 70% → flag LLM scoring reliability in the report.

Verification:

  • Report contains pre/post difference, Cohen's d, opposition-only comparison, baseline comparison, and extremity-stratified pass rates.
  • Manual audit table shows agreement rate ≥ 70% (or unreliable flag set).
  • Charts clearly show the 2024 break, the baseline reference, and pre/post pass rates by extremity bucket.
  • Report includes an explicit statement about statistical limitations (small-N time series).

U3. SVD Center Drift with Stability Validation

Goal: Quantify whether the parliamentary center shifted rightward on the political compass, but only if axes are stable enough for cross-window comparison.

Requirements: R5

Dependencies: U2

Files:

  • Create: analysis/right_wing/overton_svd_drift.py
  • Create table: overton_svd_center (window_id, centrist_mean_axis1, centrist_mean_axis2, right_mean_axis1, right_mean_axis2, stability_score)
  • Output: reports/overton_window/svd_drift_chart.png
  • Output: reports/overton_window/svd_stability_report.md

Approach:

  1. Load SVD vectors from svd_vectors table for all annual windows.

  2. Compute mean position per party per window using compute_party_positions_from_vectors logic.

  3. Axis stability validation (GATE):

    • For each pair of consecutive windows, compute Spearman correlation of party rankings on axis 1 and axis 2
    • If correlation < 0.7 for either axis, flag the window pair as unstable
    • If >2 unstable pairs exist, ABORT the cross-window comparison and report: "SVD axes are too unstable for longitudinal comparison. Positions may reflect re-orientation rather than genuine drift."
    • If stable, proceed.

  4. Calculate the "center of gravity":

    • Mean of centrist parties' axis-1 and axis-2 positions per window
    • Also compute mean of right-wing parties' positions as a reference point
    • Store in overton_svd_center

  5. Plot:

    • Centrist center trajectory over time on the 2D compass
    • Right-wing center trajectory as a dashed reference line
    • Annual points labeled with year

  6. Compute drift metrics:

    • Euclidean distance between consecutive annual centrist centers (drift speed)
    • Net displacement from first to last window
    • Angular direction of drift (is it toward the right-wing cluster?)

  7. Report: Markdown with stability validation results, drift metrics, and a clear statement about whether the drift is interpretable.

Patterns to follow:

  • analysis/explorer_data.py — load_party_scores_all_windows_aligned(), compute_party_positions_from_vectors
  • explorer.py — compass plotting with PARTY_COLOURS

Test scenarios:

  • Happy path: table has one row per annual window with non-null centrist means and stability_score.
  • Edge case: missing party in a window → compute mean over available centrist parties.
  • Edge case: axis instability detected → script reports instability and skips drift interpretation.

Verification:

  • overton_svd_center table has ≥8 rows (one per annual window).
  • Stability validation is documented in the report.
  • If stable, chart shows a clear centrist trajectory; 2024 position is visually rightward of 2020–2023.
  • If unstable, report explicitly states that cross-window comparison is uninterpretable.

U4. Compile Findings Report

Goal: Synthesize all indicators into a concise, publishable narrative with honest uncertainty framing.

Requirements: R6

Dependencies: U2, U3

Files:

  • Create: reports/overton_window/findings_report.md

Approach:

  1. Structure the report around the three indicators:

    1. Breakpoint & Coalition Control: Did centrist support and pass rates break in 2024? Does the effect persist for opposition motions? (U2)
    2. Content Extremity Trend: Did the extremity of proposed motions increase over time? How valid are the LLM scores? (U2 audit)
    3. Spatial Drift: Did the parliamentary center move right on SVD axes? (U3)
    4. Domain Specificity: Is the shift migration-specific or broader? (U2)
    5. Tolerance Shift: Did centrists become more accepting of high-extremity content specifically? (U2 extremity-stratified)

  2. Include embedded charts from U2 (two figures) and U3.

  3. Conclusion with uncertainty hierarchy:

    • Strong evidence: multiple indicators converge (e.g., opposition support rises + high-extremity pass rate rises + SVD drifts right + content extremity increases)
    • Mixed evidence: some indicators shift, others don't
    • Weak/unclear evidence: indicators conflict or data is too sparse
    • Explicitly state that the Overton window is a theoretical construct; we measure observable correlates, not the construct itself.

  4. Cap at ~1,500 words. Let charts and tables carry the data.

  5. Include a "Limitations" section:

    • Small-N time series (8–11 years)
    • LLM extremity scores are content-based, not independently validated beyond the manual audit (step 8)
    • SVD axis stability caveat
    • Coalition composition uncertainty
    • Submitter party identification is parsed from motion titles and may be inaccurate for multi-submitter motions
    • Keyword penetration not analyzed (circular)

Patterns to follow:

  • reports/drift/report.md — existing report format

Verification:

  • Report is readable, all chart references (2 figures from U2, 1 from U3) resolve, conclusion is grounded in the indicators.
  • Report includes an explicit limitations section.
  • Report does not claim statistical "proof" or "confirmation."

System-Wide Impact

  • New table: overton_svd_center is additive; no existing tables modified.
  • No UI changes: Charts are saved as PNGs; report is markdown.
  • No agent_tools changes: Analysis scripts are standalone.
  • Reproducibility: All indicators are deterministic (no LLM calls). Rerunning produces identical results.

Risks & Dependencies

Risk Mitigation
Coalition composition is wrong for a year Document assumptions; if uncertain, note sensitivity in report.
SVD positions are noisy or unstable Stability validation gate (U3); if unstable, report explicitly says so.
Report overclaims causality Frame all findings as "indicators" or "consistent with," not proof. The Overton window is a theoretical construct; we measure observable correlates.
Opposition-only subset is too small post-2024 Report N for each comparison; if N<10, flag as unreliable.
Submitter party misidentified from title Use voting_results as sanity check (flag if parsed party has 0 "voor" votes); document as known limitation.
Small-N time series invites spurious pattern-matching Explicit limitations section; no statistical tests presented as confirmatory.

Documentation / Operational Notes

  • The report is intended for internal research consumption and potential academic/public communication.
  • All code is standalone analysis scripts; no operational deployment.
  • overton_svd_center table can be reused for future longitudinal analyses if SVD axes are validated as stable.

Sources & References

  • Origin analysis: analysis/right_wing/direction3_migration_antidemocratic.py
  • SVD computation: analysis/explorer_data.py
  • Classification logic: analysis/right_wing/classify_motions.py
  • Keywords: analysis/right_wing/right_wing_keywords.json
  • Chart patterns: scripts/motion_drift.py