Commit Graph
15 Commits
Author SHA1 Message Date
sgeboers 7df961ba83 feat(overton): address 7 critical gaps in Overton window analysis
U1: Temporal trajectory — quarterly granularity reveals immediate
electoral jump at 2024-Q1 (+0.180), peak at 2024-Q4 (0.648), reversion
to 0.334 by 2026-Q1.

U2: 2D extremity temporal — single-score masks divergence. Material
impact decreased (-0.146) while stylistic increased (+0.097).
Wilcoxon p=0.002 confirms systematic divergence.

U3: Systematic mechanism classification — 150 motions. Consensus
framing confirmed (24% high-CS vs 8% low-CS, p=0.014). Post-2024
high-CS dominated by procedural (32%), consensus (24%), targeted
restriction (17%).

U4: Causal timing — shift is electorally driven (after Nov 2023 PVV
election, before Jul 2024 Schoof cabinet). Rules out coalition
dynamics, gradual learning, European contagion.

U5: Left-wing response — barely changed (21.3%→20.2%, -1.1pp).
Centrist shift (d=+1.89) is 18.3x larger than left hardening
(d=-0.75). Volt is only left party that softened (+12.9pp).

U6: Success correlation — significant trend (p<0.001) but success
premium only +3.2%, ceiling effect at 96%+ limits practical meaning.

U7: Synthesis update — integrated all findings, updated verdict
to note electoral-cycle effect and 2026-Q1 reversion.
2026-05-26 23:33:13 +02:00
sgeboers 80c68c0112 fix(right-wing): match store_scores column names and value order to DB schema 2026-05-25 00:01:31 +02:00
sgeboers bf37f84a8b feat(extremity): two-dimensional rescoring with subagent pipeline
- Project-local skill .opencode/skills/score-extremity/ for subagent dispatch
- Orchestrator extremity_rescore_2d.py with load_skill/sample/format/validate/store
- 16 TDD tests covering all orchestrator functions
- 117 motions scored by deepseek v4 flash subagents (12 parallel batches)
- Pearson r=0.45 between stylistic and material dimensions — separable
- Key finding: 36.8% of motions use restrained language for consequential policies
- 2d_extremity_correlation_report.md documents distribution, divergence patterns,
  and implications for the Overton acceptance-without-conversion narrative
2026-05-24 23:13:42 +02:00
sgeboers be007165b1 fix(right-wing): add resume support to extremity and sentiment scorers
Use CREATE TABLE IF NOT EXISTS and skip already-scored motions
to allow resuming interrupted batch runs.
2026-05-24 22:33:44 +02:00
sgeboers 711a410df3 chore: simplify Overton scripts, update README, add stemwijzer.db to gitignore
- Extracted EXTREMITY_BUCKET_ORDER constant and _extremity_bucket() helper (4 duplications removed)
- Merged two-pass query loop in compute_yearly_baseline into single pass
- Removed unused import (mticker), dead code (year_titles_map), 12 obvious comments
- Extracted _fmt_axis() helper in SVD drift script
- Updated README analysis/ description to include right-wing motion analysis
2026-05-24 22:19:21 +02:00
sgeboers 2a081ade25 fix(overton): strict centrist definition + left support analysis
- Reclassified centrist to {D66, CDA, CU, NSC} — removing VVD/BBB
  which are center-right coalition partners
- Added centrist_support_strict (0.251→0.507, d=+0.65), center_right_support,
  and left_support_mp columns via migration script
- Figure 1 now shows center-right (VVD/BBB) support as orange dashed line
- New Figure 3: bar chart of left-party support for right-wing motions
  (0.268→0.202, left opposition hardened)
- New report Section 6 covering left-wing support trends
- All analysis now uses strict centrist definition throughout
2026-05-09 00:45:38 +02:00
sgeboers e478235c84 fix(overton): correct SVD axis interpretation, drop pass rate, synthesis rewrite
- SVD axis 2 sign corrected: negative = nationalist (PVV -0.56, FVD -0.36), positive = kosmopolitisch (Volt +0.27). Centrists moved LEFT on both axes while right-wing moved further right culturally (+0.146 gap). 'Acceptance without conversion' named as unifying interpretation.
- U1: Figure 1 merged to single panel, pass rate removed, 5 centrist_support lines
- U2: Pass rate columns dropped from all breakpoint tables, PR narrative cut
- U3: Findings report rewritten: SVD section replaced, synthesis restructured into 3 tiers, extremity LLM bias qualified
- U4: Axis labels and sign convention added to svd_stability_report.md
- Added centrist_support_mp column (MP-weighted, correlates 0.998 with party-level)
2026-05-09 00:21:46 +02:00
sgeboers 76b499cdc0 feat(analysis): Overton window breakpoint analysis with opposition control and SVD drift
Quantify 2024 breakpoint in centrist support (d=+0.68 overall, d=+0.85 opposition-only),
domain decomposition, extremity-stratified pass rates, and manual LLM audit (75% agreement).
SVD center drift aborted due to axis instability (9/10 consecutive window pairs fail stability threshold).
2026-05-08 23:14:34 +02:00
sgeboers d170444bda feat(analysis): add migration anti-democratic overlap analysis 2026-05-08 22:56:38 +02:00
sgeboers fbf92c82cf feat(right-wing): dual-scoring extremity/sentiment + derived categories
Extremity Scorer (U4 enhanced):
- Now scores BOTH original motion text AND layman explanation separately
- Schema: text_score, text_explanation, layman_score, layman_explanation
- Text scores: 1→7, 2→33, 3→5, 4→5 (mild-to-moderate)
- Layman scores: 1→12, 2→20, 3→17, 4→1 (slightly milder)

Sentiment Analysis (U5 enhanced):
- Now scores BOTH original motion text AND layman explanation separately
- Schema: text_score, text_explanation, layman_score, layman_explanation
- Text sentiment avg: 0.294 (slightly positive)
- Layman sentiment avg: 0.416 (more positive - summaries tone down hostility)

Category Derivation (new):
- Two-phase LLM approach: derive taxonomy from sample, then apply to all
- Discovered 7 categories from 30-motion sample:
  veiligheid/justitie, corona/pandemie, economie/belasting, klimaat/milieu,
  defensie/buitenland, asiel/vreemdelingen, overig
- Applied to 50 motions with distribution shown in DB
- Adds category + category_explanation columns to right_wing_motions
2026-05-05 21:40:58 +02:00
sgeboers f94edc3d04 feat(right-wing): sentiment analysis pipeline for right-wing motions
Implements U5: sentiment_analysis.py uses LLM batch calls (fallback when no
local Dutch sentiment model is available) to score motion sentiment on [-1, 1]
scale.

Design:
- Prompt asks for sentiment from -1 (hostile/aggressive) to 1 (constructive)
- JSON schema enforces numeric score + Dutch explanation
- Batch size 10, max_workers 5 for parallel API calls
- Stores results in  table
- Updates  with avg_sentiment, sentiment_std,
  pct_strongly_negative per year

Sample validation (50 motions): good variance across [-0.9, 1.0] range.
2026-05-05 21:25:42 +02:00
sgeboers d2310edfc4 feat(right-wing): LLM-based policy extremity scoring
Implements U4: extremity_scorer.py uses ai_provider.chat_completion_json_parallel
with a JSON schema enforcing integer 1-5 + Dutch explanation.

Design:
- Batch size 10, max_workers 5 for parallel API calls
- Prompt asks for concrete policy + radicalism score in Dutch
- Stores results in  table (motion_id, score, explanation, error)
- Updates  with yearly averages
- Default sample=50 for validation; --sample -1 scores all motions

Sample validation (50 motions): scores distributed 1→2, 2→34, 3→7, 4→7,
yearly averages ~2.0-2.5 (mild-to-moderate radicalism).
2026-05-05 21:23:08 +02:00
sgeboers 1bc83c4384 feat(right-wing): temporal aggregation of right-wing motion trends
Implements U3: temporal_analysis.py computes yearly_summary from the
right_wing_motions table (U2 output).

Metrics per year:
- total_right_wing, pct_of_total, total_motions
- avg_right_support, avg_left_opposition, centrist_support
- avg_right_keyword_matches, extremity_index (U4 placeholder)
- yoy_right_wing_delta, yoy_pct_delta

Key finding: right-wing motions grew from ~4% (2018) to ~12% (2024-2025)
of all motions, with rising centrist support over time.
2026-05-05 21:20:12 +02:00
sgeboers d3dfb0ce2f feat(right-wing): hybrid motion classifier using keywords + votes
Implements U2: classify_motions.py loads keywords from U1 and classifies
motions as right-wing when:
- right_support >= 60% (CANONICAL_RIGHT parties voting 'voor')
- left_opposition >= 40% (CANONICAL_LEFT parties voting 'tegen')
- AND at least 1 right-wing keyword match in title/body_text

Outputs DuckDB table  with:
- motion_id, year, title, right_support, left_opposition, centrist_support
- right_keyword_matches, left_keyword_matches, classified flag

Classified 2986 of 28331 motions (10.5%) as right-wing.
2026-05-05 21:18:38 +02:00
sgeboers c6f8540671 feat(right-wing): derive right-wing keywords via differential TF-IDF
Implements U1: derive_keywords.py uses party voting patterns to classify
motions as right-wing vs left-wing, then computes differential TF-IDF on
cleaned motion titles to surface policy terms distinctive to right-wing
motions.

Key design choices:
- Vote threshold: 60% of parties in group must vote 'voor'
- Text cleaning strips motion prefixes aggressively (handles multi-word
  surnames, plural 'leden', t.v.v. parentheticals)
- Expanded Dutch stopword list filters procedural and generic noise
- Results written to analysis/right_wing/right_wing_keywords.json

Produces ~50 filtered terms including: asielzoekers, defensie, kernenergie,
boeren, vreemdelingenbeleid, stikstof, asielstop, strafrecht.
2026-05-05 21:14:11 +02:00