You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
motief/docs/plans/2026-05-05-001-feat-right-w...

447 lines
24 KiB

---
title: Right-Wing Motion Analysis Over Time
type: feat
status: active
date: 2026-05-05
---
# Right-Wing Motion Analysis Over Time
## Summary
Build a pipeline to identify, classify, and analyze right-wing motions across parliamentary history. Track how their volume, policy extremity, and cross-party support have evolved over time. Output includes a derived keyword taxonomy, a hybrid motion classifier, temporal aggregations, policy extremity scores, sentiment trajectories, and time-series visualizations.
---
## Problem Frame
The user hypothesizes that while the **volume** of right-wing motions has remained stable, the motions have become **more extreme** in policy demands and that **centrist parties** increasingly vote in favor of them. This suggests an "overton window" shift that cannot be detected by simple vote-counting alone.
The existing codebase has party-level ideological positioning (SVD/PCA) but lacks motion-level ideological scoring, keyword-based right-wing detection, or temporal analysis of how motion content has radicalized.
---
## Requirements
- **R1.** Derive a right-wing keyword taxonomy from the motions themselves (not hand-curated)
- **R2.** Build a hybrid classifier that identifies "actually right-wing" motions using both keywords and voting patterns
- **R3.** Aggregate and analyze trends per year (volume, support, cross-party adoption)
- **R4.** Score policy extremity (what the motion demands, not just sentiment)
- **R5.** Track sentiment/emotional tone over time as a proxy for radicalization
- **R6.** Produce time-series visualizations for exploration (not yet integrated into the Streamlit app)
- **R7.** Validate classifier accuracy to avoid false positives (e.g., left-wing parties discussing migration neutrally)
---
## Scope Boundaries
- **In scope:** Keyword derivation, motion classification, temporal aggregation, extremity scoring, sentiment analysis, static visualization output
- **Out of scope:** Integration into the Streamlit explorer app (deferred to follow-up)
- **Out of scope:** Real-time updates or pipeline automation
- **Out of scope:** Predictive modeling (we describe trends, we do not forecast)
### Deferred to Follow-Up Work
- **Streamlit tab integration:** Add an interactive "Right-Wing Trends" tab to the explorer (`analysis/tabs/right_wing_trends.py`): separate PR after this plan is executed
- **Motion-level ideological embedding:** Train or fine-tune a motion-specific embedding model to improve classification beyond keywords + votes
---
## Context & Research
### Relevant Code and Patterns
- **`analysis/config.py`** — Defines `CANONICAL_RIGHT = {"PVV", "FVD", "JA21", "SGP"}` and `CANONICAL_LEFT` — these sets are the ground-truth for right/left party identification
- **`scripts/derive_svd_labels.py`** — Already uses TF-IDF on motion titles with Dutch stopwords; this pattern should be reused/extened for keyword derivation
- **`scripts/motion_drift.py`** — Implements cross-ideological voting detection and semantic drift measurement; its `compute_party_voting()` logic is directly relevant for validating the hybrid classifier
- **`analysis/axis_classifier.py`** — Contains `_classify_from_titles()` with Dutch keyword regexes for 4 ideological categories; not sufficient alone but a useful reference for post-processing
- **`pipeline/text_pipeline.py`** — Generates text embeddings via API; can be reused if we need embeddings for the sentiment/extremity analysis
- **`analysis/explorer_data.py`** — `load_motions_df()` loads the full motions table with year parsing; primary data access pattern
- **`database.py`** / **`agent_tools/database.py`** — Provide DuckDB access to `motions` and `mp_votes` tables
### Institutional Learnings
- **`docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md`** — Right-wing parties must appear on the RIGHT side of all axes; same principle applies here: classification must reflect voting behavior, not just semantic content
### External References
- Dutch parliamentary motion data is already present in DuckDB (`data/motions.db`)
- `motion_drift.py` already uses Ridge/Lasso regression for axis stability; similar regression techniques can be applied to temporal trend fitting
---
## Key Technical Decisions
- **Keyword derivation method:** TF-IDF on motion titles/body_text, restricted to motions where `CANONICAL_RIGHT` parties vote predominantly *voor*. This avoids hand-curating and captures the actual language used by right-wing parties.
- **Hybrid classifier:** Two-stage: (1) keyword match for initial filtering, (2) voting-pattern confirmation requiring >60% support from right-wing parties AND <40% opposition from left-wing parties. This handles cases where left-wing parties also use migration keywords neutrally.
- **Policy extremity:** Use an LLM (via `ai_provider.py` subagent pattern) to answer "What concrete policy change does this motion demand?" and rate the radicalism of that demand on a 1-5 scale. This captures policy substance, not emotional language.
- **Sentiment analysis:** Use a lightweight Dutch sentiment model (e.g., `pysentimiento` or similar) rather than an LLM, for cost and speed. If no good Dutch model exists, fallback to LLM batch calls.
- **Temporal granularity:** Annual buckets (`YYYY`), aligning with existing SVD window conventions (`2024`, `2025`, etc.)
- **Visualization:** Static Plotly charts exported to HTML/PNG, not yet integrated into Streamlit. This decouples analysis from UI work.
---
## Open Questions
### Resolved During Planning
- **Q: Should we analyze titles only or full body_text?**
- **A:** Start with titles (fast, already cleaned), validate on a sample using body_text, and upgrade if precision is insufficient.
- **Q: How do we define "centrist parties" for cross-party support tracking?**
- **A:** Treat as the complement of `CANONICAL_RIGHT` and `CANONICAL_LEFT` within `KNOWN_MAJOR_PARTIES`: VVD, D66, CDA, NSC, BBB, CU.
### Deferred to Implementation
- **Q: What is the optimal TF-IDF threshold for keyword inclusion?** Depends on corpus distribution; will be determined by inspecting the keyword ranked list.
- **Q: Which Dutch sentiment model performs best on parliamentary language?** Requires empirical testing; candidate models to be evaluated during U5.
- **Q: Does policy extremity scoring need few-shot examples for consistency?** Will be determined during U4 implementation; if LLM outputs are inconsistent, add exemplars.
---
## High-Level Technical Design
> *This illustrates the intended approach and is directional guidance for review, not implementation specification. The implementing agent should treat it as context, not code to reproduce.*
```
┌─────────────────────────────────────────────────────────────┐
│ PHASE 1: KEYWORD DERIVATION (U1) │
│ - Filter motions where right-wing parties vote >60% voor │
│ - Run TF-IDF on titles/body_text │
│ - Extract top-N distinctive terms │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ PHASE 2: HYBRID CLASSIFICATION (U2) │
│ - Keyword filter: motion contains any top-N term │
│ - Voting filter: right >60% voor AND left <40% tegen │
│ - Output: right_wing_motions table with year, score │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ PHASE 3: TEMPORAL AGGREGATION (U3) │
│ - Group by year │
│ - Compute: count, % of total motions, avg support │
│ - Track: centrist party support over time │
│ - Output: yearly_summary DataFrame │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ PHASE 4: POLICY EXTREMITY (U4) │
│ - Sample motions per year (stratified by keyword density) │
│ - LLM prompt: "What concrete policy does this demand?" │
│ - Rate radicalism 1-5 │
│ - Output: extremity_scores table │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ PHASE 5: SENTIMENT ANALYSIS (U5) │
│ - Dutch sentiment model on motion titles/body_text │
│ - Aggregate: avg sentiment per year │
│ - Output: sentiment_by_year DataFrame │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ PHASE 6: VISUALIZATION (U6) │
│ - Time-series: volume, extremity, sentiment, centrist vote │
│ - Export: static HTML/PNG charts │
└─────────────────────────────────────────────────────────────┘
```
---
## Execution Strategy: Agent Assignments & Parallelization
| Unit | Agents | Can Parallelize With | Notes |
|------|--------|---------------------|-------|
| U1 | `general` (text pipeline), `ce-best-practices-researcher` (keyword extraction patterns) | | Foundation unit; must complete before all downstream work. DuckDB batch aggregation is I/O-bound and fast; no subagents needed. |
| U2 | `ce-best-practices-researcher` (TF-IDF best practices, Dutch NLP), `general` (implementation) | U3 (after U1) | Keyword derivation is research-heavy; the researcher can pre-fetch Dutch political TF-IDF patterns while `general` implements the aggregation. Once U1 data is ready, keyword extraction can run in parallel with U3's vote-pattern analysis. |
| U3 | `general` (DuckDB SQL + Python) | U2 (after U1) | Pure computation; no external research needed. Runs in parallel with U2 once U1 is done because U2 and U3 read the same `motion_rightness` table but produce independent features. |
| U4 | `general` (scikit-learn pipeline), `ce-framework-docs-researcher` (if exploring Dutch sentiment models) | | Depends on U2 + U3. The LLM-based extremity scorer can be delegated to `general` in batch mode; no real-time LLM calls at inference time. |
| U5 | `ce-best-practices-researcher` (time-series visualization patterns), `general` (Plotly implementation) | | Depends on U4. Visualization is straightforward once data shape is known; researcher mainly validates responsive/mobile chart patterns. |
| U6 | `general` (integration), `ce-test-browser` (if applicable) | | Depends on U5. Integration work is sequential by nature; browser tests validate the Streamlit tab render. |
**Parallelization summary:**
- **U1** is strictly sequential (foundation).
- **U2 + U3** can run in parallel once U1 completes.
- **U4** is sequential after U2+U3.
- **U5 + U6** are sequential after U4.
- Total wall-clock phases: **4** (U1 U2U3 U4 U5 U6).
---
## Implementation Units
- **U1. Keyword Derivation**
**Goal:** Extract the most distinctive terms used in right-wing motions without hand-curating.
**Requirements:** R1
**Dependencies:** None
**Files:**
- Create: `analysis/right_wing/derive_keywords.py`
- Modify: `scripts/derive_svd_labels.py` (reference only)
- Test: `tests/test_derive_keywords.py`
**Approach:**
1. Query `motions` joined with `mp_votes` to identify motions where right-wing parties vote predominantly *voor* (>60%).
2. Collect a control group: motions where left-wing parties vote predominantly *voor* (>60%).
3. Run TF-IDF on titles (and optionally body_text) for both groups, using Dutch stopwords.
4. Compute differential TF-IDF: terms with high scores in right-group and low in left-group.
5. Manually inspect top 50 terms and filter out generic parliamentary terms (e.g., "motie", "kamer").
6. Persist the final keyword list to `analysis/right_wing/right_wing_keywords.json`.
**Patterns to follow:**
- `scripts/derive_svd_labels.py` for TF-IDF pipeline
- `analysis/config.py` for `CANONICAL_RIGHT` / `CANONICAL_LEFT`
**Test scenarios:**
- Happy path: right-wing motions contain terms like "migratie", "asiel", "grenzen" in top results
- Edge case: control group does NOT have these terms in top 50
- Edge case: generic terms ("motie", "regering") are filtered out
- Integration: keyword list can be loaded as JSON and used for regex matching
**Verification:**
- `right_wing_keywords.json` exists and contains >= 20 distinct terms
- Manual inspection confirms terms are politically distinctive
---
- **U2. Hybrid Motion Classifier**
**Goal:** Identify "actually right-wing" motions using both keywords and voting patterns.
**Requirements:** R2, R7
**Dependencies:** U1
**Files:**
- Create: `analysis/right_wing/classify_motions.py`
- Test: `tests/test_classify_motions.py`
**Approach:**
1. **Keyword filter:** Match motion title/body_text against `right_wing_keywords.json` (case-insensitive, whole-word regex).
2. **Voting filter:** For each candidate motion, compute:
- `right_support` = % of `CANONICAL_RIGHT` parties voting *voor*
- `left_opposition` = % of `CANONICAL_LEFT` parties voting *tegen*
- Pass if `right_support >= 0.60` AND `left_opposition >= 0.40`
3. Output a DuckDB table `right_wing_motions` with columns: `motion_id`, `year`, `title`, `right_support`, `left_opposition`, `keyword_matches`.
4. Run a validation sample: manually inspect 20 random classified motions and 20 random non-classified motions to estimate precision/recall.
**Patterns to follow:**
- `scripts/motion_drift.py` for cross-ideological voting logic
- `analysis/explorer_data.py` for data loading patterns
**Test scenarios:**
- Happy path: a PVV motion about "asielzoekers" with 80% right support and 60% left opposition is classified
- Edge case: a PvdA motion mentioning "migratie" neutrally with 20% right support is NOT classified
- Edge case: motion with right support 60% but left opposition only 10% is NOT classified
- Error path: motion with no votes is skipped gracefully
- Integration: `right_wing_motions` table is queryable and has expected row count
**Verification:**
- Validation sample shows >80% precision and >70% recall
- `right_wing_motions` table contains >100 rows (non-empty)
---
- **U3. Temporal Aggregation**
**Goal:** Compute yearly trends in right-wing motion volume, support, and cross-party adoption.
**Requirements:** R3
**Dependencies:** U2
**Files:**
- Create: `analysis/right_wing/temporal_analysis.py`
- Test: `tests/test_temporal_analysis.py`
**Approach:**
1. Group `right_wing_motions` by `year`.
2. For each year, compute:
- `total_right_wing`: count of right-wing motions
- `pct_of_total`: % of all motions that year
- `avg_right_support`: average right-party support
- `avg_left_opposition`: average left-party opposition
- `centrist_support`: % of centrist parties (VVD, D66, CDA, NSC, BBB, CU) voting *voor*
- `extremity_index`: placeholder for U4 scores (NULL until backfilled)
3. Compute year-over-year deltas for each metric.
4. Persist to `yearly_right_wing_summary` table or DataFrame export.
**Patterns to follow:**
- `analysis/trajectory.py` for time-windowed aggregations
- `analysis/explorer_data.py` for DuckDB-to-pandas patterns
**Test scenarios:**
- Happy path: each year has a row with all metrics computed
- Edge case: year with 0 right-wing motions shows 0 count and NULL centrist support
- Edge case: year with missing vote data for some parties still computes available metrics
- Integration: output DataFrame can be merged with U4 extremity scores
**Verification:**
- One row per year from earliest to latest motion
- `pct_of_total` values sum to <= 100% when checked manually for a sample year
- Centrist support shows a plausible trend (not all 0% or 100%)
---
- **U4. Policy Extremity Scoring**
**Goal:** Score how radical the policy demand of each right-wing motion is.
**Requirements:** R4
**Dependencies:** U2
**Files:**
- Create: `analysis/right_wing/extremity_scorer.py`
- Test: `tests/test_extremity_scorer.py`
**Approach:**
1. For each right-wing motion, build a prompt:
> "Dit is een motie in het Nederlandse parlement. Wat vraagt de motie concreet? Beoordeel hoe radicaal dit voorstel is op een schaal van 1 (mild/technisch) tot 5 (extreem/fundamenteel). Geef alleen het cijfer en een korte verklaring in het Nederlands."
2. Use `ai_provider.chat_completion_json()` with a JSON schema enforcing integer 1-5 + explanation string.
3. Batch process motions (parallel API calls, 10-15 per batch) to minimize cost.
4. Store results in `extremity_scores` table: `motion_id`, `score`, `explanation`.
5. Compute yearly average and merge into U3's summary.
**Execution note:** Start with a sample of 50 motions to validate scoring consistency before running the full set.
**Patterns to follow:**
- `summarizer.py` for batch LLM processing and parallel API calls
- `ai_provider.py` for JSON-mode chat completions
**Test scenarios:**
- Happy path: a motion demanding "sluit alle AZC's" scores 5/5
- Happy path: a motion requesting "rapporteer cijfers" scores 1/5
- Edge case: LLM returns invalid JSON → fallback to retry or mark as NULL
- Error path: API failure → motion is skipped, not blocking
- Integration: extremity scores correlate with keyword intensity (sanity check)
**Verification:**
- Sample of 50 shows inter-rater consistency (same motion re-scored twice gets same score)
- Score distribution is not all 1s or all 5s (has variance)
- `extremity_scores` table covers >= 90% of right-wing motions
---
- **U5. Sentiment Analysis Pipeline**
**Goal:** Track emotional tone of right-wing motions over time as a proxy for radicalization.
**Requirements:** R5
**Dependencies:** U2
**Files:**
- Create: `analysis/right_wing/sentiment_analysis.py`
- Test: `tests/test_sentiment_analysis.py`
**Approach:**
1. Evaluate candidate Dutch sentiment models:
- `pysentimiento` (nl model)
- `transformers` pipeline with Dutch BERT (`wietsedv/bert-base-dutch-cased` fine-tuned for sentiment)
- Fallback: LLM batch calls if models are poor
2. Run on motion titles + first 200 chars of body_text (avoiding noise).
3. Map outputs to [-1, 1] scale (negative = hostile/aggressive, positive = constructive).
4. Aggregate by year: avg sentiment, std deviation, % strongly negative.
5. Merge into U3 summary.
**Patterns to follow:**
- `pipeline/text_pipeline.py` for text preprocessing
- Lightweight model evaluation script similar to `test_mistral.py`
**Test scenarios:**
- Happy path: motion with "stop de immigratie" scores negative
- Happy path: motion with "verbeter de procedure" scores neutral/positive
- Edge case: very short title (< 5 words) is handled gracefully
- Error path: model fails to load fallback to LLM or skip
- Integration: sentiment trend correlates with extremity trend (sanity check)
**Verification:**
- Sentiment scores show variance (not all identical)
- Manual inspection of 10 random motions confirms direction is plausible
- Model inference time is < 100ms per motion (acceptable for batch)
---
- **U6. Time-Series Visualization**
**Goal:** Produce static charts showing volume, extremity, sentiment, and centrist support over time.
**Requirements:** R6
**Dependencies:** U3, U4, U5
**Files:**
- Create: `analysis/right_wing/visualize_trends.py`
- Output: `output/right_wing_trends.html` and/or `.png` files
**Approach:**
1. Load `yearly_right_wing_summary` DataFrame.
2. Generate 4 charts:
- **Volume:** Line chart of `total_right_wing` + `pct_of_total` (dual axis)
- **Extremity:** Line chart of `avg_extremity_score` with error bars (std)
- **Sentiment:** Line chart of `avg_sentiment` + % strongly negative (stacked area)
- **Centrist Support:** Line chart of `centrist_support` over time with party breakdown
3. Use Plotly (consistent with `analysis/visualize.py`) with dark theme colors.
4. Export to `output/right_wing_trends.html` (interactive) and `.png` (static).
**Patterns to follow:**
- `analysis/visualize.py` for Plotly setup and theming
- `analysis/tabs/_rendering.py` for dark theme color constants
**Test scenarios:**
- Happy path: HTML file is generated and opens without errors
- Happy path: all 4 charts render with data spanning multiple years
- Edge case: missing data for some years chart shows gaps, not crashes
- Integration: charts visually confirm the user's hypothesis (stable volume, rising extremity/centrist support)
**Verification:**
- `output/right_wing_trends.html` exists and is > 100KB
- Manual inspection of charts shows clear trends
- Charts are suitable for sharing (static PNGs are readable)
---
## System-Wide Impact
- **New tables:** `right_wing_motions`, `extremity_scores` — these are derived/analysis tables, not core schema. They can be regenerated.
- **No changes to existing tables:** `motions`, `mp_votes`, `svd_vectors` are read-only for this feature.
- **No API surface changes:** This is an offline analysis script, not integrated into the app yet.
- **Performance:** TF-IDF on ~29K titles is trivial. LLM calls for U4 are the bottleneck; batching keeps cost manageable.
---
## Risks & Dependencies
| Risk | Mitigation |
|------|------------|
| Keyword list overfits to current parliamentary period | Validate across multiple years; include historical data in TF-IDF corpus |
| LLM extremity scoring is inconsistent | Add few-shot examples; validate on sample before full run; allow NULL scores |
| Dutch sentiment model performs poorly on parliamentary language | Evaluate multiple models; fallback to LLM if needed |
| Classification has false positives (left-wing motions caught) | Hybrid voting filter mitigates this; validation sample checks precision |
| LLM API costs for extremity scoring exceed budget | Batch aggressively; score a stratified sample (e.g., 30 per year) instead of all motions |
---
## Documentation / Operational Notes
- Add a README in `analysis/right_wing/` explaining how to regenerate the analysis
- Document the keyword list and classification thresholds for reproducibility
- Note the LLM model used for extremity scoring and its version/date
---
## Sources & References
- **Related code:** `scripts/motion_drift.py`, `scripts/derive_svd_labels.py`, `analysis/axis_classifier.py`
- **Related learnings:** `docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md`
- **Origin:** User request — analyze right-wing motion trends over time