docs: add improvement roadmap, research notes, and solution docs

- Add 2026-04-24 ROADMAP with 5 phases / 17 items
- Add detailed implementation plans for P1-001 through P4-005
- Add research artifacts and solution docs from ledger merge
- Add test for SVD component 1 compass alignment

docs/research/llm-motion-classification.md

@@ -0,0 +1,181 @@
+# Motion Extremity Classification with LLMs
+
+## Implementation Status
+
+**Script**: `scripts/classify_motions.py` - Ready to run
+
+**Requirements**:
+- Valid OpenRouter API key in `.env` (current key returns "User not found")
+- ~28,000 motions to classify
+
+**Usage**:
+```bash
+# Classify all motions (will take hours)
+.venv/bin/python scripts/classify_motions.py --delay 0.5
+
+# Test with small sample first
+.venv/bin/python scripts/classify_motions.py --limit 10 --delay 2
+
+# Analyze existing classifications
+.venv/bin/python scripts/classify_motions.py --analyze-only
+```
+
+## Why LLMs?
+
+Rule-based keyword matching is too crude:
+- Only captures 3-4% as "high extremity"
+- Can't understand nuance ("verbod" appears in mundane contexts)
+- Can't assess policy impact magnitude
+
+LLMs can:
+- Understand policy context and implications
+- Assess deviation from consensus/norms
+- Interpret Dutch political terminology
+
+## Proposed LLM Classification Schema
+
+### Output Format
+```json
+{
+  "extremity_score": 1-5,
+  "policy_domain": "migration|identity|economy|social|climate|foreign_policy|justice|education|health|other",
+  "policy_direction": "restrictive|permissive|neutral",
+  "deviation_type": "procedural|semantic|structural",
+  "consensus_level": "broad|partial|narrow|opposition",
+  "rationale": "1-2 sentence explanation"
+}
+```
+
+### Extremity Scale (1-5)
+
+| Score | Label | Description | Examples |
+|-------|-------|-------------|----------|
+| 1 | Mainstream | Standard governance, routine | Budget adjustments, procedural changes |
+| 2 | Minor deviation | Small policy tweaks within consensus | Minor fee changes, small program adjustments |
+| 3 | Moderate deviation | Meaningful but within coalition consensus | Immigration processing changes, targeted regulations |
+| 4 | Major deviation | Challenges status quo meaningfully | Tighter migration rules, significant policy reversals |
+| 5 | Extreme | Fundamental/populist, outside consensus | Complete bans, anti-democratic motions |
+
+### Policy Direction
+
+- **restrictive**: Limits freedoms, tightens rules, reduces access
+- **permissive**: Expands freedoms, loosens rules, increases access  
+- **neutral**: Procedural, administrative, technical
+
+### Consensus Level
+
+- **broad**: Passed with 80%+ parties voting same way
+- **partial**: Passed with 60-80% agreement
+- **narrow**: Passed with 50-60% (close vote)
+- **opposition**: Coalition parties voted against
+
+## LLM Prompt
+
+```
+SYSTEM:
+You are an expert on Dutch parliamentary politics. Classify parliamentary motions 
+on policy extremity using the provided schema.
+
+CLASSIFICATION_RUBRIC:
+- Score 1 (Mainstream): Routine governance, budget adjustments, procedural changes
+- Score 2 (Minor): Small policy tweaks within consensus
+- Score 3 (Moderate): Meaningful changes but within coalition consensus
+- Score 4 (Major): Challenges status quo, significant policy shifts
+- Score 5 (Extreme): Fundamental changes, populist, outside consensus
+
+Consider:
+- Policy impact magnitude
+- Deviation from current norms/policies
+- Coalition/opposition dynamics
+- Dutch political context
+
+USER:
+Classify this motion:
+
+Title: {title}
+Description: {description}
+Voting result: {passed/rejected}, {party_coalition} parties voted for
+
+Respond in JSON format.
+```
+
+## Batch Processing Strategy
+
+```python
+import json
+import asyncio
+from openai import AsyncOpenAI
+
+async def classify_motion_batch(motions: list[dict], model: str = "gpt-4o") -> list[dict]:
+    """Process motions in parallel batches."""
+    
+    client = AsyncOpenAI()
+    
+    async def classify_one(motion: dict) -> dict:
+        prompt = build_prompt(motion)
+        
+        response = await client.chat.completions.create(
+            model=model,
+            messages=[{"role": "system", "content": SYSTEM_PROMPT},
+                    {"role": "user", "content": prompt}],
+            response_format={"type": "json_object"}
+        )
+        
+        result = json.loads(response.choices[0].message.content)
+        result["motion_id"] = motion["id"]
+        return result
+    
+    # Process 50 in parallel
+    results = []
+    for i in range(0, len(motions), 50):
+        batch = motions[i:i+50]
+        batch_results = await asyncio.gather(*[classify_one(m) for m in batch])
+        results.extend(batch_results)
+    
+    return results
+
+async def main():
+    motions = load_motions()  # Load from database
+    classifications = await classify_motion_batch(motions)
+    save_to_database(classifications)
+
+asyncio.run(main())
+```
+
+## Cost Estimate
+
+| Dataset Size | Model | Est. Cost | Est. Time |
+|-------------|-------|-----------|-----------|
+| 35,000 motions | gpt-4o-mini | ~$5-10 | 30-60 min |
+| 35,000 motions | gpt-4o | ~$50-100 | 2-4 hours |
+
+Using `gpt-4o-mini` is sufficient for classification tasks.
+
+## Analysis After Classification
+
+Once classified, we can analyze:
+
+```python
+# Extremity by period
+df.groupby(['period', 'extremity_score']).size().unstack(fill_value=0)
+
+# Domain-Extremity heatmap
+pivot = df.pivot_table(values='motion_id', 
+                        index='policy_domain', 
+                        columns='extremity_score', 
+                        aggfunc='count')
+
+# Passed vs rejected extremity
+df.groupby('passed')['extremity_score'].mean()
+
+# Coalition shift analysis
+df[df['policy_domain'] == 'migration'].groupby(['period', 'policy_direction']).size()
+```
+
+## Expected Insights
+
+1. **Extremity distribution over time** - Has 4-5 score increased?
+2. **Domain-extremity correlation** - Which domains produce extreme policies?
+3. **Direction-extremity** - Restrictive vs permissive extremity by period
+4. **Consensus-extremity** - Are extreme policies passing with broad or narrow consensus?
+5. **Coalition voting** - Which parties support extreme policies?

docs/research/motion-classification-prompt-v2.md

@@ -0,0 +1,136 @@
+# Motion Classification Prompt - v2
+
+## Design Principles
+
+1. **Separation of concerns**: Democratic erosion (substance) is distinct from populist style and restrictiveness
+2. **Orthogonal dimensions**: Each dimension can be classified independently
+3. **Clear boundaries**: Defined transitions between levels
+4. **Dutch political context**: Accounts for EU, referenda, institutional attacks
+
+## Refined Prompt
+
+```python
+SYSTEM_PROMPT = """Je bent een expert in Nederlandse parlementaire politiek en democratische normen.
+
+Classificeer Kamermoties op vier onafhankelijke dimensies:
+
+---
+
+### 1. DEMOCRATIC_EROSION (0-4) — SUBSTANTIEEL
+Meet of deze motie de democratische instituties, rechtsstaat, of burgersrechten bedreigt.
+
+| Score | Label | Beschrijving | Voorbeelden |
+|-------|-------|-------------|-------------|
+| 0 | None | Geen impact op democratische normen | Begroting, procedureel, technische wijzigingen |
+| 1 | Minor | Kleine afwijking van gebruikelijke processen | Kleine uitzonderingen op transparantie-eisen |
+| 2 | Moderate | Betekenisvolle beleidswijziging, maar binnen constitutioneel kader | Verandering asielprocedures, strengere veiligheidsmaatregelen |
+| 3 | Significant | Vraagt om fundamentele verandering in checks & balances | Beperking rechterlijke toetsing, afschaffen referendum |
+| 4 | Critical | Ondermijnt openbaar bestuur, rechtsstaat, of universele rechten | Afschaffing persvrijheid, discriminatie bij wet, anti-EU obstructionisme |
+
+**Beslisregels:**
+- Score 4 ALLEEN bij: (a) directe aanval op persvrijheid/rechterlijke macht, OF (b) systematische discriminatie in wetgeving, OF (c) oproep tot schending internationale verdragen
+- Score 3 bij: (a) referendum afschaffen/herroepen, OF (b) EU-samenwerking fundamenteel ter discussie stellen, OF (c) bevoegdheden uitvoerende macht significant uitbreiden zonder tegenwicht
+- Score 2 is default voor significante beleidswijzigingen die niet bovenstaande raken
+
+---
+
+### 2. POPULIST_STYLE (0-1) — STIJL
+Meet of deze motie populistische retoriek gebruikt. Dit is onafhankelijk van de democratische impact.
+
+| Score | Label | Beschrijving |
+|-------|-------|-------------|
+| 0 | Normal | Zakelijke, institutionele toon |
+| 1 | Populist | Gebruikt anti-establishment framing |
+
+**Indicatoren voor score 1:**
+- "Het volk" vs "de elite"/"de Haag"/"de politiek"
+- "Wij vs zij" framing ("burgers vs bestuurders")
+- Suggestie dat "gewone mensen" anders behandeld moeten worden
+- Vragen om "direct door het volk" zonder institutionele checks
+- Emotioneel geladen taalgebruik over "de problemen van gewone mensen"
+
+**Let op:** Partijpolitieke kritiek is normaal. Alleen extreem anti-institutionele framing telt.
+
+---
+
+### 3. GROUP_TARGETING (0-2) — SELECTIEVE TOEPASSING
+Meet of het beleid specifieke groepen viseert.
+
+| Score | Label | Beschrijving |
+|-------|-------|-------------|
+| 0 | Universal | Algemeen beleid, geen specifieke groep |
+| 1 | Indirect | Algemeen beleid dat onevenredig groepen raakt |
+| 2 | Direct | Expliciet gericht op specifieke bevolkingsgroep |
+
+**Score 2 voorbeelden:**
+- "Asielzoekers" / "illegalen" specifiek viseren
+- "Moslims" / specifieke religieuze groepen
+- "Linkse" of "rechtse" politieke tegenstanders bij naam
+- "Etnische minderheden" als doelwit
+
+**Score 1 voorbeelden:**
+- Algemeen immigratiebeleid dat effectief migranten raakt
+- Veiligheidsmaatregelen die marginaliseerde groepen disproportioneel raken
+
+---
+
+### 4. RESTRICTIVENESS (-1 to +1) — RICHTING
+Meet of het beleid vrijheden/rechten beperkt of uitbreidt.
+
+| Score | Label | Beschrijving |
+|-------|-------|-------------|
+| -1 | Expansive | Breidt vrijheden of toegang uit |
+| 0 | Neutral | Geen directe impact op vrijheden |
+| +1 | Restrictive | Beperkt vrijheden, toegang, of rechten |
+
+**Let op:** Budgettaire of procedurele zaken zijn meestal 0.
+
+---
+
+## OUTPUT FORMAT
+
+Respond in JSON:
+{
+  "democratic_erosion": 0-4,
+  "populist_style": 0-1,
+  "group_targeting": 0-2,
+  "restrictiveness": -1 to 1,
+  "domain": "migration|economy|climate|social|justice|foreign|education|health|other",
+  "rationale": "1-2 zinnen uitleg"
+}
+
+---
+
+## BELANGRIJKE BESLISREGELS
+
+1. **DEMOCRATIC_EROSION en POPULIST_STYLE zijn onafhankelijk**: Een motie kan populistisch zijn (1) maar democratisch onschuldig (0), en omgekeerd.
+
+2. **GROUP_TARGETING is onafhankelijk van RESTRICTIVENESS**: Een restrictieve motie kan universeel (0) of selectief (2) zijn.
+
+3. **EU-afwijkingen gradueren**: 
+   - "Nederlandse invulling van EU-beleid" = score 0-1 erosion
+   - "Nexit/EU verlaten" = score 3-4 erosion
+   - "EU-regels overtreden" = score 2-3 erosion
+
+4. **Referendum-context**: Afschaffen referendum = score 3. Bestaand referendum gebruiken = score 0.
+
+5. **Voorbehoud bij onduidelijkheid**: Als motie tekst ambigu is, kies lagere score en noteer twijfel in rationale."""
+```
+
+## Summary of Changes
+
+| Old | New |
+|-----|-----|
+| Single EXTREMITY_SCORE (1-5) conflating substance+style | Four orthogonal dimensions |
+| "Populistische retoriek" as score 5 criterion | POPULIST_STYLE (0-1), independent of erosion |
+| Vague score boundaries | Defined decision rules with examples |
+| TARGETED_GROUP redundant with score | GROUP_TARGETING (0-2), orthogonal to restrictiveness |
+| EU deviation = score 5 | Graduated EU scores (0-4) with specific examples |
+| Missing referendum/Nexit | Explicit scoring for these patterns |
+
+## What This Enables
+
+1. **Plot RESTRICTIVENESS × DEMOCRATIC_EROSION** — 2D analysis of policy direction
+2. **Track POPULIST_STYLE over time** — Is rhetoric getting more populist?
+3. **Analyze GROUP_TARGETING** — Is group-specific targeting increasing?
+4. **Cross-correlate dimensions** — Does populist style correlate with erosion?