docs: add improvement roadmap, research notes, and solution docs
- Add 2026-04-24 ROADMAP with 5 phases / 17 items - Add detailed implementation plans for P1-001 through P4-005 - Add research artifacts and solution docs from ledger merge - Add test for SVD component 1 compass alignment
This commit is contained in:
@@ -0,0 +1,168 @@
|
||||
---
|
||||
title: Critical Anti-Patterns Discovered During Mindmodel Generation
|
||||
date: 2026-04-12
|
||||
category: docs/solutions/best-practices
|
||||
module: stemwijzer
|
||||
problem_type: best_practice
|
||||
component: documentation
|
||||
severity: critical
|
||||
applies_when:
|
||||
- When adding logging to any module
|
||||
- When working with Streamlit test isolation
|
||||
- When generating or updating .mindmodel/ for this project
|
||||
tags: [anti-patterns, logging, streamlit, mindmodel, code-quality]
|
||||
---
|
||||
|
||||
# Critical Anti-Patterns Discovered During Mindmodel Generation
|
||||
|
||||
## Context
|
||||
|
||||
During a comprehensive mindmodel generation session (Phase 1: 7 parallel analysis agents, Phase 2: constraint-writer assembly), several critical anti-patterns were discovered and documented in `.mindmodel/anti-patterns/anti-patterns.yaml`. This document captures the key findings for future reference.
|
||||
|
||||
## Guidance
|
||||
|
||||
### 1. Use Logging, Not Print Statements
|
||||
|
||||
**CRITICAL**: `api_client.py` uses `print()` instead of logging throughout (11 instances).
|
||||
|
||||
**Broken pattern:**
|
||||
```python
|
||||
# api_client.py - BAD
|
||||
print(f"Fetched {len(voting_records)} voting records from API")
|
||||
print(f"Error fetching motions from API: {e}") # No traceback
|
||||
```
|
||||
|
||||
**Correct pattern:**
|
||||
```python
|
||||
# GOOD - use logging throughout
|
||||
import logging
|
||||
|
||||
_logger = logging.getLogger(__name__)
|
||||
|
||||
def get_motions(self, ...):
|
||||
try:
|
||||
_logger.info("Fetched %d voting records from API", len(voting_records))
|
||||
except Exception as e:
|
||||
_logger.exception("Error fetching motions from API: %s", e)
|
||||
return []
|
||||
```
|
||||
|
||||
### 2. Streamlit Global State Replacement
|
||||
|
||||
**CRITICAL**: `explorer.py` has module-level `st = _DummySt()` which shadows Streamlit globally.
|
||||
|
||||
**Broken pattern:**
|
||||
```python
|
||||
# explorer.py - BAD
|
||||
try:
|
||||
import plotly.express as px
|
||||
except Exception:
|
||||
class _DummySt:
|
||||
figure = _DummyFigure
|
||||
# ...
|
||||
st = _DummySt() # Global replacement - affects all imports!
|
||||
```
|
||||
|
||||
**Correct pattern:**
|
||||
```python
|
||||
# GOOD - use conditional flags
|
||||
try:
|
||||
import plotly.express as px
|
||||
import plotly.graph_objects as go
|
||||
HAS_PLOTLY = True
|
||||
except ImportError:
|
||||
HAS_PLOTLY = False
|
||||
px = None
|
||||
go = None
|
||||
|
||||
def render_chart(data):
|
||||
if not HAS_PLOTLY:
|
||||
_logger.warning("Plotly not available")
|
||||
return
|
||||
# ... rest of chart logic
|
||||
```
|
||||
|
||||
### 3. Logger Naming Inconsistency
|
||||
|
||||
**WARNING**: 33 files split between `logger = logging.getLogger(__name__)` and `_logger = logging.getLogger(__name__)`.
|
||||
|
||||
Files with `logger` (16): api_client.py, ai_provider.py, pipeline files, analysis files
|
||||
Files with `_logger` (17): database.py, explorer.py, explorer_helpers.py
|
||||
|
||||
**Recommendation**: Standardize on `_logger` for module-level loggers. Update CODE_STYLE.md to explicitly state the convention.
|
||||
|
||||
### 4. Bare Except with Pass
|
||||
|
||||
**CRITICAL**: `database.py` line 47 has bare `except: pass` that catches KeyboardInterrupt, SystemExit, MemoryError.
|
||||
|
||||
**Broken pattern:**
|
||||
```python
|
||||
# database.py line 47 - BAD
|
||||
try:
|
||||
conn.execute("CREATE SEQUENCE IF NOT EXISTS motions_id_seq START 1")
|
||||
except: # Catches EVERYTHING
|
||||
pass
|
||||
```
|
||||
|
||||
**Correct pattern:**
|
||||
```python
|
||||
# GOOD
|
||||
try:
|
||||
conn.execute("CREATE SEQUENCE IF NOT EXISTS motions_id_seq START 1")
|
||||
except Exception as exc:
|
||||
_logger.debug("Sequence creation skipped: %s", exc)
|
||||
```
|
||||
|
||||
## Why This Matters
|
||||
|
||||
1. **Logging over Print**: Structured logging enables log aggregation, filtering by level, and includes stack traces. Print statements are invisible in production and provide no context during failures.
|
||||
|
||||
2. **Global State**: Module-level replacements of standard library modules cause subtle bugs where code imports work differently depending on import order.
|
||||
|
||||
3. **Consistency**: Mixed logger naming makes code harder to grep and grep-replace. Pick one convention and enforce it via linting.
|
||||
|
||||
4. **Bare Except**: Catching all exceptions including `KeyboardInterrupt` and `SystemExit` can prevent graceful shutdown and mask serious issues.
|
||||
|
||||
## When to Apply
|
||||
|
||||
- Before committing any logging changes: ensure using `_logger`, not `print()`
|
||||
- When adding optional dependency handling: use flags, not global replacements
|
||||
- When updating CODE_STYLE.md: add explicit logger naming convention
|
||||
- When updating .mindmodel/: verify anti-patterns section is current
|
||||
|
||||
## Examples
|
||||
|
||||
### Fixing api_client.py Logging
|
||||
|
||||
```python
|
||||
# Before (broken)
|
||||
print(f"Processed {count} motions")
|
||||
|
||||
# After (correct)
|
||||
_logger.info("Processed %d motions", count)
|
||||
```
|
||||
|
||||
### Fixing Exception Handling
|
||||
|
||||
```python
|
||||
# Before (broken)
|
||||
try:
|
||||
risky_operation()
|
||||
except:
|
||||
pass
|
||||
|
||||
# After (correct)
|
||||
try:
|
||||
risky_operation()
|
||||
except Exception as exc:
|
||||
_logger.warning("Operation failed: %s", exc)
|
||||
return safe_fallback
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- `.mindmodel/anti-patterns/anti-patterns.yaml` - Full anti-pattern documentation
|
||||
- `.mindmodel/constraints/logging.yaml` - Logging conventions
|
||||
- `.mindmodel/constraints/error-handling.yaml` - Error handling patterns
|
||||
- `CODE_STYLE.md` - Code style guide (needs update for logger naming)
|
||||
- `AGENTS.md` - Project conventions (RIGHT-wing parties on RIGHT, SVD labels = voting patterns)
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
title: "uv.lock parse error due to pytest entry missing source"
|
||||
module: tooling
|
||||
component: tooling
|
||||
problem_type: build_error
|
||||
severity: medium
|
||||
date: 2026-04-05
|
||||
tags: [uv, lockfile, pytest, packaging, streamlit]
|
||||
---
|
||||
|
||||
Problem
|
||||
-------
|
||||
|
||||
Running `uv` commands failed with a parse error in `uv.lock` caused by an ambiguous/malformed `pytest` entry that lacked a proper `source` field and conflicted with another package entry.
|
||||
|
||||
Symptoms
|
||||
--------
|
||||
|
||||
- `uv run streamlit run Home.py` failed with: "Dependency `pytest` has missing `source` field but has more than one matching package".
|
||||
- `uv lock` and `uv add` also failed because `uv.lock` could not be parsed.
|
||||
|
||||
What didn't work
|
||||
----------------
|
||||
|
||||
- Attempting `uv add "pytest>=9.0.2" --dev` failed because the lockfile parser errored before the command could modify anything.
|
||||
- Manual edits to `uv.lock` were used as a temporary stop-gap (allowed `uv` to run) but are not a durable solution because `uv.lock` is generated.
|
||||
|
||||
Solution
|
||||
--------
|
||||
|
||||
1. Regenerate the lockfile from `pyproject.toml` so `uv.lock` and project metadata are consistent:
|
||||
|
||||
- Run: `uv lock`
|
||||
- Inspect the resulting `uv.lock` to ensure `pytest` appears as a single `[[package]]` entry with a `source` field and expected hashes.
|
||||
|
||||
2. Commit the regenerated lock locally (do not push without review):
|
||||
|
||||
- `git add uv.lock`
|
||||
- `git commit -m "chore: regenerate uv.lock (resolve pytest source ambiguity)"`
|
||||
|
||||
Why this works
|
||||
--------------
|
||||
|
||||
- `uv.lock` is the canonical, generated lockfile. The parser expects each package entry to have an unambiguous `source` so `uv` can resolve hashes and reproducible installs. Regenerating produces a consistent lockfile derived from `pyproject.toml` and resolves duplicated/malformed entries.
|
||||
- Manual edits fix symptoms but can be overwritten or lead to inconsistent state; regenerating ensures upstream metadata and lockfile match the resolver's expectations.
|
||||
|
||||
Prevention
|
||||
----------
|
||||
|
||||
- Avoid hand-editing `uv.lock`. When a lockfile parse error appears, prefer regenerating with `uv lock`.
|
||||
- Add a lightweight CI check to ensure `uv lock --check` (or `uv lock` with no changes) passes before merging changes that touch dependencies or the lockfile.
|
||||
- Make `pytest` (and other dev tools) authoritative in `pyproject.toml` under `dependency-groups.dev` so the resolver has a single source of truth.
|
||||
|
||||
Verification
|
||||
------------
|
||||
|
||||
- After regeneration, verify `uv` commands work and tests run:
|
||||
|
||||
- `uv run streamlit run Home.py` → Streamlit should start and print Local/Network URL
|
||||
- `.venv/bin/python -m pytest tests/ -q` → Confirm tests run (example in this run: `171 passed, 2 skipped`).
|
||||
|
||||
Related files
|
||||
-------------
|
||||
|
||||
- `uv.lock`
|
||||
- `pyproject.toml`
|
||||
|
||||
If you want, I can:
|
||||
|
||||
1) Run the Streamlit verification now, or
|
||||
2) Propose a small CI job snippet to enforce `uv lock --check`, or
|
||||
3) Create a short PR description if you want this committed change pushed and opened as a PR.
|
||||
@@ -0,0 +1,125 @@
|
||||
---
|
||||
module: llm-classification
|
||||
tags: [polarization, nlp, prompt-design, democratic-norms]
|
||||
problem_type: classification-schema-design
|
||||
date: 2026-04-05
|
||||
reviewed_by:
|
||||
- correctness-reviewer
|
||||
- domain-expert (Dutch politics)
|
||||
- clarity-reviewer
|
||||
---
|
||||
|
||||
# LLM Motion Classification: Prompt Design Lessons
|
||||
|
||||
## Problem
|
||||
|
||||
Wanted to classify 28,000 Dutch parliamentary motions by "extremity" to measure polarization over time.
|
||||
|
||||
Initial prompt conflated multiple concepts:
|
||||
- Democratic norm erosion
|
||||
- Populist rhetoric style
|
||||
- Group targeting
|
||||
- Restrictiveness vs permissiveness
|
||||
|
||||
## Initial v1 Design (Flawed)
|
||||
|
||||
```python
|
||||
EXTREMITY_SCORE (1-5):
|
||||
- 1: Mainstream
|
||||
- 5: "Undermines checks & balances, threatens rule of law,
|
||||
discriminates groups, populist rhetoric"
|
||||
```
|
||||
|
||||
**Problems identified:**
|
||||
1. Populist rhetoric is style, not substance — shouldn't be in same score as democratic erosion
|
||||
2. "Extreme" undefined — compared to what baseline?
|
||||
3. Score 4/5 boundary unclear
|
||||
4. TARGETED_GROUP redundant with EXTREMITY_SCORE
|
||||
5. EU deviation always = score 5 (too broad)
|
||||
6. Missing Dutch-specific patterns (Nexit, referendum abolition)
|
||||
|
||||
## Refined v2 Design (Four Orthogonal Dimensions)
|
||||
|
||||
### 1. DEMOCRATIC_EROSION (0-4) — Substance only
|
||||
| Score | Label | Criteria |
|
||||
|-------|-------|----------|
|
||||
| 0 | None | No impact on democratic norms |
|
||||
| 1 | Minor | Small procedural deviations |
|
||||
| 2 | Moderate | Significant policy change, within constitutional framework |
|
||||
| 3 | Significant | Fundamental change to checks & balances |
|
||||
| 4 | Critical | Undermines rule of law, press freedom, systematic discrimination |
|
||||
|
||||
**Decision rules:**
|
||||
- Score 4 ONLY if: (a) direct attack on judiciary/press, OR (b) systematic discrimination in law, OR (c) call to violate international treaties
|
||||
- Score 3 if: (a) abolish referendum, OR (b) fundamentally question EU cooperation, OR (c) significantly expand executive powers
|
||||
|
||||
### 2. POPULIST_STYLE (0-1) — Style only
|
||||
Independent of democratic impact. A motion can be populist (1) but democratic (0).
|
||||
|
||||
**Indicators:**
|
||||
- "Het volk" vs "de elite/den Haag"
|
||||
- "Wij vs zij" framing
|
||||
- Call for "direct democracy" without checks
|
||||
- Emotionally charged language
|
||||
|
||||
### 3. GROUP_TARGETING (0-2) — Targeting only
|
||||
| Score | Label |
|
||||
|-------|-------|
|
||||
| 0 | Universal — general policy |
|
||||
| 1 | Indirect — general policy that disproportionately affects groups |
|
||||
| 2 | Direct — explicitly targets specific population group |
|
||||
|
||||
### 4. RESTRICTIVENESS (-1 to +1) — Direction only
|
||||
| Score | Label |
|
||||
|-------|-------|
|
||||
| -1 | Expansive |
|
||||
| 0 | Neutral |
|
||||
| +1 | Restrictive |
|
||||
|
||||
## Key Lessons Learned
|
||||
|
||||
### 1. Separate Style from Substance
|
||||
Populist rhetoric ≠ democratic erosion. A mainstream party using strong language isn't anti-democratic. Conflating them causes false positives.
|
||||
|
||||
### 2. Make Dimensions Orthogonal
|
||||
- DEMOCRATIC_EROSION × RESTRICTIVENESS: A policy can be erosive AND restrictive, or erosive AND permissive
|
||||
- POPULIST_STYLE × DEMOCRATIC_EROSION: Can have populist (1) with democratic (0), and vice versa
|
||||
- GROUP_TARGETING × RESTRICTIVENESS: Restrictive ≠ targeted (and vice versa)
|
||||
|
||||
### 3. Add Decision Rules for Boundaries
|
||||
Vague transitions ("significant" → "critical") cause inconsistency. Define specific triggers:
|
||||
```
|
||||
Score 4 ONLY when: (a) OR (b) OR (c)
|
||||
Score 3 when: (a) OR (b) OR (c)
|
||||
```
|
||||
|
||||
### 4. Gradate EU Deviation
|
||||
Not all EU deviation is equal:
|
||||
- Dutch implementation of EU policy → erosion 0-1
|
||||
- Nexit / leave EU → erosion 3-4
|
||||
- Violate EU rules → erosion 2-3
|
||||
|
||||
### 5. Include Domain-Specific Patterns
|
||||
Dutch context matters:
|
||||
- Referendum abolition = score 3
|
||||
- "Den Haag" / "establishment" attacks = check for populist style
|
||||
- Nexit = score 3-4 depending on framing
|
||||
|
||||
### 6. Define Reference Baselines
|
||||
"Abnormal" compared to what?
|
||||
- 2016 consensus
|
||||
- EU norms
|
||||
- Historical Dutch practice
|
||||
- International standards
|
||||
|
||||
## Testing Recommendations
|
||||
|
||||
1. **Calibration set**: 50 motions with expert annotations before production
|
||||
2. **Boundary cases**: Test score 3/4 transitions explicitly
|
||||
3. **Cross-rater reliability**: Multiple classifiers on same motions
|
||||
4. **Domain-specific test cases**: Migration, EU, constitutional reform
|
||||
|
||||
## Files
|
||||
|
||||
- `scripts/classify_motions.py` — Implementation with v2 prompt
|
||||
- `docs/research/motion-classification-prompt-v2.md` — Full prompt documentation
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
title: "Policy Extremity vs Voting Extremity: Independent Phenomena"
|
||||
date: 2026-04-05
|
||||
module: analysis
|
||||
problem_type: research
|
||||
component: motion-analysis
|
||||
tags: [polarization, policy-extremity, voting-extremity, svd, embedding-norm]
|
||||
---
|
||||
|
||||
# Policy Extremity vs Voting Extremity: Independent Phenomena
|
||||
|
||||
## Key Finding
|
||||
|
||||
**Voting extremity** (how divided parliament is) and **policy extremity** (how far motions are from political center) are **independent phenomena** with opposite trends:
|
||||
|
||||
| Measure | 2016 | 2026 | Trend |
|
||||
|---------|------|------|-------|
|
||||
| **Voting Extremity** | 0.70 | 0.46 | More divided |
|
||||
| **Policy Extremity** | 9.0 | 4.2 | Less extreme |
|
||||
|
||||
**Correlation: r = -0.011** (essentially zero)
|
||||
|
||||
## Definitions
|
||||
|
||||
### Voting Extremity
|
||||
- **Formula**: margin / total votes
|
||||
- **Interpretation**: How divided parliament is
|
||||
- 1.0 = unanimous (all votes same direction)
|
||||
- 0.0 = perfectly split (50-50)
|
||||
- **Trend**: Increased (more close votes in recent years)
|
||||
|
||||
### Policy Extremity
|
||||
- **Formula**: L2 norm of SVD embedding vector
|
||||
- **Interpretation**: How "far out" a motion is in political semantic space
|
||||
- **Trend**: Decreased (motions closer to political center)
|
||||
|
||||
## Analysis
|
||||
|
||||
### Why Are They Independent?
|
||||
|
||||
1. **Voting extremity** captures **how parties divide** on issues
|
||||
2. **Policy extremity** captures **where motions sit** in policy space
|
||||
|
||||
A motion can be:
|
||||
- Near the center (low policy extremity) but divide parties 50-50 (high voting extremity)
|
||||
- Far from center (high policy extremity) but pass unanimously (low voting extremity)
|
||||
|
||||
### Historical Pattern
|
||||
|
||||
- **2016**: Coalition passed "extreme" motions (legislative proposals) with consensus
|
||||
- **2026**: More divided votes on "moderate" motions (procedural/administrative)
|
||||
|
||||
### Interpretation
|
||||
|
||||
The parliament has become **more divided in how it votes**, but the **policies being passed are actually less extreme** in semantic space.
|
||||
|
||||
This suggests:
|
||||
- The polarization is about **different issues** dividing parties
|
||||
- The "extremes" that pass are now closer to mainstream positions
|
||||
- What changed is **what divides parties**, not **how radical the policies are**
|
||||
|
||||
## Visualization
|
||||
|
||||
See `docs/research/voting_vs_policy_extremity.png`
|
||||
|
||||
## Methodology
|
||||
|
||||
```python
|
||||
# Voting extremity = margin / total
|
||||
voting_extremity = abs(votes_for - votes_against) / total_votes
|
||||
|
||||
# Policy extremity = L2 norm of SVD embedding
|
||||
policy_extremity = np.linalg.norm(embedding_vector)
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
These findings confirm that **voting extremity ≠ policy extremity**. They capture different aspects of parliamentary behavior and should be analyzed separately.
|
||||
|
||||
The increase in voting extremity reflects genuine polarization in parliamentary divisions. But the decrease in policy extremity suggests that the policies actually being passed are not more radical—they're just more contested.
|
||||
@@ -0,0 +1,146 @@
|
||||
---
|
||||
title: "Quantifying Political Extremity: Voting vs Policy"
|
||||
date: 2026-04-05
|
||||
module: analysis
|
||||
problem_type: research
|
||||
component: motion-analysis
|
||||
tags: [polarization, voting-extremity, policy-extremity, embedding-analysis, parliamentary-motion]
|
||||
---
|
||||
|
||||
# Quantifying Political Extremity: Voting vs Policy
|
||||
|
||||
## Context
|
||||
|
||||
Initial analysis of parliamentary motions sought to measure polarization by examining how "extreme" policies have become. The hypothesis was that extremes on both sides became more extreme. The analysis revealed this hypothesis was incorrect — and surfaced two independent phenomena.
|
||||
|
||||
## Guidance
|
||||
|
||||
### Key Finding: Two Independent Measures of Extremity
|
||||
|
||||
**Voting Extremity** and **Policy Extremity** are independent phenomena with different trends:
|
||||
|
||||
| Measure | 2016 | 2026 | Trend |
|
||||
|---------|------|------|-------|
|
||||
| **Voting Extremity** (margin/total) | 0.70 | 0.46 | Parliament votes more closely |
|
||||
| **Policy Extremity** (embedding distance from mainstream) | 5.65 | 4.17 | Policies are less extreme |
|
||||
|
||||
**Correlation: r ≈ 0** — these measures are statistically independent.
|
||||
|
||||
### What Each Measures
|
||||
|
||||
**Voting Extremity** = `abs(votes_for - votes_against) / total_votes`
|
||||
- 1.0 = unanimous (all votes same direction)
|
||||
- 0.0 = perfectly split (50-50)
|
||||
- Captures how divided parliament is when voting
|
||||
|
||||
**Policy Extremity** = `||embedding - mainstream_centroid||`
|
||||
- Euclidean distance in text embedding space (2560 dims)
|
||||
- Captures how far a motion is from the political center
|
||||
|
||||
### How to Measure Each
|
||||
|
||||
```python
|
||||
# Voting Extremity
|
||||
margin = abs(votes_for - votes_against)
|
||||
total = votes_for + votes_against
|
||||
voting_extremity = margin / total
|
||||
|
||||
# Policy Extremity (using text embeddings, not SVD)
|
||||
from embeddings table (qwen/qwen3-embedding-4b)
|
||||
policy_extremity = np.linalg.norm(motion_embedding - mainstream_centroid)
|
||||
```
|
||||
|
||||
### Why Use Text Embeddings (Not SVD)
|
||||
|
||||
SVD embeddings are fitted on **voting patterns**, capturing how parties vote together. They measure **voting extremity**, not **policy extremity**.
|
||||
|
||||
For policy content, use **raw text embeddings** (`embeddings` table, 2560 dimensions) which are computed from motion text only.
|
||||
|
||||
### Bipartisan Anchor Approach
|
||||
|
||||
Define the "mainstream" as the centroid of bipartisan motions (80%+ parties vote the same way):
|
||||
|
||||
```python
|
||||
# Find bipartisan motions
|
||||
bipartisan = [m for m in motions if majority_vote_pct >= 0.80]
|
||||
|
||||
# Compute mainstream centroid
|
||||
mainstream_centroid = mean([m.embedding for m in bipartisan])
|
||||
|
||||
# Measure policy extremity
|
||||
policy_extremity = ||motion.embedding - mainstream_centroid||
|
||||
```
|
||||
|
||||
## Why This Matters
|
||||
|
||||
The hypothesis "extremes became more extreme" was wrong because:
|
||||
|
||||
1. **Voting extremity increased** — parliament votes more divided now
|
||||
2. **Policy extremity decreased** — even extreme motions are closer to center
|
||||
|
||||
This means: what divides parties changed, not how radical the policies are.
|
||||
|
||||
## Quantifying Mainstream Shift
|
||||
|
||||
Using 2018 as baseline ("last normal year"):
|
||||
|
||||
| Period | Distance from 2018 | Interpretation |
|
||||
|--------|-------------------|----------------|
|
||||
| 2016-2018 | ~0.22 | Similar mainstream |
|
||||
| **2019** | **0.46** | Shift begins |
|
||||
| 2020-2026 | **~0.71** | New stable mainstream |
|
||||
|
||||
The mainstream shifted **0.71 units** after 2018 and has remained stable.
|
||||
|
||||
### Coalition Shift on Migration Policy
|
||||
|
||||
Parties that once opposed strict migration now vote for them:
|
||||
|
||||
| Party | 2016-2018 | 2025-2026 | Change |
|
||||
|-------|------------|------------|--------|
|
||||
| VVD | 100% voor | 78% voor | ↓ |
|
||||
| CDA | 100% voor | 81% voor | ↓ |
|
||||
| D66 | 100% voor | 60% voor | ↓↓ |
|
||||
| PVV | 20% voor | 56% voor | ↑↑ |
|
||||
| NSC | 0% (new) | 56% voor | new |
|
||||
| BBB | 0% (new) | 79% voor | new |
|
||||
|
||||
## When to Apply
|
||||
|
||||
- When analyzing parliamentary polarization trends
|
||||
- When comparing policy extremity across time periods
|
||||
- When studying coalition formation and party positioning
|
||||
- When testing hypotheses about political extremism
|
||||
|
||||
## Examples
|
||||
|
||||
### Correct Analysis
|
||||
```python
|
||||
# Compare voting extremity and policy extremity separately
|
||||
voting_ext = compute_voting_margin(motion)
|
||||
policy_ext = compute_embedding_distance(motion, mainstream_centroid)
|
||||
|
||||
# Plot both trends independently
|
||||
plot_trend(years, voting_ext, label="Voting Extremity")
|
||||
plot_trend(years, policy_ext, label="Policy Extremity")
|
||||
```
|
||||
|
||||
### Incorrect Analysis
|
||||
```python
|
||||
# DON'T use SVD scores to measure policy extremity
|
||||
svd_score = motion.svd_vector[0] # This measures voting pattern, not content!
|
||||
|
||||
# DO use text embeddings for policy content
|
||||
text_embedding = embeddings_table[motion.id]
|
||||
```
|
||||
|
||||
## Related Findings
|
||||
|
||||
- `svd-stability-vs-overtone-shift.md` — SVD axes measure voting structure, not semantics
|
||||
- `policy-extremity-vs-voting-extremity.md` — Initial documentation of the distinction
|
||||
|
||||
## Visualizations
|
||||
|
||||
- `docs/research/polarization_comprehensive.png` — Combined view of all metrics
|
||||
- `docs/research/mainstream_shift.png` — Mainstream shift over time
|
||||
- `docs/research/voting_vs_policy_extremity.png` — Independent trends
|
||||
@@ -0,0 +1,90 @@
|
||||
---
|
||||
module: svd
|
||||
date: 2026-04-16
|
||||
category: docs/solutions/logic-errors
|
||||
problem_type: logic_error
|
||||
component: rails_view
|
||||
severity: medium
|
||||
symptoms:
|
||||
- "SVD tab displayed VVD component 1 score of 0.108"
|
||||
- "Compass displayed VVD component 1 score of 0.335"
|
||||
- "Significant numerical discrepancy between two views showing same data"
|
||||
root_cause: scope_issue
|
||||
resolution_type: code_fix
|
||||
tags:
|
||||
- svd
|
||||
- voting-analysis
|
||||
- filter-scope
|
||||
- party-scores
|
||||
---
|
||||
|
||||
# SVD Tab Party Scores Don't Match Compass
|
||||
|
||||
## Problem
|
||||
|
||||
The SVD tab displayed VVD component 1 score as 0.108 while the compass visualization showed 0.335 — a 3x discrepancy caused by including inactive/historical MPs in the SVD calculation.
|
||||
|
||||
## Symptoms
|
||||
|
||||
- **SVD tab**: VVD comp1 score = 0.108 (incorrect)
|
||||
- **Compass**: VVD comp1 score = 0.335 (correct, verified against compass reference)
|
||||
- **Discrepancy**: ~3x difference between views
|
||||
- **Scope**: Affects all parties but most visible for VVD (82 historical MPs vs ~50 active)
|
||||
|
||||
## What Didn't Work
|
||||
|
||||
N/A — straightforward fix once root cause identified through comparison with compass implementation at `explorer.py:1473`.
|
||||
|
||||
## Solution
|
||||
|
||||
The `_get_aligned_party_scores()` function in `views/svd.py` was missing an `active_MP` filter when calculating party means for the current parliament window.
|
||||
|
||||
**Before (buggy code):**
|
||||
|
||||
```python
|
||||
def _get_aligned_party_scores(party_id: str, dimension: str, ...) -> list:
|
||||
raw_scores = execute_query(score_query, ...)
|
||||
# Missing: no active_MP filter
|
||||
return raw_scores
|
||||
```
|
||||
|
||||
**After (fixed code):**
|
||||
|
||||
```python
|
||||
def get_aligned_party_scores(party_id: str, dimension: str, ...) -> list:
|
||||
raw_scores = execute_query(score_query, ...)
|
||||
|
||||
# Filter to only active MPs (matches compass behavior at explorer.py:1473)
|
||||
active_mps = {m[0] for m in active_mp_query if m[0] is not None}
|
||||
scores = [s for s in raw_scores if s[0] in active_mps]
|
||||
|
||||
return scores
|
||||
```
|
||||
|
||||
Key changes:
|
||||
1. Extracted function to module-level for testability
|
||||
2. Added active MP filtering using the same query pattern as compass (`explorer.py:1473`)
|
||||
3. Filter ensures only MPs in current parliament window are included
|
||||
|
||||
**Verification:**
|
||||
- Without filter: VVD comp1 = 0.1083
|
||||
- With filter: VVD comp1 = 0.3366 (matches compass reference of 0.3350)
|
||||
- Test suite: 169/169 tests passing
|
||||
|
||||
## Why This Works
|
||||
|
||||
The root cause was including all 82 historical VVD MPs instead of only the active ones. The database (`data/motions.db`) contains MPs from multiple parliaments, and the `_get_aligned_party_scores()` function wasn't filtering by `active_MP`. The compass correctly applied this filter, explaining the discrepancy.
|
||||
|
||||
## Prevention
|
||||
|
||||
1. **Test suite**: Comprehensive tests in `tests/svd_test.py` covering alignment calculations with active MP filtering
|
||||
2. **Cross-view validation**: Compare SVD and compass scores for each party — assert values match within tolerance
|
||||
3. **Query pattern documentation**: All score queries must include `active_MP` filter when calculating party means
|
||||
4. **Code review checklist**: Require active_MP filter for any new score calculation queries
|
||||
5. **Automated regression**: Add CI check that runs comparison between SVD tab and compass for all parties
|
||||
|
||||
## Related Issues
|
||||
|
||||
- `docs/solutions/logic-errors/svd-theme-divergence-from-party-positions.md` — Related domain issue: SVD scores not matching actual party positions
|
||||
- `docs/solutions/logic-errors/svd-component-labels-mismatch.md` — Related theme: Labels/data alignment mismatches
|
||||
- `docs/solutions/best-practices/svd-labels-voting-patterns-not-semantics.md` — Core principle: SVD captures voting patterns, verify against actual voting data
|
||||
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: "SVD axis labels: derive left/right from runtime flip, not static fields"
|
||||
date: 2026-04-12
|
||||
module: analysis
|
||||
problem_type: ui_bug
|
||||
component: analysis
|
||||
symptoms:
|
||||
- "SVD axis labels showed wrong orientation for components where runtime flip differed from static flip value"
|
||||
- "Right-wing parties (PVV, FVD) appeared on the LEFT side of axes despite being canonical right parties"
|
||||
- "Components 3-10 in tijdtraject view showed scores incomparable with single-window view"
|
||||
root_cause: logic_error
|
||||
resolution_type: code_fix
|
||||
severity: high
|
||||
tags:
|
||||
- svd
|
||||
- axis-labels
|
||||
- pole-labels
|
||||
- parliamentary-explorer
|
||||
- left-right-axis
|
||||
- procrustes
|
||||
---
|
||||
|
||||
# SVD Axis Labels: Derive Left/Right from Runtime Flip, Not Static Fields
|
||||
|
||||
## Problem
|
||||
SVD axis pole labels showed wrong orientation after the runtime flip mechanism was applied. Right-wing parties appeared on the LEFT side of axes despite being canonical right parties. Additionally, components 3-10 in the tijdtraject (time trajectory) view showed party scores that were incomparable with the single-window view.
|
||||
|
||||
## Symptoms
|
||||
- Axis labels like "← PVV en FVD — soevereiniteit en anti-establishment" appeared on the left side when they should be on the right
|
||||
- The flip mechanism (`compute_flip_direction`) correctly negated party scores, but labels were tied to static pre-computed fields
|
||||
- Components 3-10 in `build_svd_components_tab` used Procrustes-aligned scores that were rotated by the component 1-2 alignment, making them meaningless
|
||||
|
||||
## What Didn't Work
|
||||
The 2026-04-05 fix added static `left_pole`/`right_pole` fields to `SVD_THEMES`, pre-computed based on the static `flip` value in config. This failed because:
|
||||
|
||||
1. `compute_flip_direction()` determines flip at **runtime** by comparing mean scores of canonical right vs left parties against actual voting data
|
||||
2. The static `flip` value in config could differ from the runtime result when voting patterns shift
|
||||
3. When runtime flip differed from the static config, the pre-computed `left_pole`/`right_pole` pointed to the wrong side
|
||||
|
||||
### Root Cause Detail: Dynamic Flip Override
|
||||
|
||||
The bug was compounded by `explorer.py` lines 2636-2649, where `compute_flip_direction()` dynamically overwrites `SVD_THEMES[comp]["flip"]` for **all** components (1-10) at runtime:
|
||||
|
||||
```python
|
||||
# explorer.py lines 2677-2690
|
||||
for comp in range(1, 11):
|
||||
flip = compute_flip_direction(comp, party_scores)
|
||||
if comp in SVD_THEMES:
|
||||
SVD_THEMES[comp]["flip"] = flip
|
||||
```
|
||||
|
||||
When PVV/FVD had negative scores on component 2:
|
||||
1. `compute_flip_direction(2, party_scores)` returned `True` (right parties have lower mean)
|
||||
2. `SVD_THEMES[2]["flip"]` was overwritten from `False` to `True`
|
||||
3. With `flip=True`, scores were negated (PVV/FVD became positive → appeared on RIGHT)
|
||||
4. But the **label derivation logic** (`explorer.py` lines 954-957, 1073-1077) was backwards:
|
||||
```python
|
||||
left_label = theme.get("left_pole", pos_pole if flip else neg_pole)
|
||||
right_label = theme.get("right_pole", neg_pole if flip else pos_pole)
|
||||
```
|
||||
When `flip=True`, `left_label` was set to `pos_pole` (which described PVV/FVD), but PVV/FVD were now on the **RIGHT** side after negation.
|
||||
|
||||
This meant labels were misaligned with the actual data whenever the runtime flip differed from the static config flip.
|
||||
|
||||
## Solution
|
||||
|
||||
### Bug 1: Label derivation
|
||||
|
||||
Removed static `left_pole`/`right_pole` from all 10 `SVD_THEMES` entries in `analysis/config.py`. Labels are now always derived at render time from `positive_pole`/`negative_pole` and the runtime flip direction:
|
||||
|
||||
```python
|
||||
# analysis/svd_labels.py — derive left/right from runtime flip
|
||||
if flip:
|
||||
left_pole, right_pole = pos_pole, neg_pole # flip=True: positive on left
|
||||
else:
|
||||
left_pole, right_pole = neg_pole, pos_pole # flip=False: negative on left
|
||||
```
|
||||
|
||||
The key insight: **`negative_pole` always describes what's on the LEFT, `positive_pole` always describes what's on the RIGHT** — regardless of flip. The flip only affects which raw SVD direction maps to left vs right.
|
||||
|
||||
### Bug 2: Score mismatch in tijdtraject view
|
||||
|
||||
Changed components 3-10 in `build_svd_components_tab` from `load_party_scores_all_windows_aligned()` to `load_party_scores_all_windows()`:
|
||||
|
||||
```python
|
||||
# explorer.py — components 3-10 use per-window scores (not Procrustes-aligned)
|
||||
party_scores_by_window = load_party_scores_all_windows(db_path, all_windows)
|
||||
```
|
||||
|
||||
**Why:** Procrustes alignment rotates the full 50-dim vector space to align components 1-2 across windows, but this also transforms components 3-10, making their scores incomparable with the single-window view. Per-window flip computation already handles orientation alignment for components 3-10.
|
||||
|
||||
### Bug 3: Config as canonical SVD_THEMES source
|
||||
|
||||
Updated `analysis/svd_labels.py` to prefer `analysis.config` as the canonical source for `SVD_THEMES`, falling back to `explorer` only when config is unavailable. Config is intentionally lightweight and free of heavy runtime dependencies (duckdb, plotly).
|
||||
|
||||
### Prevention: Tests added
|
||||
|
||||
Added `tests/test_svd_axis_alignment.py` with 3 tests:
|
||||
- `test_right_wing_on_right_all_components`: Verifies canonical right parties appear on right for all 10 components
|
||||
- `test_label_derivation_matches_fallback`: Verifies label derivation logic
|
||||
- `test_config_no_deprecated_fields`: Asserts no `left_pole`/`right_pole` in config
|
||||
|
||||
Run with: `.venv/bin/python -m pytest tests/test_svd_axis_alignment.py -v`
|
||||
|
||||
## Why This Works
|
||||
The flip direction is determined by comparing canonical right vs left party average scores against actual voting data. The label derivation follows a simple rule: `negative_pole` = left, `positive_pole` = right. Since the flip operation moves the canonical right parties to the positive side, the labels always match.
|
||||
|
||||
For components 3-10, per-window scores are computed independently with per-window flip, so they remain comparable with single-window views. Procrustes only needs to align components 1-2 (the political compass axes).
|
||||
|
||||
## Prevention
|
||||
- Never add static `left_pole`/`right_pole` fields to `SVD_THEMES` — derive them at render time
|
||||
- Run `tests/test_svd_axis_alignment.py` after any SVD recomputation
|
||||
- Components 3-10 in tijdtraject view must use `load_party_scores_all_windows()`, not the aligned variant
|
||||
- The key invariant: `negative_pole` = LEFT, `positive_pole` = RIGHT — flip only determines which raw direction maps to which side
|
||||
|
||||
## Related Files
|
||||
- `analysis/config.py` — SVD_THEMES (no `left_pole`/`right_pole`)
|
||||
- `analysis/svd_labels.py` — `_get_svd_themes()` preferring config source
|
||||
- `explorer.py` — label derivation in trajectory rendering, component 3-10 scoring fix
|
||||
- `tests/test_svd_axis_alignment.py` — new tests validating alignment
|
||||
- `scripts/validate_svd_themes.py` — validation hook (updated to not expect `left_pole`/`right_pole`)
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "Trajectories diagnostic script produced false alarm due to mocked empty data"
|
||||
date: 2026-03-31
|
||||
module: explorer
|
||||
problem_type: workflow_issue
|
||||
component: explorer
|
||||
severity: medium
|
||||
symptoms:
|
||||
- "Diagnostic JSON showed party_map_count: 0 for all scenarios"
|
||||
- "Trajectories appeared broken based on diagnostic output"
|
||||
- "Mindmodel anti-pattern flagged compute_party_coords party_map mismatch"
|
||||
root_cause: logic_error
|
||||
resolution_type: process_fix
|
||||
tags:
|
||||
- trajectories
|
||||
- diagnostics
|
||||
- false-alarm
|
||||
- mocking
|
||||
- testing
|
||||
---
|
||||
|
||||
# Trajectories Diagnostic Script Produced False Alarm
|
||||
|
||||
## Problem
|
||||
|
||||
The `scripts/diagnose_trajectories_cli.py` diagnostic script reported `party_map_count: 0` across all scenarios, suggesting that party trajectories in the Explorer were broken. This triggered an investigation, a mindmodel anti-pattern flag, and multiple design docs for fixing "missing trajectories."
|
||||
|
||||
## Symptoms
|
||||
|
||||
- `thoughts/shared/diagnostics/2026-03-31-trajectories-diagnostics.json` showed `party_map_count: 0` in every scenario
|
||||
- The mindmodel generation process flagged `explorer_helpers.py:compute_party_coords` as a top anti-pattern (party_map key/value mismatch hypothesis)
|
||||
- Multiple implementation plans were drafted to add fallback rendering, MP-level trajectories, and debug instrumentation
|
||||
|
||||
## Root Cause
|
||||
|
||||
The diagnostic script itself was the bug. It artificially passed `load_party_map_ret={}` (empty dict) in **all** scenarios, creating a false alarm that had no relation to production behavior.
|
||||
|
||||
When tested with real data:
|
||||
- `party_map` has **1,036 entries** (not 0)
|
||||
- `select_trajectory_plot_data` returns `trace_count=6` with real data
|
||||
- Annual view shows CDA, D66, VVD traces; quarterly view shows 6 party traces
|
||||
- Trajectories work correctly — no production bug exists
|
||||
|
||||
The anti-pattern detected (`compute_party_coords` party_map mismatch) was also a false alarm: `svd_vectors` entity_ids are ALL MP names, never party names (no `entity_type='party'` rows exist in the database).
|
||||
|
||||
## What Didn't Work
|
||||
|
||||
- Trusting the diagnostic script output without validating against real data
|
||||
- Investigating based on the JSON artifact alone
|
||||
- The `2026-03-31-trajectories-diagnostics.json` was created by a script that passes `load_party_map_ret={}` artificially
|
||||
|
||||
## Solution
|
||||
|
||||
**No production code changes needed** — trajectories work correctly. The fix is to the diagnostic script and the process:
|
||||
|
||||
1. **Fix `scripts/diagnose_trajectories_cli.py`** to use real data paths (`data/motions.db`) and real `load_party_map` / `load_positions` calls instead of mocking everything to empty
|
||||
2. **Re-run the fixed diagnostic script** to produce a correct `trajectories-diagnostics.json` artifact
|
||||
3. **Remove the incorrect anti-pattern** from the mindmodel manifest (the party_map mismatch hypothesis does not apply since no party-level entity_ids exist in `svd_vectors`)
|
||||
|
||||
## Prevention
|
||||
|
||||
1. **Never trust a diagnostic that mocks data to empty without a real-data validation step**
|
||||
2. **Always compare diagnostic output against a known-good baseline** — run the same checks with real data before concluding there's a bug
|
||||
3. **Diagnostic scripts should use real data paths by default** — if mocking is needed for unit tests, keep it in tests, not in diagnostic CLI scripts
|
||||
4. **Verify DB state directly** before investigating based on intermediary artifacts:
|
||||
```python
|
||||
# Quick sanity check
|
||||
from explorer import load_positions, load_party_map
|
||||
positions_by_window, _ = load_positions("data/motions.db", "annual")
|
||||
party_map = load_party_map("data/motions.db")
|
||||
assert len(party_map) > 0, "party_map is empty — investigate data pipeline"
|
||||
```
|
||||
5. **Add an integration test** that calls `select_trajectory_plot_data` with real DB data and asserts `trace_count > 0`
|
||||
|
||||
## Related
|
||||
|
||||
- `docs/solutions/best-practices/refactoring-streamlit-data-loading.md` — Data loading patterns for explorer
|
||||
- `thoughts/shared/plans/2026-03-31-debug-trajectories-not-showing.md` — Original debugging plan (based on false alarm)
|
||||
- `thoughts/shared/designs/2026-03-31-diagnose-no-plot-trajectories-design.md` — Original design doc (based on false alarm)
|
||||
Reference in New Issue
Block a user