parent
bed776f295
commit
a1c3e92fab
@ -0,0 +1,879 @@ |
||||
# SVD Label Unification Implementation Plan |
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
||||
|
||||
**Goal:** Unify SVD component labels into a single source of truth and automatically compute axis flip directions so right-wing parties consistently appear on the right side of all SVD component axes. |
||||
|
||||
**Architecture:** Create a new `analysis/svd_labels.py` module that imports `SVD_THEMES` from explorer.py at runtime and provides helper functions for label lookup and flip direction computation. Update `axis_classifier.py` to use this module instead of hardcoded labels. Add 1D party position charts for components 3-10 in the SVD Components tab. |
||||
|
||||
**Tech Stack:** Python, Streamlit, NumPy, DuckDB |
||||
|
||||
--- |
||||
|
||||
## File Structure |
||||
|
||||
| File | Responsibility | |
||||
|------|---------------| |
||||
| `analysis/svd_labels.py` | **NEW** - Unified label system with auto-flip computation | |
||||
| `analysis/axis_classifier.py` | Remove `_LABELS`, import from svd_labels | |
||||
| `explorer.py` | Remove fallback labels, add 1D charts for components 3-10 | |
||||
| `tests/test_svd_labels.py` | **NEW** - Tests for svd_labels module | |
||||
| `tests/test_axis_label_fallback.py` | Update to use new label system | |
||||
| `tests/test_political_compass.py` | Update assertions for new labels | |
||||
|
||||
--- |
||||
|
||||
## Task 1: Create `analysis/svd_labels.py` with core functions |
||||
|
||||
**Files:** |
||||
- Create: `analysis/svd_labels.py` |
||||
- Test: `tests/test_svd_labels.py` |
||||
|
||||
- [ ] **Step 1: Write the failing test for `get_svd_label`** |
||||
|
||||
```python |
||||
# tests/test_svd_labels.py |
||||
"""Tests for analysis/svd_labels module.""" |
||||
|
||||
def test_get_svd_label_returns_correct_label(): |
||||
"""Test that get_svd_label returns the correct label for each component.""" |
||||
from analysis.svd_labels import get_svd_label |
||||
|
||||
# Component 1 should return EU-integratie label |
||||
label1 = get_svd_label(1) |
||||
assert "EU-integratie" in label1 or "Nationalisme" in label1 |
||||
|
||||
# Component 2 should return Populistisch label |
||||
label2 = get_svd_label(2) |
||||
assert "Populistisch" in label2 or "Institutioneel" in label2 |
||||
|
||||
# Component 3 should return Verzorgingsstaat label |
||||
label3 = get_svd_label(3) |
||||
assert "Verzorgingsstaat" in label3 or "Marktwerking" in label3 |
||||
``` |
||||
|
||||
- [ ] **Step 2: Run test to verify it fails** |
||||
|
||||
Run: `uv run pytest tests/test_svd_labels.py::test_get_svd_label_returns_correct_label -v` |
||||
Expected: FAIL with "ModuleNotFoundError: No module named 'analysis.svd_labels'" |
||||
|
||||
- [ ] **Step 3: Create `analysis/svd_labels.py` with `get_svd_label` function** |
||||
|
||||
```python |
||||
# analysis/svd_labels.py |
||||
"""Unified SVD component labels and automatic flip direction computation. |
||||
|
||||
This module provides a single source of truth for SVD component labels, |
||||
deriving them from SVD_THEMES in explorer.py. It also computes flip |
||||
directions automatically based on party centroids. |
||||
""" |
||||
|
||||
import logging |
||||
from typing import Dict, List, Optional, Tuple |
||||
|
||||
_logger = logging.getLogger(__name__) |
||||
|
||||
# Canonical party sets for orientation |
||||
# Right-wing parties that should appear on the right side of axes |
||||
RIGHT_PARTIES = { |
||||
"PVV", "VVD", "FVD", "BBB", "JA21", |
||||
"Nieuw Sociaal Contract", "SGP", "CDA", "ChristenUnie" |
||||
} |
||||
|
||||
# Left-wing parties that should appear on the left side of axes |
||||
LEFT_PARTIES = { |
||||
"SP", "PvdA", "GL", "GroenLinks", "GroenLinks-PvdA", |
||||
"DENK", "PvdD", "Volt" |
||||
} |
||||
|
||||
# Cache for SVD_THEMES to avoid repeated imports |
||||
_svd_themes_cache: Optional[Dict[int, Dict[str, str]]] = None |
||||
|
||||
|
||||
def _get_svd_themes() -> Dict[int, Dict[str, str]]: |
||||
"""Lazy import SVD_THEMES from explorer.py to avoid circular imports. |
||||
|
||||
Returns: |
||||
Dict mapping component number to theme dict with keys: |
||||
- label: Short label for the component |
||||
- explanation: Detailed explanation |
||||
- positive_pole: Description of positive pole |
||||
- negative_pole: Description of negative pole |
||||
- flip: Whether to flip the axis |
||||
""" |
||||
global _svd_themes_cache |
||||
if _svd_themes_cache is not None: |
||||
return _svd_themes_cache |
||||
|
||||
try: |
||||
# Import at runtime to avoid circular dependency |
||||
# explorer.py imports from analysis/ but we don't import from explorer.py |
||||
# at module load time |
||||
import importlib.util |
||||
import os |
||||
|
||||
# Find explorer.py |
||||
explorer_path = os.path.join(os.path.dirname(__file__), "..", "explorer.py") |
||||
if not os.path.exists(explorer_path): |
||||
_logger.warning("explorer.py not found at %s", explorer_path) |
||||
return {} |
||||
|
||||
spec = importlib.util.spec_from_file_location("explorer", explorer_path) |
||||
if spec is None or spec.loader is None: |
||||
_logger.warning("Could not load spec from explorer.py") |
||||
return {} |
||||
|
||||
explorer_module = importlib.util.module_from_spec(spec) |
||||
spec.loader.exec_module(explorer_module) |
||||
|
||||
# Get SVD_THEMES from the build_svd_components_tab function's local scope |
||||
# This is a bit hacky but avoids importing the entire Streamlit app |
||||
# We'll need to refactor explorer.py to export SVD_THEMES at module level |
||||
# For now, we'll define it here as a fallback |
||||
|
||||
# Fallback: define SVD_THEMES here if we can't import from explorer |
||||
_svd_themes_cache = {} |
||||
return _svd_themes_cache |
||||
|
||||
except Exception as e: |
||||
_logger.exception("Failed to load SVD_THEMES from explorer.py: %s", e) |
||||
return {} |
||||
|
||||
|
||||
def get_svd_label(component: int) -> str: |
||||
"""Get short label for SVD component. |
||||
|
||||
Args: |
||||
component: SVD component number (1-indexed) |
||||
|
||||
Returns: |
||||
Short label string (e.g., 'EU-integratie–Nationalisme') |
||||
|
||||
Raises: |
||||
ValueError: If component < 1 |
||||
""" |
||||
if component < 1: |
||||
raise ValueError(f"Component must be >= 1, got {component}") |
||||
|
||||
themes = _get_svd_themes() |
||||
if component in themes: |
||||
return themes[component].get("label", f"As {component}") |
||||
|
||||
# Fallback labels for components 1-3 (most commonly used) |
||||
fallback_labels = { |
||||
1: "EU-integratie–Nationalisme", |
||||
2: "Populistisch–Institutioneel", |
||||
3: "Verzorgingsstaat–Marktwerking", |
||||
} |
||||
return fallback_labels.get(component, f"As {component}") |
||||
|
||||
|
||||
def get_svd_theme(component: int) -> Dict[str, str]: |
||||
"""Get full theme dict for SVD component. |
||||
|
||||
Args: |
||||
component: SVD component number (1-indexed) |
||||
|
||||
Returns: |
||||
Dict with keys: label, explanation, positive_pole, negative_pole, flip |
||||
""" |
||||
if component < 1: |
||||
raise ValueError(f"Component must be >= 1, got {component}") |
||||
|
||||
themes = _get_svd_themes() |
||||
if component in themes: |
||||
return themes[component] |
||||
|
||||
# Return minimal fallback |
||||
return { |
||||
"label": get_svd_label(component), |
||||
"explanation": "", |
||||
"positive_pole": "", |
||||
"negative_pole": "", |
||||
"flip": False, |
||||
} |
||||
|
||||
|
||||
def compute_flip_direction( |
||||
component: int, |
||||
party_scores: Dict[str, List[float]] |
||||
) -> bool: |
||||
"""Compute flip direction so right parties appear on the right side. |
||||
|
||||
Args: |
||||
component: SVD component number (1-indexed) |
||||
party_scores: Dict mapping party name to list of scores per component |
||||
(party_scores[party][0] is score for component 1, etc.) |
||||
|
||||
Returns: |
||||
True if axis should be flipped so right parties are on right. |
||||
False otherwise. |
||||
""" |
||||
if component < 1: |
||||
return False |
||||
|
||||
idx = component - 1 # Convert to 0-indexed |
||||
|
||||
right_scores = [] |
||||
left_scores = [] |
||||
|
||||
for party, scores in party_scores.items(): |
||||
if len(scores) <= idx: |
||||
continue |
||||
|
||||
score = scores[idx] |
||||
if party in RIGHT_PARTIES: |
||||
right_scores.append(score) |
||||
elif party in LEFT_PARTIES: |
||||
left_scores.append(score) |
||||
|
||||
if not right_scores or not left_scores: |
||||
return False # Default: no flip if insufficient data |
||||
|
||||
right_mean = sum(right_scores) / len(right_scores) |
||||
left_mean = sum(left_scores) / len(left_scores) |
||||
|
||||
# Flip if right parties have lower mean (they're on the left) |
||||
return right_mean < left_mean |
||||
|
||||
|
||||
def get_fallback_labels() -> Tuple[str, str]: |
||||
"""Get fallback labels for x and y axes (components 1 and 2). |
||||
|
||||
Returns: |
||||
Tuple of (x_label, y_label) |
||||
""" |
||||
return (get_svd_label(1), get_svd_label(2)) |
||||
``` |
||||
|
||||
- [ ] **Step 4: Run test to verify it passes** |
||||
|
||||
Run: `uv run pytest tests/test_svd_labels.py::test_get_svd_label_returns_correct_label -v` |
||||
Expected: PASS |
||||
|
||||
- [ ] **Step 5: Write test for `compute_flip_direction`** |
||||
|
||||
```python |
||||
# tests/test_svd_labels.py (append) |
||||
|
||||
def test_compute_flip_direction_right_on_left(): |
||||
"""Test that flip is True when right parties are on the left.""" |
||||
from analysis.svd_labels import compute_flip_direction |
||||
|
||||
# Right parties have negative scores (on left), left parties have positive |
||||
party_scores = { |
||||
"VVD": [-0.5, 0.0], # Right party, component 1 score = -0.5 |
||||
"PVV": [-0.8, 0.0], # Right party |
||||
"SP": [0.6, 0.0], # Left party, component 1 score = 0.6 |
||||
"DENK": [0.4, 0.0], # Left party |
||||
} |
||||
|
||||
# Component 1: right_mean = -0.65, left_mean = 0.5 |
||||
# right_mean < left_mean, so flip = True |
||||
assert compute_flip_direction(1, party_scores) is True |
||||
|
||||
|
||||
def test_compute_flip_direction_right_on_right(): |
||||
"""Test that flip is False when right parties are already on the right.""" |
||||
from analysis.svd_labels import compute_flip_direction |
||||
|
||||
# Right parties have positive scores (on right), left parties have negative |
||||
party_scores = { |
||||
"VVD": [0.5, 0.0], # Right party, component 1 score = 0.5 |
||||
"PVV": [0.8, 0.0], # Right party |
||||
"SP": [-0.6, 0.0], # Left party |
||||
"DENK": [-0.4, 0.0], # Left party |
||||
} |
||||
|
||||
# Component 1: right_mean = 0.65, left_mean = -0.5 |
||||
# right_mean > left_mean, so flip = False |
||||
assert compute_flip_direction(1, party_scores) is False |
||||
|
||||
|
||||
def test_compute_flip_direction_insufficient_data(): |
||||
"""Test that flip is False when there's insufficient data.""" |
||||
from analysis.svd_labels import compute_flip_direction |
||||
|
||||
# No right parties in data |
||||
party_scores = { |
||||
"SP": [0.6, 0.0], |
||||
"DENK": [0.4, 0.0], |
||||
} |
||||
|
||||
assert compute_flip_direction(1, party_scores) is False |
||||
|
||||
# No left parties in data |
||||
party_scores = { |
||||
"VVD": [0.5, 0.0], |
||||
"PVV": [0.8, 0.0], |
||||
} |
||||
|
||||
assert compute_flip_direction(1, party_scores) is False |
||||
``` |
||||
|
||||
- [ ] **Step 6: Run tests for `compute_flip_direction`** |
||||
|
||||
Run: `uv run pytest tests/test_svd_labels.py -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 7: Commit** |
||||
|
||||
```bash |
||||
git add analysis/svd_labels.py tests/test_svd_labels.py |
||||
git commit -m "feat: add svd_labels module for unified label system" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 2: Refactor explorer.py to export SVD_THEMES at module level |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` (move SVD_THEMES to module level) |
||||
- Modify: `analysis/svd_labels.py` (update to import from explorer) |
||||
|
||||
- [ ] **Step 1: Move SVD_THEMES from function to module level in explorer.py** |
||||
|
||||
Find the `SVD_THEMES` dict inside `build_svd_components_tab` function (around line 2459) and move it to module level (near the top of the file, after imports). |
||||
|
||||
```python |
||||
# explorer.py (near top of file, after imports and constants) |
||||
|
||||
# Political polarisation themes per SVD component (1-indexed, window=2025) |
||||
# Produced by per-axis analysis of all 10 unique top motions (zero cross-axis overlap). |
||||
# This is the canonical source of truth for SVD component labels. |
||||
SVD_THEMES: dict[int, dict[str, str]] = { |
||||
1: { |
||||
"label": "EU-integratie en internationalisme versus nationalisme", |
||||
# ... rest of the dict |
||||
}, |
||||
# ... rest of components |
||||
} |
||||
``` |
||||
|
||||
- [ ] **Step 2: Update `analysis/svd_labels.py` to import SVD_THEMES properly** |
||||
|
||||
```python |
||||
# analysis/svd_labels.py (update _get_svd_themes function) |
||||
|
||||
def _get_svd_themes() -> Dict[int, Dict[str, str]]: |
||||
"""Lazy import SVD_THEMES from explorer.py to avoid circular imports.""" |
||||
global _svd_themes_cache |
||||
if _svd_themes_cache is not None: |
||||
return _svd_themes_cache |
||||
|
||||
try: |
||||
# Import at runtime to avoid circular dependency at module load time |
||||
# explorer.py imports from analysis/ but we delay our import |
||||
import explorer |
||||
_svd_themes_cache = explorer.SVD_THEMES |
||||
return _svd_themes_cache |
||||
except ImportError as e: |
||||
_logger.warning("Could not import explorer.SVD_THEMES: %s", e) |
||||
return {} |
||||
except Exception as e: |
||||
_logger.exception("Failed to load SVD_THEMES from explorer.py: %s", e) |
||||
return {} |
||||
``` |
||||
|
||||
- [ ] **Step 3: Run tests to verify nothing broke** |
||||
|
||||
Run: `uv run pytest tests/test_svd_labels.py -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 4: Commit** |
||||
|
||||
```bash |
||||
git add explorer.py analysis/svd_labels.py |
||||
git commit -m "refactor: move SVD_THEMES to module level for import" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 3: Update `axis_classifier.py` to use svd_labels |
||||
|
||||
**Files:** |
||||
- Modify: `analysis/axis_classifier.py` |
||||
- Modify: `tests/test_axis_label_fallback.py` |
||||
|
||||
- [ ] **Step 1: Write test for updated `display_label_for_modal`** |
||||
|
||||
```python |
||||
# tests/test_axis_label_fallback.py (update existing tests) |
||||
|
||||
def test_display_label_for_modal_uses_svd_themes(): |
||||
"""Test that display_label_for_modal uses SVD_THEMES for fallback labels.""" |
||||
from analysis.axis_classifier import display_label_for_modal |
||||
|
||||
# None should return fallback from SVD_THEMES |
||||
x_label = display_label_for_modal(None, "x") |
||||
y_label = display_label_for_modal(None, "y") |
||||
|
||||
# Should return component 1 and 2 labels from SVD_THEMES |
||||
assert "EU-integratie" in x_label or "Nationalisme" in x_label |
||||
assert "Populistisch" in y_label or "Institutioneel" in y_label |
||||
|
||||
|
||||
def test_display_label_for_modal_maps_as_labels(): |
||||
"""Test that 'As 1' and 'As 2' are mapped to semantic labels.""" |
||||
from analysis.axis_classifier import display_label_for_modal |
||||
|
||||
x_label = display_label_for_modal("As 1", "x") |
||||
y_label = display_label_for_modal("As 2", "y") |
||||
|
||||
# Should return component 1 and 2 labels |
||||
assert "EU-integratie" in x_label or "Nationalisme" in x_label |
||||
assert "Populistisch" in y_label or "Institutioneel" in y_label |
||||
``` |
||||
|
||||
- [ ] **Step 2: Run test to verify it fails** |
||||
|
||||
Run: `uv run pytest tests/test_axis_label_fallback.py -v` |
||||
Expected: Some tests FAIL (current implementation returns hardcoded labels) |
||||
|
||||
- [ ] **Step 3: Update `axis_classifier.py` to use svd_labels** |
||||
|
||||
```python |
||||
# analysis/axis_classifier.py (update the module) |
||||
|
||||
# Remove the hardcoded _LABELS dict (lines 25-35) |
||||
# Replace with imports from svd_labels |
||||
|
||||
# At the top of the file, after imports: |
||||
from analysis.svd_labels import get_svd_label, get_fallback_labels |
||||
|
||||
# Remove _LABELS dict entirely |
||||
|
||||
# Update display_label_for_modal function (lines 42-55): |
||||
def display_label_for_modal(modal_label: Optional[str], axis: str) -> str: |
||||
"""Return a user-facing axis label for a modal/internal label. |
||||
|
||||
Maps numeric fallback names 'As 1' / 'Stempatroon As 1' to the |
||||
semantic labels from SVD_THEMES. Any other label is returned unchanged. |
||||
None is treated as the semantic fallback for the axis. |
||||
""" |
||||
if modal_label is None: |
||||
# Fallback to component 1 (x) or 2 (y) |
||||
comp = 1 if axis == "x" else 2 |
||||
return get_svd_label(comp) |
||||
|
||||
# Map "As 1" / "As 2" to semantic labels |
||||
if axis == "x" and modal_label in ("As 1", "Stempatroon As 1"): |
||||
return get_svd_label(1) |
||||
if axis == "y" and modal_label in ("As 2", "Stempatroon As 2"): |
||||
return get_svd_label(2) |
||||
|
||||
return modal_label |
||||
``` |
||||
|
||||
- [ ] **Step 4: Update `_INTERPRETATION_TEMPLATES` to use svd_labels** |
||||
|
||||
The interpretation templates still need to reference the correct labels. Keep them as-is for now since they're used for motion classification, not display. |
||||
|
||||
- [ ] **Step 5: Update `_MOTION_LABEL_TEMPLATE_KEY` to use svd_labels** |
||||
|
||||
Keep the mapping as-is for now since it's used for motion classification. |
||||
|
||||
- [ ] **Step 6: Update `_KEYWORDS` to use svd_labels** |
||||
|
||||
Keep the keywords as-is for now since they're used for motion classification. |
||||
|
||||
- [ ] **Step 7: Update fallback label references in classify_axes** |
||||
|
||||
Find lines 610 and 615 where `_LABELS["fallback_x"]` and `_LABELS["fallback_y"]` are used and replace: |
||||
|
||||
```python |
||||
# Before: |
||||
x_lbl = _LABELS["fallback_x"] |
||||
y_lbl = _LABELS["fallback_y"] |
||||
|
||||
# After: |
||||
x_lbl, y_lbl = get_fallback_labels() |
||||
``` |
||||
|
||||
- [ ] **Step 8: Run tests to verify they pass** |
||||
|
||||
Run: `uv run pytest tests/test_axis_label_fallback.py tests/test_political_compass.py -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 9: Commit** |
||||
|
||||
```bash |
||||
git add analysis/axis_classifier.py tests/test_axis_label_fallback.py |
||||
git commit -m "refactor: use svd_labels for axis labels in axis_classifier" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 4: Update explorer.py to use svd_labels for fallback labels |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` |
||||
|
||||
- [ ] **Step 1: Find and update fallback label references in explorer.py** |
||||
|
||||
Find lines 1440-1441 where hardcoded fallback labels are used: |
||||
|
||||
```python |
||||
# Before: |
||||
_x_label = _raw_x or "EU-integratie–Nationalisme" |
||||
_y_label = _raw_y or "Populistisch–Institutioneel" |
||||
|
||||
# After: |
||||
from analysis.svd_labels import get_fallback_labels |
||||
_x_fallback, _y_fallback = get_fallback_labels() |
||||
_x_label = _raw_x or _x_fallback |
||||
_y_label = _raw_y or _y_fallback |
||||
``` |
||||
|
||||
- [ ] **Step 2: Run tests to verify nothing broke** |
||||
|
||||
Run: `uv run pytest tests/test_political_compass.py -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 3: Commit** |
||||
|
||||
```bash |
||||
git add explorer.py |
||||
git commit -m "refactor: use svd_labels for fallback labels in explorer" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 5: Add 1D party position charts for components 3-10 |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` |
||||
- Test: `tests/test_explorer_chart.py` |
||||
|
||||
- [ ] **Step 1: Write test for 1D party position chart** |
||||
|
||||
```python |
||||
# tests/test_explorer_chart.py (append) |
||||
|
||||
def test_render_party_axis_chart_1d_renders(): |
||||
"""Test that _render_party_axis_chart_1d renders a figure.""" |
||||
from explorer import _render_party_axis_chart_1d |
||||
|
||||
party_coords = { |
||||
"VVD": (0.5,), |
||||
"SP": (-0.6,), |
||||
"PVV": (0.8,), |
||||
"DENK": (-0.4,), |
||||
} |
||||
|
||||
theme = { |
||||
"label": "Test Component", |
||||
"positive_pole": "Positive", |
||||
"negative_pole": "Negative", |
||||
"flip": False, |
||||
} |
||||
|
||||
fig = _render_party_axis_chart_1d(party_coords, 3, theme) |
||||
assert fig is not None |
||||
# Check that figure has traces |
||||
assert len(fig.data) > 0 |
||||
``` |
||||
|
||||
- [ ] **Step 2: Run test to verify it fails** |
||||
|
||||
Run: `uv run pytest tests/test_explorer_chart.py::test_render_party_axis_chart_1d_renders -v` |
||||
Expected: FAIL with "cannot import name '_render_party_axis_chart_1d'" |
||||
|
||||
- [ ] **Step 3: Implement `_render_party_axis_chart_1d` function in explorer.py** |
||||
|
||||
Add this function near `_render_party_axis_chart` (around line 2900): |
||||
|
||||
```python |
||||
# explorer.py (add new function) |
||||
|
||||
def _render_party_axis_chart_1d( |
||||
party_coords: dict, |
||||
component: int, |
||||
theme: dict, |
||||
bootstrap_data: Optional[Dict] = None, |
||||
) -> go.Figure: |
||||
"""Render a 1D party position chart for a single SVD component. |
||||
|
||||
Args: |
||||
party_coords: Dict mapping party name to (score,) tuple |
||||
component: SVD component number (1-indexed) |
||||
theme: Dict with label, positive_pole, negative_pole, flip |
||||
bootstrap_data: Optional bootstrap confidence intervals |
||||
|
||||
Returns: |
||||
Plotly Figure object |
||||
""" |
||||
import plotly.graph_objects as go |
||||
import numpy as np |
||||
|
||||
if not party_coords: |
||||
fig = go.Figure() |
||||
fig.add_annotation( |
||||
text="Geen partijposities beschikbaar", |
||||
xref="paper", yref="paper", |
||||
x=0.5, y=0.5, showarrow=False, |
||||
font=dict(size=14) |
||||
) |
||||
return fig |
||||
|
||||
# Extract scores and parties |
||||
parties = list(party_coords.keys()) |
||||
scores = [coords[0] for coords in party_coords.values()] |
||||
|
||||
# Apply flip if needed |
||||
flip = theme.get("flip", False) |
||||
if flip: |
||||
scores = [-s for s in scores] |
||||
|
||||
# Get party colors |
||||
party_colors = [PARTY_COLOURS.get(p, "#9E9E9E") for p in parties] |
||||
|
||||
# Create horizontal bar chart |
||||
fig = go.Figure() |
||||
|
||||
# Sort by score for better visualization |
||||
sorted_indices = np.argsort(scores) |
||||
sorted_parties = [parties[i] for i in sorted_indices] |
||||
sorted_scores = [scores[i] for i in sorted_indices] |
||||
sorted_colors = [party_colors[i] for i in sorted_indices] |
||||
|
||||
fig.add_trace(go.Bar( |
||||
y=sorted_parties, |
||||
x=sorted_scores, |
||||
orientation='h', |
||||
marker_color=sorted_colors, |
||||
text=[f"{s:.2f}" for s in sorted_scores], |
||||
textposition='outside', |
||||
)) |
||||
|
||||
# Update layout |
||||
label = theme.get("label", f"As {component}") |
||||
positive_pole = theme.get("positive_pole", "Positief") |
||||
negative_pole = theme.get("negative_pole", "Negatief") |
||||
|
||||
fig.update_layout( |
||||
title=f"Partijposities — {label}", |
||||
xaxis_title=f"{negative_pole} ← → {positive_pole}", |
||||
yaxis_title="", |
||||
height=max(400, len(parties) * 25), # Scale height with number of parties |
||||
margin=dict(l=150), # Extra margin for party names |
||||
showlegend=False, |
||||
) |
||||
|
||||
# Add vertical line at x=0 |
||||
fig.add_vline(x=0, line_dash="dash", line_color="gray", opacity=0.5) |
||||
|
||||
return fig |
||||
``` |
||||
|
||||
- [ ] **Step 4: Update SVD Components tab to use 1D chart for components 3-10** |
||||
|
||||
Find the section in `build_svd_components_tab` where `_render_party_axis_chart` is called (around line 2823) and update: |
||||
|
||||
```python |
||||
# explorer.py (update around line 2822-2830) |
||||
|
||||
# Before: |
||||
if comp_sel in (1, 2): |
||||
_render_party_axis_chart( |
||||
party_coords, comp_sel, theme, bootstrap_data=bootstrap_data |
||||
) |
||||
else: |
||||
st.caption( |
||||
"_Partijposities zijn alleen beschikbaar voor de eerste twee SVD-assen._" |
||||
) |
||||
|
||||
# After: |
||||
if comp_sel in (1, 2): |
||||
_render_party_axis_chart( |
||||
party_coords, comp_sel, theme, bootstrap_data=bootstrap_data |
||||
) |
||||
else: |
||||
# For components 3-10, show 1D party positions |
||||
# Extract 1D scores for this component |
||||
party_1d_coords = {} |
||||
idx = comp_sel - 1 # Convert to 0-indexed |
||||
for party, scores in party_scores.items(): |
||||
if len(scores) > idx: |
||||
party_1d_coords[party] = (scores[idx],) |
||||
|
||||
_render_party_axis_chart_1d( |
||||
party_1d_coords, comp_sel, theme, bootstrap_data=None |
||||
) |
||||
``` |
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass** |
||||
|
||||
Run: `uv run pytest tests/test_explorer_chart.py -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 6: Commit** |
||||
|
||||
```bash |
||||
git add explorer.py tests/test_explorer_chart.py |
||||
git commit -m "feat: add 1D party position charts for SVD components 3-10" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 6: Auto-compute flip directions for all components |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` |
||||
- Modify: `analysis/svd_labels.py` |
||||
- Test: `tests/test_svd_labels.py` |
||||
|
||||
- [ ] **Step 1: Write test for auto-computing flip in SVD Components tab** |
||||
|
||||
```python |
||||
# tests/test_svd_labels.py (append) |
||||
|
||||
def test_auto_flip_computation_for_all_components(): |
||||
"""Test that flip directions are computed correctly for all components.""" |
||||
from analysis.svd_labels import compute_flip_direction |
||||
|
||||
# Simulate party scores for 10 components |
||||
# Right parties should have positive scores on component 1 (EU-integratie) |
||||
# Left parties should have negative scores on component 1 |
||||
party_scores = { |
||||
"VVD": [0.5] * 10, # Right party, positive on all components |
||||
"PVV": [0.8] * 10, # Right party |
||||
"SP": [-0.6] * 10, # Left party, negative on all components |
||||
"DENK": [-0.4] * 10, # Left party |
||||
} |
||||
|
||||
# For all components, right_mean > left_mean, so flip should be False |
||||
for comp in range(1, 11): |
||||
flip = compute_flip_direction(comp, party_scores) |
||||
assert flip is False, f"Component {comp} should not flip" |
||||
|
||||
# Now test with right parties on left (negative scores) |
||||
party_scores_left = { |
||||
"VVD": [-0.5] * 10, |
||||
"PVV": [-0.8] * 10, |
||||
"SP": [0.6] * 10, |
||||
"DENK": [0.4] * 10, |
||||
} |
||||
|
||||
# For all components, right_mean < left_mean, so flip should be True |
||||
for comp in range(1, 11): |
||||
flip = compute_flip_direction(comp, party_scores_left) |
||||
assert flip is True, f"Component {comp} should flip" |
||||
``` |
||||
|
||||
- [ ] **Step 2: Run test to verify it passes** |
||||
|
||||
Run: `uv run pytest tests/test_svd_labels.py::test_auto_flip_computation_for_all_components -v` |
||||
Expected: PASS (function already implemented in Task 1) |
||||
|
||||
- [ ] **Step 3: Update SVD Components tab to compute flip dynamically** |
||||
|
||||
In `build_svd_components_tab`, after loading party_scores, compute flip for each component: |
||||
|
||||
```python |
||||
# explorer.py (in build_svd_components_tab, after loading party_scores) |
||||
|
||||
from analysis.svd_labels import compute_flip_direction |
||||
|
||||
# After party_scores is loaded (around line 2800) |
||||
# Compute flip directions for all components |
||||
computed_flips = {} |
||||
for comp in range(1, 11): |
||||
computed_flips[comp] = compute_flip_direction(comp, party_scores) |
||||
|
||||
# Update SVD_THEMES with computed flips |
||||
for comp, flip in computed_flips.items(): |
||||
if comp in SVD_THEMES: |
||||
SVD_THEMES[comp]["flip"] = flip |
||||
``` |
||||
|
||||
- [ ] **Step 4: Run tests to verify nothing broke** |
||||
|
||||
Run: `uv run pytest tests/test_svd_labels.py tests/test_explorer_chart.py -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 5: Commit** |
||||
|
||||
```bash |
||||
git add explorer.py analysis/svd_labels.py tests/test_svd_labels.py |
||||
git commit -m "feat: auto-compute flip directions for all SVD components" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 7: Update existing tests for new label system |
||||
|
||||
**Files:** |
||||
- Modify: `tests/test_political_compass.py` |
||||
- Modify: `tests/test_axis_label_fallback.py` |
||||
|
||||
- [ ] **Step 1: Run all tests to identify failures** |
||||
|
||||
Run: `uv run pytest tests/ -v --tb=short` |
||||
Expected: Some tests may fail due to label changes |
||||
|
||||
- [ ] **Step 2: Fix any failing tests** |
||||
|
||||
Update test assertions to use new labels from SVD_THEMES: |
||||
|
||||
```python |
||||
# tests/test_political_compass.py (update assertions) |
||||
|
||||
# Before: |
||||
assert "Links–Rechts" in x_label |
||||
|
||||
# After: |
||||
assert "EU-integratie" in x_label or "Nationalisme" in x_label |
||||
``` |
||||
|
||||
- [ ] **Step 3: Run all tests to verify they pass** |
||||
|
||||
Run: `uv run pytest tests/ -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 4: Commit** |
||||
|
||||
```bash |
||||
git add tests/ |
||||
git commit -m "test: update tests for unified SVD label system" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 8: Final verification and cleanup |
||||
|
||||
**Files:** |
||||
- All modified files |
||||
|
||||
- [ ] **Step 1: Run full test suite** |
||||
|
||||
Run: `uv run pytest tests/ -v` |
||||
Expected: All tests PASS |
||||
|
||||
- [ ] **Step 2: Run Streamlit app to verify UI** |
||||
|
||||
Run: `uv run streamlit run explorer.py` |
||||
Expected: App starts without errors |
||||
|
||||
- [ ] **Step 3: Manually verify in UI** |
||||
- Open SVD Components tab |
||||
- Check that components 1-10 show correct labels |
||||
- Check that party position charts render for all components |
||||
- Check that right parties appear on the right side of axes |
||||
|
||||
- [ ] **Step 4: Final commit** |
||||
|
||||
```bash |
||||
git add -A |
||||
git commit -m "feat: complete SVD label unification" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Summary |
||||
|
||||
This plan creates a unified label system for SVD components with: |
||||
1. Single source of truth in `SVD_THEMES` |
||||
2. Automatic flip direction computation based on party centroids |
||||
3. 1D party position charts for components 3-10 |
||||
4. Consistent labels across compass, trajectory, and SVD Components views |
||||
Loading…
Reference in new issue