diff --git a/docs/superpowers/plans/2026-03-24-svd-tab-redesign.md b/docs/superpowers/plans/2026-03-24-svd-tab-redesign.md new file mode 100644 index 0000000..286dddd --- /dev/null +++ b/docs/superpowers/plans/2026-03-24-svd-tab-redesign.md @@ -0,0 +1,397 @@ +# SVD Tab Redesign Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Redesign the SVD components tab to split motions by pole, add a 1D party-axis chart, replace the session-state detail pane with inline expand/collapse expanders, and show voting breakdowns per motion. + +**Architecture:** All changes are in `explorer.py`. Three additive changes (new constant, two new functions) followed by a targeted replacement of the motion-list section in `build_svd_components_tab`. No new files, no DB schema changes. + +**Tech Stack:** Python, Streamlit, DuckDB, Plotly (`plotly.graph_objects` already imported), JSON + +--- + +## File Map + +| File | Change | +|------|--------| +| `explorer.py` | Add `ChristenUnie` to `PARTY_COLOURS` (line ~52) | +| `explorer.py` | Add `CURRENT_PARLIAMENT_PARTIES` frozenset (line ~69) | +| `explorer.py` | Add `load_party_axis_scores()` top-level function (after `load_party_map`, line ~179) | +| `explorer.py` | Add `_render_party_axis_chart()` top-level function (after `load_party_axis_scores`) | +| `explorer.py` | Replace lines 854–896 in `build_svd_components_tab` with pole-split layout, party chart call, batch DB query, and expanders with voting | + +--- + +## Task 1: Add constants + +**Files:** +- Modify: `explorer.py` (two locations: `PARTY_COLOURS` dict and after `KNOWN_MAJOR_PARTIES`) + +- [ ] **Step 1: Add `ChristenUnie` to `PARTY_COLOURS`** + +In `explorer.py`, the `PARTY_COLOURS` dict currently ends at line 52 with `"Unknown": "#9E9E9E"`. Add an entry just before `"Unknown"`: + +```python + "ChristenUnie": "#0288D1", # alias — stored as ChristenUnie in svd_vectors +``` + +Result: `PARTY_COLOURS` now maps both `"CU"` (already present via `"CU": "#0288D1"`) and `"ChristenUnie"`. + +Wait — `"CU"` key is NOT in `PARTY_COLOURS` already (the dict only has full names). The existing code uses `"CU"` in `KNOWN_MAJOR_PARTIES` but the lookup key `CU` has no mapping. Only add `"ChristenUnie": "#0288D1"`. + +- [ ] **Step 2: Add `CURRENT_PARLIAMENT_PARTIES` after `KNOWN_MAJOR_PARTIES` (line ~69)** + +```python +# Parties currently seated in the Tweede Kamer (2023 election cycle). +# These are the entity_ids as stored in svd_vectors for window='2025'. +CURRENT_PARLIAMENT_PARTIES: frozenset[str] = frozenset({ + "PVV", + "VVD", + "NSC", + "BBB", + "D66", + "GroenLinks-PvdA", + "CDA", + "SP", + "ChristenUnie", + "SGP", + "Volt", + "DENK", + "PvdD", + "JA21", + "FVD", +}) +``` + +- [ ] **Step 3: Syntax-check and commit** + +```bash +python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" +git add explorer.py +git commit -m "feat(explorer): add ChristenUnie colour alias and CURRENT_PARLIAMENT_PARTIES constant" +``` + +--- + +## Task 2: Add `load_party_axis_scores` + +**Files:** +- Modify: `explorer.py` — insert after `load_party_map` function (currently ends around line 179) + +- [ ] **Step 1: Insert the function** + +Add this function immediately after the closing of `load_party_map` (the `return {}` / `finally con.close()` block, around line 179): + +```python +@st.cache_data(show_spinner="Partijposities op SVD-assen laden…") +def load_party_axis_scores(db_path: str) -> Dict[str, List[float]]: + """Return per-party SVD vectors for window='2025'. + + Queries svd_vectors WHERE entity_type='mp' AND window_id='2025' + AND entity_id is a known current-parliament party. + + Returns: + {party_name: [float * k]} — k = 50 for the canonical 2025 window + """ + try: + con = duckdb.connect(database=db_path, read_only=True) + placeholders = ", ".join("?" for _ in CURRENT_PARLIAMENT_PARTIES) + rows = con.execute( + f"SELECT entity_id, vector FROM svd_vectors " + f"WHERE entity_type='mp' AND window_id='2025' " + f"AND entity_id IN ({placeholders})", + list(CURRENT_PARLIAMENT_PARTIES), + ).fetchall() + con.close() + return { + row[0]: json.loads(row[1]) if isinstance(row[1], str) else list(row[1]) + for row in rows + } + except Exception: + logger.exception("Failed to load party axis scores") + return {} +``` + +- [ ] **Step 2: Syntax-check and commit** + +```bash +python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" +git add explorer.py +git commit -m "feat(explorer): add load_party_axis_scores helper" +``` + +--- + +## Task 3: Add `_render_party_axis_chart` + +**Files:** +- Modify: `explorer.py` — insert after `load_party_axis_scores` + +- [ ] **Step 1: Insert the function** + +Add immediately after `load_party_axis_scores`: + +```python +def _render_party_axis_chart( + party_scores: Dict[str, List[float]], comp_sel: int +) -> None: + """Render a 1D horizontal Plotly scatter of party positions on SVD axis `comp_sel`. + + Each party is plotted at its score on a single horizontal axis (y=0). + """ + if not party_scores: + st.caption("_Partijdata niet beschikbaar voor deze as._") + return + + axis_idx = comp_sel - 1 # 0-based index into the 50-dim vector + data: list[dict] = [] + for party, vec in party_scores.items(): + if axis_idx < len(vec): + data.append({"party": party, "score": vec[axis_idx]}) + + if not data: + st.caption("_Geen partijscores voor deze as._") + return + + scores = [d["score"] for d in data] + parties = [d["party"] for d in data] + colours = [PARTY_COLOURS.get(p, "#9E9E9E") for p in parties] + hover = [f"{p}: {s:.3f}" for p, s in zip(parties, scores)] + + fig = go.Figure() + # Baseline + x_min, x_max = min(scores) * 1.15, max(scores) * 1.15 + fig.add_trace( + go.Scatter( + x=[x_min, x_max], + y=[0, 0], + mode="lines", + line={"color": "#cccccc", "width": 1}, + hoverinfo="skip", + showlegend=False, + ) + ) + # Party markers + fig.add_trace( + go.Scatter( + x=scores, + y=[0] * len(scores), + mode="markers+text", + text=parties, + textposition="top center", + marker={"size": 12, "color": colours}, + hovertext=hover, + hoverinfo="text", + showlegend=False, + ) + ) + fig.update_layout( + height=160, + margin={"l": 10, "r": 10, "t": 10, "b": 30}, + xaxis={ + "title": "← Negatieve pool | Positieve pool →", + "zeroline": True, + "zerolinecolor": "#aaaaaa", + }, + yaxis={"visible": False, "range": [-1, 2]}, + plot_bgcolor="white", + ) + st.plotly_chart(fig, use_container_width=True) +``` + +- [ ] **Step 2: Syntax-check and commit** + +```bash +python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" +git add explorer.py +git commit -m "feat(explorer): add _render_party_axis_chart helper" +``` + +--- + +## Task 4: Restructure motion display in `build_svd_components_tab` + +**Files:** +- Modify: `explorer.py` lines 841–896 (theme + poles + motion list + detail pane) + +The current code at lines 841–896: +```python + # Show theme explanation + poles + theme = SVD_THEMES.get(comp_sel, {}) + if theme: + st.info(f"**{theme['label']}** — {theme['explanation']}") + pos = theme.get("positive_pole", "") + neg = theme.get("negative_pole", "") + if pos or neg: + pcol, ncol = st.columns(2) + with pcol: + st.success(f"▲ **Positieve pool:** {pos}") + with ncol: + st.error(f"▼ **Negatieve pool:** {neg}") + + motions = comp_map.get(comp_sel, []) + + col1, col2 = st.columns([1, 2]) + with col1: + st.markdown("**Top-moties (titels)**") + for m in motions: + mid = m.get("motion_id") + score = m.get("score", 0.0) + title = m.get("title") or f"Motie #{mid}" + sign = "▲" if score >= 0 else "▼" + if st.button(f"{sign} {mid}: {title[:72]}", key=f"btn_{comp_sel}_{mid}"): + st.session_state["svd_selected_mid"] = mid + + with col2: + sel_mid = st.session_state.get("svd_selected_mid") + if not sel_mid and motions: + sel_mid = motions[0].get("motion_id") + if sel_mid: + # fetch motion metadata from DB for completeness + try: + con = duckdb.connect(database=db_path, read_only=True) + row = con.execute( + "SELECT id, title, date, policy_area, url, body_text FROM motions WHERE id=?", + [int(sel_mid)], + ).fetchone() + con.close() + except Exception: + row = None + + if row: + st.markdown(f"### {row[1] or f'Motie #{row[0]}'}") + try: + date_str = str(row[2])[:10] + except Exception: + date_str = "?" + st.caption(f"📅 {date_str} | {row[3]}") + if row[4] and str(row[4]).startswith("http"): + st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})") + if row[5]: + with st.expander("Toon volledige tekst"): + st.write(row[5]) + else: + st.info(f"Metadata not found in DB for motion {sel_mid}") +``` + +Replace entirely with: + +```python + # Show theme explanation + theme = SVD_THEMES.get(comp_sel, {}) + if theme: + st.info(f"**{theme['label']}** — {theme['explanation']}") + + motions = comp_map.get(comp_sel, []) + + # Party axis chart + party_scores = load_party_axis_scores(db_path) + _render_party_axis_chart(party_scores, comp_sel) + + # Batch-fetch motion details (title, date, policy_area, url, body_text, voting_results) + motion_ids = [m.get("motion_id") for m in motions if m.get("motion_id") is not None] + motion_details: Dict[int, tuple] = {} + if motion_ids: + try: + placeholders = ", ".join("?" for _ in motion_ids) + con = duckdb.connect(database=db_path, read_only=True) + db_rows = con.execute( + f"SELECT id, title, date, policy_area, url, body_text, voting_results " + f"FROM motions WHERE id IN ({placeholders})", + [int(mid) for mid in motion_ids], + ).fetchall() + con.close() + motion_details = {r[0]: r for r in db_rows} + except Exception: + logger.exception("Failed to batch-fetch motion details") + + # Split motions by pole sign + pos_motions = [m for m in motions if float(m.get("score", 0.0)) >= 0] + neg_motions = [m for m in motions if float(m.get("score", 0.0)) < 0] + + pos_pole = theme.get("positive_pole", "Positieve pool") if theme else "Positieve pool" + neg_pole = theme.get("negative_pole", "Negatieve pool") if theme else "Negatieve pool" + + pcol, ncol = st.columns(2) + + with pcol: + st.success(f"▲ **Positieve pool:** {pos_pole}") + for m in pos_motions: + mid = m.get("motion_id") + raw_title = m.get("title") or f"Motie #{mid}" + with st.expander(f"▲ {raw_title[:80]}"): + row = motion_details.get(int(mid)) if mid is not None else None + if row: + try: + date_str = str(row[2])[:10] + except Exception: + date_str = "?" + st.caption(f"📅 {date_str} | {row[3] or '—'}") + if row[4] and str(row[4]).startswith("http"): + st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})") + if row[5]: + with st.expander("Toon volledige tekst"): + st.write(row[5]) + _render_voting_results(row[6]) + else: + st.caption("_Geen metadata beschikbaar_") + + with ncol: + st.error(f"▼ **Negatieve pool:** {neg_pole}") + for m in neg_motions: + mid = m.get("motion_id") + raw_title = m.get("title") or f"Motie #{mid}" + with st.expander(f"▼ {raw_title[:80]}"): + row = motion_details.get(int(mid)) if mid is not None else None + if row: + try: + date_str = str(row[2])[:10] + except Exception: + date_str = "?" + st.caption(f"📅 {date_str} | {row[3] or '—'}") + if row[4] and str(row[4]).startswith("http"): + st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})") + if row[5]: + with st.expander("Toon volledige tekst"): + st.write(row[5]) + _render_voting_results(row[6]) + else: + st.caption("_Geen metadata beschikbaar_") +``` + +- [ ] **Step 1: Apply the replacement** + +Use the Edit tool to replace the block from ` # Show theme explanation + poles` through the closing `st.info(f"Metadata not found in DB for motion {sel_mid}")` line with the new code above. + +- [ ] **Step 2: Syntax-check** + +```bash +python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" +``` + +Expected output: `OK` + +- [ ] **Step 3: Commit** + +```bash +git add explorer.py +git commit -m "feat(explorer): restructure SVD tab — pole-split motions, party axis chart, inline expanders with voting" +``` + +--- + +## Final Verification + +- [ ] **Syntax check** + +```bash +python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" +``` + +- [ ] **Smoke test (manual):** Run `streamlit run explorer.py`, open the SVD tab, verify: + - Selectbox shows axis labels + - Explanation text renders + - Party axis chart renders with ~15 coloured dots + - Positive motions appear in green (left) column, negative in red (right) column + - Clicking an expander shows date, URL, body text sub-expander, and voting breakdown + - Switching axes updates all sections correctly