parent
8c79aaf917
commit
521385c832
@ -0,0 +1,397 @@ |
||||
# SVD Tab Redesign Implementation Plan |
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
||||
|
||||
**Goal:** Redesign the SVD components tab to split motions by pole, add a 1D party-axis chart, replace the session-state detail pane with inline expand/collapse expanders, and show voting breakdowns per motion. |
||||
|
||||
**Architecture:** All changes are in `explorer.py`. Three additive changes (new constant, two new functions) followed by a targeted replacement of the motion-list section in `build_svd_components_tab`. No new files, no DB schema changes. |
||||
|
||||
**Tech Stack:** Python, Streamlit, DuckDB, Plotly (`plotly.graph_objects` already imported), JSON |
||||
|
||||
--- |
||||
|
||||
## File Map |
||||
|
||||
| File | Change | |
||||
|------|--------| |
||||
| `explorer.py` | Add `ChristenUnie` to `PARTY_COLOURS` (line ~52) | |
||||
| `explorer.py` | Add `CURRENT_PARLIAMENT_PARTIES` frozenset (line ~69) | |
||||
| `explorer.py` | Add `load_party_axis_scores()` top-level function (after `load_party_map`, line ~179) | |
||||
| `explorer.py` | Add `_render_party_axis_chart()` top-level function (after `load_party_axis_scores`) | |
||||
| `explorer.py` | Replace lines 854–896 in `build_svd_components_tab` with pole-split layout, party chart call, batch DB query, and expanders with voting | |
||||
|
||||
--- |
||||
|
||||
## Task 1: Add constants |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` (two locations: `PARTY_COLOURS` dict and after `KNOWN_MAJOR_PARTIES`) |
||||
|
||||
- [ ] **Step 1: Add `ChristenUnie` to `PARTY_COLOURS`** |
||||
|
||||
In `explorer.py`, the `PARTY_COLOURS` dict currently ends at line 52 with `"Unknown": "#9E9E9E"`. Add an entry just before `"Unknown"`: |
||||
|
||||
```python |
||||
"ChristenUnie": "#0288D1", # alias — stored as ChristenUnie in svd_vectors |
||||
``` |
||||
|
||||
Result: `PARTY_COLOURS` now maps both `"CU"` (already present via `"CU": "#0288D1"`) and `"ChristenUnie"`. |
||||
|
||||
Wait — `"CU"` key is NOT in `PARTY_COLOURS` already (the dict only has full names). The existing code uses `"CU"` in `KNOWN_MAJOR_PARTIES` but the lookup key `CU` has no mapping. Only add `"ChristenUnie": "#0288D1"`. |
||||
|
||||
- [ ] **Step 2: Add `CURRENT_PARLIAMENT_PARTIES` after `KNOWN_MAJOR_PARTIES` (line ~69)** |
||||
|
||||
```python |
||||
# Parties currently seated in the Tweede Kamer (2023 election cycle). |
||||
# These are the entity_ids as stored in svd_vectors for window='2025'. |
||||
CURRENT_PARLIAMENT_PARTIES: frozenset[str] = frozenset({ |
||||
"PVV", |
||||
"VVD", |
||||
"NSC", |
||||
"BBB", |
||||
"D66", |
||||
"GroenLinks-PvdA", |
||||
"CDA", |
||||
"SP", |
||||
"ChristenUnie", |
||||
"SGP", |
||||
"Volt", |
||||
"DENK", |
||||
"PvdD", |
||||
"JA21", |
||||
"FVD", |
||||
}) |
||||
``` |
||||
|
||||
- [ ] **Step 3: Syntax-check and commit** |
||||
|
||||
```bash |
||||
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" |
||||
git add explorer.py |
||||
git commit -m "feat(explorer): add ChristenUnie colour alias and CURRENT_PARLIAMENT_PARTIES constant" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 2: Add `load_party_axis_scores` |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` — insert after `load_party_map` function (currently ends around line 179) |
||||
|
||||
- [ ] **Step 1: Insert the function** |
||||
|
||||
Add this function immediately after the closing of `load_party_map` (the `return {}` / `finally con.close()` block, around line 179): |
||||
|
||||
```python |
||||
@st.cache_data(show_spinner="Partijposities op SVD-assen laden…") |
||||
def load_party_axis_scores(db_path: str) -> Dict[str, List[float]]: |
||||
"""Return per-party SVD vectors for window='2025'. |
||||
|
||||
Queries svd_vectors WHERE entity_type='mp' AND window_id='2025' |
||||
AND entity_id is a known current-parliament party. |
||||
|
||||
Returns: |
||||
{party_name: [float * k]} — k = 50 for the canonical 2025 window |
||||
""" |
||||
try: |
||||
con = duckdb.connect(database=db_path, read_only=True) |
||||
placeholders = ", ".join("?" for _ in CURRENT_PARLIAMENT_PARTIES) |
||||
rows = con.execute( |
||||
f"SELECT entity_id, vector FROM svd_vectors " |
||||
f"WHERE entity_type='mp' AND window_id='2025' " |
||||
f"AND entity_id IN ({placeholders})", |
||||
list(CURRENT_PARLIAMENT_PARTIES), |
||||
).fetchall() |
||||
con.close() |
||||
return { |
||||
row[0]: json.loads(row[1]) if isinstance(row[1], str) else list(row[1]) |
||||
for row in rows |
||||
} |
||||
except Exception: |
||||
logger.exception("Failed to load party axis scores") |
||||
return {} |
||||
``` |
||||
|
||||
- [ ] **Step 2: Syntax-check and commit** |
||||
|
||||
```bash |
||||
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" |
||||
git add explorer.py |
||||
git commit -m "feat(explorer): add load_party_axis_scores helper" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 3: Add `_render_party_axis_chart` |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` — insert after `load_party_axis_scores` |
||||
|
||||
- [ ] **Step 1: Insert the function** |
||||
|
||||
Add immediately after `load_party_axis_scores`: |
||||
|
||||
```python |
||||
def _render_party_axis_chart( |
||||
party_scores: Dict[str, List[float]], comp_sel: int |
||||
) -> None: |
||||
"""Render a 1D horizontal Plotly scatter of party positions on SVD axis `comp_sel`. |
||||
|
||||
Each party is plotted at its score on a single horizontal axis (y=0). |
||||
""" |
||||
if not party_scores: |
||||
st.caption("_Partijdata niet beschikbaar voor deze as._") |
||||
return |
||||
|
||||
axis_idx = comp_sel - 1 # 0-based index into the 50-dim vector |
||||
data: list[dict] = [] |
||||
for party, vec in party_scores.items(): |
||||
if axis_idx < len(vec): |
||||
data.append({"party": party, "score": vec[axis_idx]}) |
||||
|
||||
if not data: |
||||
st.caption("_Geen partijscores voor deze as._") |
||||
return |
||||
|
||||
scores = [d["score"] for d in data] |
||||
parties = [d["party"] for d in data] |
||||
colours = [PARTY_COLOURS.get(p, "#9E9E9E") for p in parties] |
||||
hover = [f"{p}: {s:.3f}" for p, s in zip(parties, scores)] |
||||
|
||||
fig = go.Figure() |
||||
# Baseline |
||||
x_min, x_max = min(scores) * 1.15, max(scores) * 1.15 |
||||
fig.add_trace( |
||||
go.Scatter( |
||||
x=[x_min, x_max], |
||||
y=[0, 0], |
||||
mode="lines", |
||||
line={"color": "#cccccc", "width": 1}, |
||||
hoverinfo="skip", |
||||
showlegend=False, |
||||
) |
||||
) |
||||
# Party markers |
||||
fig.add_trace( |
||||
go.Scatter( |
||||
x=scores, |
||||
y=[0] * len(scores), |
||||
mode="markers+text", |
||||
text=parties, |
||||
textposition="top center", |
||||
marker={"size": 12, "color": colours}, |
||||
hovertext=hover, |
||||
hoverinfo="text", |
||||
showlegend=False, |
||||
) |
||||
) |
||||
fig.update_layout( |
||||
height=160, |
||||
margin={"l": 10, "r": 10, "t": 10, "b": 30}, |
||||
xaxis={ |
||||
"title": "← Negatieve pool | Positieve pool →", |
||||
"zeroline": True, |
||||
"zerolinecolor": "#aaaaaa", |
||||
}, |
||||
yaxis={"visible": False, "range": [-1, 2]}, |
||||
plot_bgcolor="white", |
||||
) |
||||
st.plotly_chart(fig, use_container_width=True) |
||||
``` |
||||
|
||||
- [ ] **Step 2: Syntax-check and commit** |
||||
|
||||
```bash |
||||
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" |
||||
git add explorer.py |
||||
git commit -m "feat(explorer): add _render_party_axis_chart helper" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Task 4: Restructure motion display in `build_svd_components_tab` |
||||
|
||||
**Files:** |
||||
- Modify: `explorer.py` lines 841–896 (theme + poles + motion list + detail pane) |
||||
|
||||
The current code at lines 841–896: |
||||
```python |
||||
# Show theme explanation + poles |
||||
theme = SVD_THEMES.get(comp_sel, {}) |
||||
if theme: |
||||
st.info(f"**{theme['label']}** — {theme['explanation']}") |
||||
pos = theme.get("positive_pole", "") |
||||
neg = theme.get("negative_pole", "") |
||||
if pos or neg: |
||||
pcol, ncol = st.columns(2) |
||||
with pcol: |
||||
st.success(f"▲ **Positieve pool:** {pos}") |
||||
with ncol: |
||||
st.error(f"▼ **Negatieve pool:** {neg}") |
||||
|
||||
motions = comp_map.get(comp_sel, []) |
||||
|
||||
col1, col2 = st.columns([1, 2]) |
||||
with col1: |
||||
st.markdown("**Top-moties (titels)**") |
||||
for m in motions: |
||||
mid = m.get("motion_id") |
||||
score = m.get("score", 0.0) |
||||
title = m.get("title") or f"Motie #{mid}" |
||||
sign = "▲" if score >= 0 else "▼" |
||||
if st.button(f"{sign} {mid}: {title[:72]}", key=f"btn_{comp_sel}_{mid}"): |
||||
st.session_state["svd_selected_mid"] = mid |
||||
|
||||
with col2: |
||||
sel_mid = st.session_state.get("svd_selected_mid") |
||||
if not sel_mid and motions: |
||||
sel_mid = motions[0].get("motion_id") |
||||
if sel_mid: |
||||
# fetch motion metadata from DB for completeness |
||||
try: |
||||
con = duckdb.connect(database=db_path, read_only=True) |
||||
row = con.execute( |
||||
"SELECT id, title, date, policy_area, url, body_text FROM motions WHERE id=?", |
||||
[int(sel_mid)], |
||||
).fetchone() |
||||
con.close() |
||||
except Exception: |
||||
row = None |
||||
|
||||
if row: |
||||
st.markdown(f"### {row[1] or f'Motie #{row[0]}'}") |
||||
try: |
||||
date_str = str(row[2])[:10] |
||||
except Exception: |
||||
date_str = "?" |
||||
st.caption(f"📅 {date_str} | {row[3]}") |
||||
if row[4] and str(row[4]).startswith("http"): |
||||
st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})") |
||||
if row[5]: |
||||
with st.expander("Toon volledige tekst"): |
||||
st.write(row[5]) |
||||
else: |
||||
st.info(f"Metadata not found in DB for motion {sel_mid}") |
||||
``` |
||||
|
||||
Replace entirely with: |
||||
|
||||
```python |
||||
# Show theme explanation |
||||
theme = SVD_THEMES.get(comp_sel, {}) |
||||
if theme: |
||||
st.info(f"**{theme['label']}** — {theme['explanation']}") |
||||
|
||||
motions = comp_map.get(comp_sel, []) |
||||
|
||||
# Party axis chart |
||||
party_scores = load_party_axis_scores(db_path) |
||||
_render_party_axis_chart(party_scores, comp_sel) |
||||
|
||||
# Batch-fetch motion details (title, date, policy_area, url, body_text, voting_results) |
||||
motion_ids = [m.get("motion_id") for m in motions if m.get("motion_id") is not None] |
||||
motion_details: Dict[int, tuple] = {} |
||||
if motion_ids: |
||||
try: |
||||
placeholders = ", ".join("?" for _ in motion_ids) |
||||
con = duckdb.connect(database=db_path, read_only=True) |
||||
db_rows = con.execute( |
||||
f"SELECT id, title, date, policy_area, url, body_text, voting_results " |
||||
f"FROM motions WHERE id IN ({placeholders})", |
||||
[int(mid) for mid in motion_ids], |
||||
).fetchall() |
||||
con.close() |
||||
motion_details = {r[0]: r for r in db_rows} |
||||
except Exception: |
||||
logger.exception("Failed to batch-fetch motion details") |
||||
|
||||
# Split motions by pole sign |
||||
pos_motions = [m for m in motions if float(m.get("score", 0.0)) >= 0] |
||||
neg_motions = [m for m in motions if float(m.get("score", 0.0)) < 0] |
||||
|
||||
pos_pole = theme.get("positive_pole", "Positieve pool") if theme else "Positieve pool" |
||||
neg_pole = theme.get("negative_pole", "Negatieve pool") if theme else "Negatieve pool" |
||||
|
||||
pcol, ncol = st.columns(2) |
||||
|
||||
with pcol: |
||||
st.success(f"▲ **Positieve pool:** {pos_pole}") |
||||
for m in pos_motions: |
||||
mid = m.get("motion_id") |
||||
raw_title = m.get("title") or f"Motie #{mid}" |
||||
with st.expander(f"▲ {raw_title[:80]}"): |
||||
row = motion_details.get(int(mid)) if mid is not None else None |
||||
if row: |
||||
try: |
||||
date_str = str(row[2])[:10] |
||||
except Exception: |
||||
date_str = "?" |
||||
st.caption(f"📅 {date_str} | {row[3] or '—'}") |
||||
if row[4] and str(row[4]).startswith("http"): |
||||
st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})") |
||||
if row[5]: |
||||
with st.expander("Toon volledige tekst"): |
||||
st.write(row[5]) |
||||
_render_voting_results(row[6]) |
||||
else: |
||||
st.caption("_Geen metadata beschikbaar_") |
||||
|
||||
with ncol: |
||||
st.error(f"▼ **Negatieve pool:** {neg_pole}") |
||||
for m in neg_motions: |
||||
mid = m.get("motion_id") |
||||
raw_title = m.get("title") or f"Motie #{mid}" |
||||
with st.expander(f"▼ {raw_title[:80]}"): |
||||
row = motion_details.get(int(mid)) if mid is not None else None |
||||
if row: |
||||
try: |
||||
date_str = str(row[2])[:10] |
||||
except Exception: |
||||
date_str = "?" |
||||
st.caption(f"📅 {date_str} | {row[3] or '—'}") |
||||
if row[4] and str(row[4]).startswith("http"): |
||||
st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})") |
||||
if row[5]: |
||||
with st.expander("Toon volledige tekst"): |
||||
st.write(row[5]) |
||||
_render_voting_results(row[6]) |
||||
else: |
||||
st.caption("_Geen metadata beschikbaar_") |
||||
``` |
||||
|
||||
- [ ] **Step 1: Apply the replacement** |
||||
|
||||
Use the Edit tool to replace the block from ` # Show theme explanation + poles` through the closing `st.info(f"Metadata not found in DB for motion {sel_mid}")` line with the new code above. |
||||
|
||||
- [ ] **Step 2: Syntax-check** |
||||
|
||||
```bash |
||||
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" |
||||
``` |
||||
|
||||
Expected output: `OK` |
||||
|
||||
- [ ] **Step 3: Commit** |
||||
|
||||
```bash |
||||
git add explorer.py |
||||
git commit -m "feat(explorer): restructure SVD tab — pole-split motions, party axis chart, inline expanders with voting" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Final Verification |
||||
|
||||
- [ ] **Syntax check** |
||||
|
||||
```bash |
||||
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')" |
||||
``` |
||||
|
||||
- [ ] **Smoke test (manual):** Run `streamlit run explorer.py`, open the SVD tab, verify: |
||||
- Selectbox shows axis labels |
||||
- Explanation text renders |
||||
- Party axis chart renders with ~15 coloured dots |
||||
- Positive motions appear in green (left) column, negative in red (right) column |
||||
- Clicking an expander shows date, URL, body text sub-expander, and voting breakdown |
||||
- Switching axes updates all sections correctly |
||||
Loading…
Reference in new issue