You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
397 lines
14 KiB
397 lines
14 KiB
# SVD Tab Redesign Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Redesign the SVD components tab to split motions by pole, add a 1D party-axis chart, replace the session-state detail pane with inline expand/collapse expanders, and show voting breakdowns per motion.
|
|
|
|
**Architecture:** All changes are in `explorer.py`. Three additive changes (new constant, two new functions) followed by a targeted replacement of the motion-list section in `build_svd_components_tab`. No new files, no DB schema changes.
|
|
|
|
**Tech Stack:** Python, Streamlit, DuckDB, Plotly (`plotly.graph_objects` already imported), JSON
|
|
|
|
---
|
|
|
|
## File Map
|
|
|
|
| File | Change |
|
|
|------|--------|
|
|
| `explorer.py` | Add `ChristenUnie` to `PARTY_COLOURS` (line ~52) |
|
|
| `explorer.py` | Add `CURRENT_PARLIAMENT_PARTIES` frozenset (line ~69) |
|
|
| `explorer.py` | Add `load_party_axis_scores()` top-level function (after `load_party_map`, line ~179) |
|
|
| `explorer.py` | Add `_render_party_axis_chart()` top-level function (after `load_party_axis_scores`) |
|
|
| `explorer.py` | Replace lines 854–896 in `build_svd_components_tab` with pole-split layout, party chart call, batch DB query, and expanders with voting |
|
|
|
|
---
|
|
|
|
## Task 1: Add constants
|
|
|
|
**Files:**
|
|
- Modify: `explorer.py` (two locations: `PARTY_COLOURS` dict and after `KNOWN_MAJOR_PARTIES`)
|
|
|
|
- [ ] **Step 1: Add `ChristenUnie` to `PARTY_COLOURS`**
|
|
|
|
In `explorer.py`, the `PARTY_COLOURS` dict currently ends at line 52 with `"Unknown": "#9E9E9E"`. Add an entry just before `"Unknown"`:
|
|
|
|
```python
|
|
"ChristenUnie": "#0288D1", # alias — stored as ChristenUnie in svd_vectors
|
|
```
|
|
|
|
Result: `PARTY_COLOURS` now maps both `"CU"` (already present via `"CU": "#0288D1"`) and `"ChristenUnie"`.
|
|
|
|
Wait — `"CU"` key is NOT in `PARTY_COLOURS` already (the dict only has full names). The existing code uses `"CU"` in `KNOWN_MAJOR_PARTIES` but the lookup key `CU` has no mapping. Only add `"ChristenUnie": "#0288D1"`.
|
|
|
|
- [ ] **Step 2: Add `CURRENT_PARLIAMENT_PARTIES` after `KNOWN_MAJOR_PARTIES` (line ~69)**
|
|
|
|
```python
|
|
# Parties currently seated in the Tweede Kamer (2023 election cycle).
|
|
# These are the entity_ids as stored in svd_vectors for window='2025'.
|
|
CURRENT_PARLIAMENT_PARTIES: frozenset[str] = frozenset({
|
|
"PVV",
|
|
"VVD",
|
|
"NSC",
|
|
"BBB",
|
|
"D66",
|
|
"GroenLinks-PvdA",
|
|
"CDA",
|
|
"SP",
|
|
"ChristenUnie",
|
|
"SGP",
|
|
"Volt",
|
|
"DENK",
|
|
"PvdD",
|
|
"JA21",
|
|
"FVD",
|
|
})
|
|
```
|
|
|
|
- [ ] **Step 3: Syntax-check and commit**
|
|
|
|
```bash
|
|
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
|
|
git add explorer.py
|
|
git commit -m "feat(explorer): add ChristenUnie colour alias and CURRENT_PARLIAMENT_PARTIES constant"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Add `load_party_axis_scores`
|
|
|
|
**Files:**
|
|
- Modify: `explorer.py` — insert after `load_party_map` function (currently ends around line 179)
|
|
|
|
- [ ] **Step 1: Insert the function**
|
|
|
|
Add this function immediately after the closing of `load_party_map` (the `return {}` / `finally con.close()` block, around line 179):
|
|
|
|
```python
|
|
@st.cache_data(show_spinner="Partijposities op SVD-assen laden…")
|
|
def load_party_axis_scores(db_path: str) -> Dict[str, List[float]]:
|
|
"""Return per-party SVD vectors for window='2025'.
|
|
|
|
Queries svd_vectors WHERE entity_type='mp' AND window_id='2025'
|
|
AND entity_id is a known current-parliament party.
|
|
|
|
Returns:
|
|
{party_name: [float * k]} — k = 50 for the canonical 2025 window
|
|
"""
|
|
try:
|
|
con = duckdb.connect(database=db_path, read_only=True)
|
|
placeholders = ", ".join("?" for _ in CURRENT_PARLIAMENT_PARTIES)
|
|
rows = con.execute(
|
|
f"SELECT entity_id, vector FROM svd_vectors "
|
|
f"WHERE entity_type='mp' AND window_id='2025' "
|
|
f"AND entity_id IN ({placeholders})",
|
|
list(CURRENT_PARLIAMENT_PARTIES),
|
|
).fetchall()
|
|
con.close()
|
|
return {
|
|
row[0]: json.loads(row[1]) if isinstance(row[1], str) else list(row[1])
|
|
for row in rows
|
|
}
|
|
except Exception:
|
|
logger.exception("Failed to load party axis scores")
|
|
return {}
|
|
```
|
|
|
|
- [ ] **Step 2: Syntax-check and commit**
|
|
|
|
```bash
|
|
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
|
|
git add explorer.py
|
|
git commit -m "feat(explorer): add load_party_axis_scores helper"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Add `_render_party_axis_chart`
|
|
|
|
**Files:**
|
|
- Modify: `explorer.py` — insert after `load_party_axis_scores`
|
|
|
|
- [ ] **Step 1: Insert the function**
|
|
|
|
Add immediately after `load_party_axis_scores`:
|
|
|
|
```python
|
|
def _render_party_axis_chart(
|
|
party_scores: Dict[str, List[float]], comp_sel: int
|
|
) -> None:
|
|
"""Render a 1D horizontal Plotly scatter of party positions on SVD axis `comp_sel`.
|
|
|
|
Each party is plotted at its score on a single horizontal axis (y=0).
|
|
"""
|
|
if not party_scores:
|
|
st.caption("_Partijdata niet beschikbaar voor deze as._")
|
|
return
|
|
|
|
axis_idx = comp_sel - 1 # 0-based index into the 50-dim vector
|
|
data: list[dict] = []
|
|
for party, vec in party_scores.items():
|
|
if axis_idx < len(vec):
|
|
data.append({"party": party, "score": vec[axis_idx]})
|
|
|
|
if not data:
|
|
st.caption("_Geen partijscores voor deze as._")
|
|
return
|
|
|
|
scores = [d["score"] for d in data]
|
|
parties = [d["party"] for d in data]
|
|
colours = [PARTY_COLOURS.get(p, "#9E9E9E") for p in parties]
|
|
hover = [f"{p}: {s:.3f}" for p, s in zip(parties, scores)]
|
|
|
|
fig = go.Figure()
|
|
# Baseline
|
|
x_min, x_max = min(scores) * 1.15, max(scores) * 1.15
|
|
fig.add_trace(
|
|
go.Scatter(
|
|
x=[x_min, x_max],
|
|
y=[0, 0],
|
|
mode="lines",
|
|
line={"color": "#cccccc", "width": 1},
|
|
hoverinfo="skip",
|
|
showlegend=False,
|
|
)
|
|
)
|
|
# Party markers
|
|
fig.add_trace(
|
|
go.Scatter(
|
|
x=scores,
|
|
y=[0] * len(scores),
|
|
mode="markers+text",
|
|
text=parties,
|
|
textposition="top center",
|
|
marker={"size": 12, "color": colours},
|
|
hovertext=hover,
|
|
hoverinfo="text",
|
|
showlegend=False,
|
|
)
|
|
)
|
|
fig.update_layout(
|
|
height=160,
|
|
margin={"l": 10, "r": 10, "t": 10, "b": 30},
|
|
xaxis={
|
|
"title": "← Negatieve pool | Positieve pool →",
|
|
"zeroline": True,
|
|
"zerolinecolor": "#aaaaaa",
|
|
},
|
|
yaxis={"visible": False, "range": [-1, 2]},
|
|
plot_bgcolor="white",
|
|
)
|
|
st.plotly_chart(fig, use_container_width=True)
|
|
```
|
|
|
|
- [ ] **Step 2: Syntax-check and commit**
|
|
|
|
```bash
|
|
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
|
|
git add explorer.py
|
|
git commit -m "feat(explorer): add _render_party_axis_chart helper"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Restructure motion display in `build_svd_components_tab`
|
|
|
|
**Files:**
|
|
- Modify: `explorer.py` lines 841–896 (theme + poles + motion list + detail pane)
|
|
|
|
The current code at lines 841–896:
|
|
```python
|
|
# Show theme explanation + poles
|
|
theme = SVD_THEMES.get(comp_sel, {})
|
|
if theme:
|
|
st.info(f"**{theme['label']}** — {theme['explanation']}")
|
|
pos = theme.get("positive_pole", "")
|
|
neg = theme.get("negative_pole", "")
|
|
if pos or neg:
|
|
pcol, ncol = st.columns(2)
|
|
with pcol:
|
|
st.success(f"▲ **Positieve pool:** {pos}")
|
|
with ncol:
|
|
st.error(f"▼ **Negatieve pool:** {neg}")
|
|
|
|
motions = comp_map.get(comp_sel, [])
|
|
|
|
col1, col2 = st.columns([1, 2])
|
|
with col1:
|
|
st.markdown("**Top-moties (titels)**")
|
|
for m in motions:
|
|
mid = m.get("motion_id")
|
|
score = m.get("score", 0.0)
|
|
title = m.get("title") or f"Motie #{mid}"
|
|
sign = "▲" if score >= 0 else "▼"
|
|
if st.button(f"{sign} {mid}: {title[:72]}", key=f"btn_{comp_sel}_{mid}"):
|
|
st.session_state["svd_selected_mid"] = mid
|
|
|
|
with col2:
|
|
sel_mid = st.session_state.get("svd_selected_mid")
|
|
if not sel_mid and motions:
|
|
sel_mid = motions[0].get("motion_id")
|
|
if sel_mid:
|
|
# fetch motion metadata from DB for completeness
|
|
try:
|
|
con = duckdb.connect(database=db_path, read_only=True)
|
|
row = con.execute(
|
|
"SELECT id, title, date, policy_area, url, body_text FROM motions WHERE id=?",
|
|
[int(sel_mid)],
|
|
).fetchone()
|
|
con.close()
|
|
except Exception:
|
|
row = None
|
|
|
|
if row:
|
|
st.markdown(f"### {row[1] or f'Motie #{row[0]}'}")
|
|
try:
|
|
date_str = str(row[2])[:10]
|
|
except Exception:
|
|
date_str = "?"
|
|
st.caption(f"📅 {date_str} | {row[3]}")
|
|
if row[4] and str(row[4]).startswith("http"):
|
|
st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})")
|
|
if row[5]:
|
|
with st.expander("Toon volledige tekst"):
|
|
st.write(row[5])
|
|
else:
|
|
st.info(f"Metadata not found in DB for motion {sel_mid}")
|
|
```
|
|
|
|
Replace entirely with:
|
|
|
|
```python
|
|
# Show theme explanation
|
|
theme = SVD_THEMES.get(comp_sel, {})
|
|
if theme:
|
|
st.info(f"**{theme['label']}** — {theme['explanation']}")
|
|
|
|
motions = comp_map.get(comp_sel, [])
|
|
|
|
# Party axis chart
|
|
party_scores = load_party_axis_scores(db_path)
|
|
_render_party_axis_chart(party_scores, comp_sel)
|
|
|
|
# Batch-fetch motion details (title, date, policy_area, url, body_text, voting_results)
|
|
motion_ids = [m.get("motion_id") for m in motions if m.get("motion_id") is not None]
|
|
motion_details: Dict[int, tuple] = {}
|
|
if motion_ids:
|
|
try:
|
|
placeholders = ", ".join("?" for _ in motion_ids)
|
|
con = duckdb.connect(database=db_path, read_only=True)
|
|
db_rows = con.execute(
|
|
f"SELECT id, title, date, policy_area, url, body_text, voting_results "
|
|
f"FROM motions WHERE id IN ({placeholders})",
|
|
[int(mid) for mid in motion_ids],
|
|
).fetchall()
|
|
con.close()
|
|
motion_details = {r[0]: r for r in db_rows}
|
|
except Exception:
|
|
logger.exception("Failed to batch-fetch motion details")
|
|
|
|
# Split motions by pole sign
|
|
pos_motions = [m for m in motions if float(m.get("score", 0.0)) >= 0]
|
|
neg_motions = [m for m in motions if float(m.get("score", 0.0)) < 0]
|
|
|
|
pos_pole = theme.get("positive_pole", "Positieve pool") if theme else "Positieve pool"
|
|
neg_pole = theme.get("negative_pole", "Negatieve pool") if theme else "Negatieve pool"
|
|
|
|
pcol, ncol = st.columns(2)
|
|
|
|
with pcol:
|
|
st.success(f"▲ **Positieve pool:** {pos_pole}")
|
|
for m in pos_motions:
|
|
mid = m.get("motion_id")
|
|
raw_title = m.get("title") or f"Motie #{mid}"
|
|
with st.expander(f"▲ {raw_title[:80]}"):
|
|
row = motion_details.get(int(mid)) if mid is not None else None
|
|
if row:
|
|
try:
|
|
date_str = str(row[2])[:10]
|
|
except Exception:
|
|
date_str = "?"
|
|
st.caption(f"📅 {date_str} | {row[3] or '—'}")
|
|
if row[4] and str(row[4]).startswith("http"):
|
|
st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})")
|
|
if row[5]:
|
|
with st.expander("Toon volledige tekst"):
|
|
st.write(row[5])
|
|
_render_voting_results(row[6])
|
|
else:
|
|
st.caption("_Geen metadata beschikbaar_")
|
|
|
|
with ncol:
|
|
st.error(f"▼ **Negatieve pool:** {neg_pole}")
|
|
for m in neg_motions:
|
|
mid = m.get("motion_id")
|
|
raw_title = m.get("title") or f"Motie #{mid}"
|
|
with st.expander(f"▼ {raw_title[:80]}"):
|
|
row = motion_details.get(int(mid)) if mid is not None else None
|
|
if row:
|
|
try:
|
|
date_str = str(row[2])[:10]
|
|
except Exception:
|
|
date_str = "?"
|
|
st.caption(f"📅 {date_str} | {row[3] or '—'}")
|
|
if row[4] and str(row[4]).startswith("http"):
|
|
st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})")
|
|
if row[5]:
|
|
with st.expander("Toon volledige tekst"):
|
|
st.write(row[5])
|
|
_render_voting_results(row[6])
|
|
else:
|
|
st.caption("_Geen metadata beschikbaar_")
|
|
```
|
|
|
|
- [ ] **Step 1: Apply the replacement**
|
|
|
|
Use the Edit tool to replace the block from ` # Show theme explanation + poles` through the closing `st.info(f"Metadata not found in DB for motion {sel_mid}")` line with the new code above.
|
|
|
|
- [ ] **Step 2: Syntax-check**
|
|
|
|
```bash
|
|
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
|
|
```
|
|
|
|
Expected output: `OK`
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add explorer.py
|
|
git commit -m "feat(explorer): restructure SVD tab — pole-split motions, party axis chart, inline expanders with voting"
|
|
```
|
|
|
|
---
|
|
|
|
## Final Verification
|
|
|
|
- [ ] **Syntax check**
|
|
|
|
```bash
|
|
python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
|
|
```
|
|
|
|
- [ ] **Smoke test (manual):** Run `streamlit run explorer.py`, open the SVD tab, verify:
|
|
- Selectbox shows axis labels
|
|
- Explanation text renders
|
|
- Party axis chart renders with ~15 coloured dots
|
|
- Positive motions appear in green (left) column, negative in red (right) column
|
|
- Clicking an expander shows date, URL, body text sub-expander, and voting breakdown
|
|
- Switching axes updates all sections correctly
|
|
|