14 KiB

Raw Blame History

SVD Tab Redesign Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Redesign the SVD components tab to split motions by pole, add a 1D party-axis chart, replace the session-state detail pane with inline expand/collapse expanders, and show voting breakdowns per motion.

Architecture: All changes are in explorer.py. Three additive changes (new constant, two new functions) followed by a targeted replacement of the motion-list section in build_svd_components_tab. No new files, no DB schema changes.

Tech Stack: Python, Streamlit, DuckDB, Plotly (plotly.graph_objects already imported), JSON

File Map

File	Change
`explorer.py`	Add `ChristenUnie` to `PARTY_COLOURS` (line ~52)
`explorer.py`	Add `CURRENT_PARLIAMENT_PARTIES` frozenset (line ~69)
`explorer.py`	Add `load_party_axis_scores()` top-level function (after `load_party_map`, line ~179)
`explorer.py`	Add `_render_party_axis_chart()` top-level function (after `load_party_axis_scores`)
`explorer.py`	Replace lines 854–896 in `build_svd_components_tab` with pole-split layout, party chart call, batch DB query, and expanders with voting

Task 1: Add constants

Files:

Modify: explorer.py (two locations: PARTY_COLOURS dict and after KNOWN_MAJOR_PARTIES)
Step 1: Add ChristenUnie to PARTY_COLOURS

In explorer.py, the PARTY_COLOURS dict currently ends at line 52 with "Unknown": "#9E9E9E". Add an entry just before "Unknown":

    "ChristenUnie": "#0288D1",  # alias — stored as ChristenUnie in svd_vectors

Result: PARTY_COLOURS now maps both "CU" (already present via "CU": "#0288D1") and "ChristenUnie".

Wait — "CU" key is NOT in PARTY_COLOURS already (the dict only has full names). The existing code uses "CU" in KNOWN_MAJOR_PARTIES but the lookup key CU has no mapping. Only add "ChristenUnie": "#0288D1".

Step 2: Add CURRENT_PARLIAMENT_PARTIES after KNOWN_MAJOR_PARTIES (line ~69)

# Parties currently seated in the Tweede Kamer (2023 election cycle).
# These are the entity_ids as stored in svd_vectors for window='2025'.
CURRENT_PARLIAMENT_PARTIES: frozenset[str] = frozenset({
    "PVV",
    "VVD",
    "NSC",
    "BBB",
    "D66",
    "GroenLinks-PvdA",
    "CDA",
    "SP",
    "ChristenUnie",
    "SGP",
    "Volt",
    "DENK",
    "PvdD",
    "JA21",
    "FVD",
})

Step 3: Syntax-check and commit

python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
git add explorer.py
git commit -m "feat(explorer): add ChristenUnie colour alias and CURRENT_PARLIAMENT_PARTIES constant"

Task 2: Add `load_party_axis_scores`

Files:

Modify: explorer.py — insert after load_party_map function (currently ends around line 179)
Step 1: Insert the function

Add this function immediately after the closing of load_party_map (the return {} / finally con.close() block, around line 179):

@st.cache_data(show_spinner="Partijposities op SVD-assen laden…")
def load_party_axis_scores(db_path: str) -> Dict[str, List[float]]:
    """Return per-party SVD vectors for window='2025'.

    Queries svd_vectors WHERE entity_type='mp' AND window_id='2025'
    AND entity_id is a known current-parliament party.

    Returns:
        {party_name: [float * k]}  — k = 50 for the canonical 2025 window
    """
    try:
        con = duckdb.connect(database=db_path, read_only=True)
        placeholders = ", ".join("?" for _ in CURRENT_PARLIAMENT_PARTIES)
        rows = con.execute(
            f"SELECT entity_id, vector FROM svd_vectors "
            f"WHERE entity_type='mp' AND window_id='2025' "
            f"AND entity_id IN ({placeholders})",
            list(CURRENT_PARLIAMENT_PARTIES),
        ).fetchall()
        con.close()
        return {
            row[0]: json.loads(row[1]) if isinstance(row[1], str) else list(row[1])
            for row in rows
        }
    except Exception:
        logger.exception("Failed to load party axis scores")
        return {}

Step 2: Syntax-check and commit

python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
git add explorer.py
git commit -m "feat(explorer): add load_party_axis_scores helper"

Task 3: Add `_render_party_axis_chart`

Files:

Modify: explorer.py — insert after load_party_axis_scores
Step 1: Insert the function

Add immediately after load_party_axis_scores:

def _render_party_axis_chart(
    party_scores: Dict[str, List[float]], comp_sel: int
) -> None:
    """Render a 1D horizontal Plotly scatter of party positions on SVD axis `comp_sel`.

    Each party is plotted at its score on a single horizontal axis (y=0).
    """
    if not party_scores:
        st.caption("_Partijdata niet beschikbaar voor deze as._")
        return

    axis_idx = comp_sel - 1  # 0-based index into the 50-dim vector
    data: list[dict] = []
    for party, vec in party_scores.items():
        if axis_idx < len(vec):
            data.append({"party": party, "score": vec[axis_idx]})

    if not data:
        st.caption("_Geen partijscores voor deze as._")
        return

    scores = [d["score"] for d in data]
    parties = [d["party"] for d in data]
    colours = [PARTY_COLOURS.get(p, "#9E9E9E") for p in parties]
    hover = [f"{p}: {s:.3f}" for p, s in zip(parties, scores)]

    fig = go.Figure()
    # Baseline
    x_min, x_max = min(scores) * 1.15, max(scores) * 1.15
    fig.add_trace(
        go.Scatter(
            x=[x_min, x_max],
            y=[0, 0],
            mode="lines",
            line={"color": "#cccccc", "width": 1},
            hoverinfo="skip",
            showlegend=False,
        )
    )
    # Party markers
    fig.add_trace(
        go.Scatter(
            x=scores,
            y=[0] * len(scores),
            mode="markers+text",
            text=parties,
            textposition="top center",
            marker={"size": 12, "color": colours},
            hovertext=hover,
            hoverinfo="text",
            showlegend=False,
        )
    )
    fig.update_layout(
        height=160,
        margin={"l": 10, "r": 10, "t": 10, "b": 30},
        xaxis={
            "title": "← Negatieve pool  |  Positieve pool →",
            "zeroline": True,
            "zerolinecolor": "#aaaaaa",
        },
        yaxis={"visible": False, "range": [-1, 2]},
        plot_bgcolor="white",
    )
    st.plotly_chart(fig, use_container_width=True)

Step 2: Syntax-check and commit

python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"
git add explorer.py
git commit -m "feat(explorer): add _render_party_axis_chart helper"

Task 4: Restructure motion display in `build_svd_components_tab`

Files:

Modify: explorer.py lines 841–896 (theme + poles + motion list + detail pane)

The current code at lines 841–896:

    # Show theme explanation + poles
    theme = SVD_THEMES.get(comp_sel, {})
    if theme:
        st.info(f"**{theme['label']}** — {theme['explanation']}")
        pos = theme.get("positive_pole", "")
        neg = theme.get("negative_pole", "")
        if pos or neg:
            pcol, ncol = st.columns(2)
            with pcol:
                st.success(f"▲ **Positieve pool:** {pos}")
            with ncol:
                st.error(f"▼ **Negatieve pool:** {neg}")

    motions = comp_map.get(comp_sel, [])

    col1, col2 = st.columns([1, 2])
    with col1:
        st.markdown("**Top-moties (titels)**")
        for m in motions:
            mid = m.get("motion_id")
            score = m.get("score", 0.0)
            title = m.get("title") or f"Motie #{mid}"
            sign = "▲" if score >= 0 else "▼"
            if st.button(f"{sign} {mid}: {title[:72]}", key=f"btn_{comp_sel}_{mid}"):
                st.session_state["svd_selected_mid"] = mid

    with col2:
        sel_mid = st.session_state.get("svd_selected_mid")
        if not sel_mid and motions:
            sel_mid = motions[0].get("motion_id")
    if sel_mid:
        # fetch motion metadata from DB for completeness
        try:
            con = duckdb.connect(database=db_path, read_only=True)
            row = con.execute(
                "SELECT id, title, date, policy_area, url, body_text FROM motions WHERE id=?",
                [int(sel_mid)],
            ).fetchone()
            con.close()
        except Exception:
            row = None

        if row:
            st.markdown(f"### {row[1] or f'Motie #{row[0]}'}")
            try:
                date_str = str(row[2])[:10]
            except Exception:
                date_str = "?"
            st.caption(f"📅 {date_str}  |  {row[3]}")
            if row[4] and str(row[4]).startswith("http"):
                st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})")
            if row[5]:
                with st.expander("Toon volledige tekst"):
                    st.write(row[5])
        else:
            st.info(f"Metadata not found in DB for motion {sel_mid}")

Replace entirely with:

    # Show theme explanation
    theme = SVD_THEMES.get(comp_sel, {})
    if theme:
        st.info(f"**{theme['label']}** — {theme['explanation']}")

    motions = comp_map.get(comp_sel, [])

    # Party axis chart
    party_scores = load_party_axis_scores(db_path)
    _render_party_axis_chart(party_scores, comp_sel)

    # Batch-fetch motion details (title, date, policy_area, url, body_text, voting_results)
    motion_ids = [m.get("motion_id") for m in motions if m.get("motion_id") is not None]
    motion_details: Dict[int, tuple] = {}
    if motion_ids:
        try:
            placeholders = ", ".join("?" for _ in motion_ids)
            con = duckdb.connect(database=db_path, read_only=True)
            db_rows = con.execute(
                f"SELECT id, title, date, policy_area, url, body_text, voting_results "
                f"FROM motions WHERE id IN ({placeholders})",
                [int(mid) for mid in motion_ids],
            ).fetchall()
            con.close()
            motion_details = {r[0]: r for r in db_rows}
        except Exception:
            logger.exception("Failed to batch-fetch motion details")

    # Split motions by pole sign
    pos_motions = [m for m in motions if float(m.get("score", 0.0)) >= 0]
    neg_motions = [m for m in motions if float(m.get("score", 0.0)) < 0]

    pos_pole = theme.get("positive_pole", "Positieve pool") if theme else "Positieve pool"
    neg_pole = theme.get("negative_pole", "Negatieve pool") if theme else "Negatieve pool"

    pcol, ncol = st.columns(2)

    with pcol:
        st.success(f"▲ **Positieve pool:** {pos_pole}")
        for m in pos_motions:
            mid = m.get("motion_id")
            raw_title = m.get("title") or f"Motie #{mid}"
            with st.expander(f"▲ {raw_title[:80]}"):
                row = motion_details.get(int(mid)) if mid is not None else None
                if row:
                    try:
                        date_str = str(row[2])[:10]
                    except Exception:
                        date_str = "?"
                    st.caption(f"📅 {date_str}  |  {row[3] or '—'}")
                    if row[4] and str(row[4]).startswith("http"):
                        st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})")
                    if row[5]:
                        with st.expander("Toon volledige tekst"):
                            st.write(row[5])
                    _render_voting_results(row[6])
                else:
                    st.caption("_Geen metadata beschikbaar_")

    with ncol:
        st.error(f"▼ **Negatieve pool:** {neg_pole}")
        for m in neg_motions:
            mid = m.get("motion_id")
            raw_title = m.get("title") or f"Motie #{mid}"
            with st.expander(f"▼ {raw_title[:80]}"):
                row = motion_details.get(int(mid)) if mid is not None else None
                if row:
                    try:
                        date_str = str(row[2])[:10]
                    except Exception:
                        date_str = "?"
                    st.caption(f"📅 {date_str}  |  {row[3] or '—'}")
                    if row[4] and str(row[4]).startswith("http"):
                        st.markdown(f"[🔗 Bekijk op Tweede Kamer]({row[4]})")
                    if row[5]:
                        with st.expander("Toon volledige tekst"):
                            st.write(row[5])
                    _render_voting_results(row[6])
                else:
                    st.caption("_Geen metadata beschikbaar_")

Step 1: Apply the replacement

Use the Edit tool to replace the block from # Show theme explanation + poles through the closing st.info(f"Metadata not found in DB for motion {sel_mid}") line with the new code above.

Step 2: Syntax-check

python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"

Expected output: OK

Step 3: Commit

git add explorer.py
git commit -m "feat(explorer): restructure SVD tab — pole-split motions, party axis chart, inline expanders with voting"

Final Verification

Syntax check

python3 -c "import ast; ast.parse(open('explorer.py').read()); print('OK')"

Smoke test (manual): Run streamlit run explorer.py, open the SVD tab, verify:
- Selectbox shows axis labels
- Explanation text renders
- Party axis chart renders with ~15 coloured dots
- Positive motions appear in green (left) column, negative in red (right) column
- Clicking an expander shows date, URL, body text sub-expander, and voting breakdown
- Switching axes updates all sections correctly

14 KiB Raw Blame History

SVD Tab Redesign Implementation Plan

File Map

Task 1: Add constants

Task 2: Add load_party_axis_scores

Task 3: Add _render_party_axis_chart

Task 4: Restructure motion display in build_svd_components_tab

Final Verification

14 KiB

Raw Blame History

Task 2: Add `load_party_axis_scores`

Task 3: Add `_render_party_axis_chart`

Task 4: Restructure motion display in `build_svd_components_tab`