motief/docs/superpowers/plans/2026-03-28-compass-ui-improvements.md

Compass UI Improvements Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Fix three independent UI issues in the political compass: (1) update stale axis 3/4/5 descriptions in SVD_THEMES, (2) fix broken Y-axis direction arrows, (3) add voting discipline section below compass.

Architecture: All changes are confined to explorer.py. No new files. No schema changes. The discipline helper reads mp_votes read-only via DuckDB. Tests for the discipline function live in tests/test_political_compass.py.

Tech Stack: Python, Streamlit, Plotly Express/Graph Objects, DuckDB, pytest (run via uv run pytest)

---

File Map

File	Change
explorer.py	Update SVD_THEMES axes 3–5; fix Y-axis labels in two px.scatter calls and one update_layout; add compute_party_discipline; add discipline rendering in build_compass_tab
tests/test_political_compass.py	Add tests for compute_party_discipline

---

Task 1: Update SVD_THEMES axes 3, 4, 5

Files:

• Modify: explorer.py:1156–1204

These descriptions were written for an earlier dataset. The new text reflects stable multi-year patterns (not a single year's snapshot). The flip booleans are unchanged.

New text for axis 3 (flip=True — currently "Sociaal-economisch links versus marktliberaal en landelijk rechts"):

The positive pole consistently shows SP, PvdD, GL-PvdA on social welfare motions; the negative pole shows VVD on market-oriented motions. But PVV also appears prominently positive (anti-establishment spending motions), meaning this is not a clean left-right economic axis — it's more accurately described as state intervention versus market liberalism, with populist anti-establishment motions on the same side as the socialist left.

New text for axis 4 (flip=True — currently "Christelijk-sociaal centrum versus populistisch-soevereinistisch"):

NSC, SGP, CU and CDA consistently dominate the positive pole; VVD and GL-PvdA the negative. FVD scores near zero across years. The axis captures religious-conservative institutionalism vs secular liberalism, not populism vs mainstream.

New text for axis 5 (flip=False — currently "Christelijk-conservatief en ruraal sociaal versus seculier-progressief"):

CDA, CU, SGP score positive; SP, PvdD, DENK score negative. D66 tends positive (not strongly negative), and NSC tends negative. The stable pattern is established-institutional vs protest-populist rather than a religious/secular split.

• Step 1: Replace axis 3 entry in explorer.py

In explorer.py, replace lines 1156–1171:

        3: {
            "label": "Staatsingrijpen en publieke sector versus marktliberalisme",
            "explanation": (
                "Deze as weerspiegelt de spanning tussen staatsingrijpen en marktliberalisme. "
                "Aan de positieve kant staan SP-moties die bezuinigingen op zorg en gemeentefonds "
                "willen terugdraaien, winstuitkeringen in de zorg verbieden en publieke controle "
                "over fusies eisen. Ook PVV stemt positief — niet vanuit sociaal ideaal maar vanuit "
                "anti-establishment populisme dat neigt naar overheidsinterventie voor de eigen achterban. "
                "Aan de negatieve kant staan VVD-moties over marktwerking en deregulering, en NSC- en "
                "BBB-moties met een agrarisch-marktgericht karakter. "
                "Samengevat: de as scheidt voorstanders van staatsingrijpen (links én populistisch-rechts) "
                "van marktliberalen en agrarisch-rechts."
            ),
            "positive_pole": "Staatsingrijpen: SP, PvdD, GL-PvdA, PVV (populistisch)",
            "negative_pole": "Marktliberaal en agrarisch-rechts: VVD, NSC, BBB",
            "flip": True,
        },

• Step 2: Replace axis 4 entry in explorer.py

Replace lines 1172–1188:

        4: {
            "label": "Christelijk-conservatief institutionalisme versus seculier-liberalisme",
            "explanation": (
                "Deze as scheidt christelijk-conservatieve partijen die hechten aan traditionele "
                "instituties en religieuze waarden (NSC, SGP, CU, CDA) van seculier-liberale partijen "
                "die nadruk leggen op individuele rechten en internationale openheid (VVD, GL-PvdA). "
                "CU-moties over vaderbetrokkenheid, huwelijksrecht en internationale samenwerking staan "
                "aan de positieve kant; VVD-moties over marktregulering en D66-moties over internationale "
                "verdragen aan de negatieve kant. FVD scoort dicht bij nul — het past noch in het "
                "christelijk-conservatieve noch in het seculier-liberale kamp op deze as. "
                "Dit is geen populisme-as maar een religieus-institutionele breuklijn."
            ),
            "positive_pole": "Christelijk-conservatief institutioneel: NSC, SGP, CU, CDA",
            "negative_pole": "Seculier-liberaal: VVD, GL-PvdA, D66",
            "flip": True,
        },

• Step 3: Replace axis 5 entry in explorer.py

Replace lines 1189–1204:

        5: {
            "label": "Gevestigd-institutioneel versus protest en populistisch",
            "explanation": (
                "Deze as scheidt gevestigde centrumpartijen die vertrouwen op bestaande instituties "
                "(CDA, CU, SGP, D66) van protest- en populistische partijen die dat vertrouwen "
                "afwijzen (SP, PvdD, DENK, NSC). CDA-moties over vrijwilligers in schuldhulp, "
                "maatschappelijke diensttijd en WW-hervorming staan aan de positieve kant. "
                "SP- en PvdD-moties over meerouderschap, abortusrecht en buitenlandse beïnvloeding "
                "staan aan de negatieve kant. NSC scoort negatief — ondanks zijn christelijk-conservatieve "
                "karakter op andere assen is het hier een protest­partij die systeemkritiek uitdraagt. "
                "D66 scoort licht positief, consistent met zijn institutionele en pro-EU profiel."
            ),
            "positive_pole": "Gevestigd-institutioneel: CDA, CU, SGP, D66",
            "negative_pole": "Protest en populistisch: SP, PvdD, DENK, NSC",
            "flip": False,
        },

• Step 4: Run tests to confirm nothing broken

uv run pytest tests/test_political_compass.py -v

Expected: all 3 tests PASS (these tests don't touch SVD_THEMES).

• Step 5: Commit

git add explorer.py
git commit -m "fix: update SVD_THEMES axes 3-5 descriptions to reflect stable multi-year patterns"

---

Task 2: Fix Y-axis direction indicators in compass and trajectories

Files:

• Modify: explorer.py:810–812 (partijen scatter labels)
• Modify: explorer.py:830 (kamerleden scatter labels)
• Modify: explorer.py:833–838 (compass update_layout — add helper call)
• Modify: explorer.py:921–927 (trajectories update_layout)

Plotly rotates Y-axis titles 90° counter-clockwise, so ↑ and ↓ in the title string point sideways. Fix: strip arrows from the axis title; add two fig.add_annotation calls to place ▲ Progressief at the top and ▼ Conservatief at the bottom of the chart area using xref="paper", yref="paper".

• Step 1: Add _add_y_direction_annotations helper near top of build_compass_tab

Add this function just before build_compass_tab (after line 692, before line 694):

def _add_y_direction_annotations(fig: go.Figure) -> None:
    """Add ▲ Progressief / ▼ Conservatief labels above and below the Y axis."""
    common = dict(
        xref="paper",
        yref="paper",
        x=-0.07,
        showarrow=False,
        font=dict(size=11, color="#666666"),
    )
    fig.add_annotation(**common, y=1.02, text="▲ Progressief", xanchor="center")
    fig.add_annotation(**common, y=-0.06, text="▼ Conservatief", xanchor="center")

• Step 2: Update labels in the "Partijen" scatter (line 810–814)

Change:

            labels={
                "x": "Links ← → Rechts",
                "y": "Progressief ↑ / Conservatief ↓",
                "n": "Kamerleden",
            },

To:

            labels={
                "x": "Links ← → Rechts",
                "y": "Progressief / Conservatief",
                "n": "Kamerleden",
            },

• Step 3: Update labels in the "Kamerleden" scatter (line 830)

Change:

            labels={"x": "Links ← → Rechts", "y": "Progressief ↑ / Conservatief ↓"},

To:

            labels={"x": "Links ← → Rechts", "y": "Progressief / Conservatief"},

• Step 4: Call the annotation helper after fig.update_layout in build_compass_tab (after line 838)

Change:

    fig.update_layout(
        height=600,
        legend_title_text="Partij",
        xaxis={"range": [-1, 1]},
        yaxis={"range": [-0.6, 0.6]},
    )

    with col1:
        st.plotly_chart(fig, use_container_width=True)

To:

    fig.update_layout(
        height=600,
        legend_title_text="Partij",
        xaxis={"range": [-1, 1]},
        yaxis={"range": [-0.6, 0.6]},
    )
    _add_y_direction_annotations(fig)

    with col1:
        st.plotly_chart(fig, use_container_width=True)

• Step 5: Fix trajectories Y-axis title (line 924) and add annotation

Change build_trajectories_tab update_layout block:

    fig.update_layout(
        title="Partij trajectories",
        xaxis_title="Links ← → Rechts",
        yaxis_title="Progressief ↑ / Conservatief ↓",
        height=600,
        legend_title_text="Partij",
    )
    st.plotly_chart(fig, use_container_width=True)

To:

    fig.update_layout(
        title="Partij trajectories",
        xaxis_title="Links ← → Rechts",
        yaxis_title="Progressief / Conservatief",
        height=600,
        legend_title_text="Partij",
    )
    _add_y_direction_annotations(fig)
    st.plotly_chart(fig, use_container_width=True)

• Step 6: Run tests

uv run pytest tests/test_political_compass.py -v

Expected: all 3 tests PASS.

• Step 7: Commit

git add explorer.py
git commit -m "fix: replace sideways Y-axis arrows with proper top/bottom annotations"

---

Task 3: Add voting discipline section below compass

Files:

• Modify: explorer.py — add compute_party_discipline function; add rendering block in build_compass_tab
• Modify: tests/test_political_compass.py — add two tests

3a: Write the failing tests first

• Step 1: Add tests to tests/test_political_compass.py

Append at the end of tests/test_political_compass.py:

# ---------------------------------------------------------------------------
# Tests for compute_party_discipline
# ---------------------------------------------------------------------------


def _make_mp_votes_db():
    """Create an in-memory DuckDB with mp_votes fixture data.

    6 motions, 2 parties (SP, VVD), each with 4 MPs.
    SP is perfectly disciplined (all 4 vote the same each time).
    VVD has 1 dissident on 2 of 6 motions → Rice index = (4+4+4+4+3+3)/6/4 ≈ 0.917.
    Dates span 2023-01-01 to 2023-12-31.
    """
    import duckdb

    conn = duckdb.connect(":memory:")
    conn.execute("""
        CREATE TABLE mp_votes (
            id INTEGER,
            motion_id VARCHAR,
            mp_name VARCHAR,
            party VARCHAR,
            vote VARCHAR,
            date DATE,
            created_at TIMESTAMP
        )
    """)
    rows = []
    # motions 1-6, dates in 2023
    dates = [
        "2023-01-10",
        "2023-03-15",
        "2023-05-20",
        "2023-07-25",
        "2023-09-30",
        "2023-11-05",
    ]
    sp_mps = ["Janssen, A.", "Pietersen, B.", "Willemsen, C.", "Hendriksen, D."]
    vvd_mps = ["Adams, E.", "Bakker, F.", "Claassen, G.", "Dekker, H."]
    for i, date in enumerate(dates, start=1):
        m_id = f"M{i:03d}"
        # SP: all vote 'voor' every motion (perfectly disciplined)
        for mp in sp_mps:
            rows.append((i * 10 + 1, m_id, mp, "SP", "voor", date, "2023-01-01"))
        # VVD: motions 5 and 6 have one dissident (votes 'tegen' while others vote 'voor')
        if i <= 4:
            for mp in vvd_mps:
                rows.append((i * 10 + 2, m_id, mp, "VVD", "voor", date, "2023-01-01"))
        else:
            for mp in vvd_mps[:3]:
                rows.append((i * 10 + 2, m_id, mp, "VVD", "voor", date, "2023-01-01"))
            rows.append((i * 10 + 3, m_id, vvd_mps[3], "VVD", "tegen", date, "2023-01-01"))
    conn.executemany(
        "INSERT INTO mp_votes VALUES (?, ?, ?, ?, ?, ?, ?)", rows
    )
    return conn


def test_compute_party_discipline_basic(monkeypatch):
    """compute_party_discipline returns correct Rice index for fixture data."""
    import duckdb as _duckdb

    fixture_conn = _make_mp_votes_db()

    monkeypatch.setattr(
        _duckdb, "connect", lambda path, **kw: fixture_conn
    )

    # Import after monkeypatch so explorer can be imported without Streamlit crashing
    import importlib
    import sys

    # explorer imports streamlit — provide a minimal stub if not already stubbed
    if "streamlit" not in sys.modules:
        import types
        st_stub = types.ModuleType("streamlit")
        st_stub.cache_data = lambda **kw: (lambda f: f)
        sys.modules["streamlit"] = st_stub

    # We need the function directly; import the module
    import explorer as _explorer
    importlib.reload(_explorer)

    df = _explorer.compute_party_discipline(
        db_path="dummy",
        start_date="2023-01-01",
        end_date="2023-12-31",
    )

    assert not df.empty
    assert set(df.columns) >= {"party", "n_motions", "discipline"}

    sp_row = df[df["party"] == "SP"].iloc[0]
    vvd_row = df[df["party"] == "VVD"].iloc[0]

    assert sp_row["n_motions"] == 6
    assert sp_row["discipline"] == pytest.approx(1.0, abs=1e-6)

    assert vvd_row["n_motions"] == 6
    # 4 motions fully disciplined (4/4=1.0), 2 motions with one dissident (3/4=0.75)
    expected_vvd = (4 * 1.0 + 2 * 0.75) / 6
    assert vvd_row["discipline"] == pytest.approx(expected_vvd, abs=1e-4)

    # All values in [0, 1]
    assert (df["discipline"] >= 0).all() and (df["discipline"] <= 1).all()


def test_compute_party_discipline_empty_range(monkeypatch):
    """Returns empty DataFrame when no motions fall in the date range."""
    import duckdb as _duckdb

    fixture_conn = _make_mp_votes_db()
    monkeypatch.setattr(_duckdb, "connect", lambda path, **kw: fixture_conn)

    import importlib, sys

    if "streamlit" not in sys.modules:
        import types
        st_stub = types.ModuleType("streamlit")
        st_stub.cache_data = lambda **kw: (lambda f: f)
        sys.modules["streamlit"] = st_stub

    import explorer as _explorer
    importlib.reload(_explorer)

    df = _explorer.compute_party_discipline(
        db_path="dummy",
        start_date="2000-01-01",
        end_date="2000-12-31",
    )

    assert df.empty

• Step 2: Run the failing tests

uv run pytest tests/test_political_compass.py::test_compute_party_discipline_basic tests/test_political_compass.py::test_compute_party_discipline_empty_range -v

Expected: FAIL with AttributeError: module 'explorer' has no attribute 'compute_party_discipline'

3b: Implement compute_party_discipline

• Step 3: Add compute_party_discipline to explorer.py

Add this function after the load_active_mps function (find it, then place compute_party_discipline immediately after). The function must be a plain function (not decorated with @st.cache_data) so tests can monkeypatch duckdb.connect.

def compute_party_discipline(
    db_path: str,
    start_date: str,
    end_date: str,
) -> pd.DataFrame:
    """Compute per-party voting discipline (Rice index) for roll-call votes in a date range.

    Only individual MP vote rows are used (mp_name LIKE '%,%').
    Returns a DataFrame with columns [party, n_motions, discipline] sorted by discipline ascending.
    Returns an empty DataFrame if fewer than 1 qualifying motion exists or on any DB error.

    Rice index per motion per party = fraction of party MPs voting with the party majority.
    The per-party score is the average Rice index across all motions in the date range.
    """
    try:
        conn = duckdb.connect(db_path, read_only=True)
        result = conn.execute(
            """
            WITH individual_votes AS (
                -- Only individual MP rows (mp_name contains a comma, e.g. "Janssen, A.")
                SELECT
                    motion_id,
                    party,
                    LOWER(vote) AS vote
                FROM mp_votes
                WHERE mp_name LIKE '%,%'
                  AND date >= CAST(? AS DATE)
                  AND date <= CAST(? AS DATE)
                  AND vote IN ('voor', 'tegen', 'afwezig', 'onthouden')
            ),
            vote_counts AS (
                -- Count each vote token per (motion, party)
                SELECT
                    motion_id,
                    party,
                    vote,
                    COUNT(*) AS cnt
                FROM individual_votes
                GROUP BY motion_id, party, vote
            ),
            majority_vote AS (
                -- Determine the majority vote token per (motion, party)
                SELECT
                    motion_id,
                    party,
                    FIRST(vote ORDER BY cnt DESC, vote ASC) AS maj_vote,
                    SUM(cnt) AS total_mp_votes
                FROM vote_counts
                GROUP BY motion_id, party
            ),
            rice_per_motion AS (
                -- Rice index: fraction voting with majority
                SELECT
                    mv.motion_id,
                    mv.party,
                    SUM(CASE WHEN vc.vote = mv.maj_vote THEN vc.cnt ELSE 0 END)
                        * 1.0 / mv.total_mp_votes AS rice
                FROM majority_vote mv
                JOIN vote_counts vc
                  ON mv.motion_id = vc.motion_id AND mv.party = vc.party
                GROUP BY mv.motion_id, mv.party, mv.total_mp_votes
            )
            SELECT
                party,
                COUNT(DISTINCT motion_id) AS n_motions,
                AVG(rice) AS discipline
            FROM rice_per_motion
            GROUP BY party
            ORDER BY discipline ASC
            """,
            [start_date, end_date],
        ).fetchdf()
        conn.close()
        return result
    except Exception as exc:
        logger.warning("compute_party_discipline failed: %s", exc)
        return pd.DataFrame(columns=["party", "n_motions", "discipline"])

• Step 4: Run the tests to confirm they pass

uv run pytest tests/test_political_compass.py::test_compute_party_discipline_basic tests/test_political_compass.py::test_compute_party_discipline_empty_range -v

Expected: both PASS.

• Step 5: Run all compass tests

uv run pytest tests/test_political_compass.py -v

Expected: all 5 tests PASS.

3c: Render discipline section in build_compass_tab

• Step 6: Add _window_to_dates helper just before build_compass_tab

Add this function just before build_compass_tab (around line 692, after the _add_y_direction_annotations helper added in Task 2):

def _window_to_dates(window_id: str) -> tuple[str, str]:
    """Return (start_date, end_date) ISO strings for a given window_id.

    Annual windows like '2024' → ('2024-01-01', '2024-12-31').
    'current_parliament' → ('2023-11-22', '2099-12-31') (2023 formation date, open end).
    Unknown formats → ('2000-01-01', '2099-12-31') (effectively all time).
    """
    if window_id == "current_parliament":
        return ("2023-11-22", "2099-12-31")
    if re.fullmatch(r"\d{4}", window_id):
        return (f"{window_id}-01-01", f"{window_id}-12-31")
    # Quarterly e.g. '2020-Q3' → 2020-07-01 to 2020-09-30
    m = re.fullmatch(r"(\d{4})-Q([1-4])", window_id)
    if m:
        year, q = int(m.group(1)), int(m.group(2))
        starts = {1: "01-01", 2: "04-01", 3: "07-01", 4: "10-01"}
        ends = {1: "03-31", 2: "06-30", 3: "09-30", 4: "12-31"}
        return (f"{year}-{starts[q]}", f"{year}-{ends[q]}")
    return ("2000-01-01", "2099-12-31")

• Step 7: Add discipline rendering after st.plotly_chart in build_compass_tab

The current end of build_compass_tab is (around line 840–841):

    with col1:
        st.plotly_chart(fig, use_container_width=True)

Add the discipline section immediately after (still inside the function, but outside the with col1: block):

    # --- Voting discipline section ---
    _MIN_MOTIONS_FOR_DISCIPLINE = 5
    start_date, end_date = _window_to_dates(window_idx)
    disc_df = compute_party_discipline(db_path, start_date, end_date)

    st.subheader("Stemgedrag cohesie")
    if disc_df.empty or disc_df["n_motions"].max() < _MIN_MOTIONS_FOR_DISCIPLINE:
        st.caption(
            "Te weinig hoofdelijke stemmingen in dit venster voor een cohesieanalyse."
        )
    else:
        # Filter to parties that appear in the compass
        compass_parties = set(df_pos["party"].unique())
        disc_df = disc_df[disc_df["party"].isin(compass_parties)].copy()

        if disc_df.empty:
            st.caption("Geen overlappende partijen tussen kompas en stemmingsdata.")
        else:
            disc_df["discipline_pct"] = (disc_df["discipline"] * 100).round(1)
            disc_df["party_label"] = disc_df.apply(
                lambda r: f"{r['party']} ({int(r['n_motions'])} moties)", axis=1
            )

            bar_fig = px.bar(
                disc_df.sort_values("discipline"),
                x="discipline_pct",
                y="party_label",
                orientation="h",
                color="discipline_pct",
                color_continuous_scale="RdYlGn",
                range_color=[80, 100],
                labels={"discipline_pct": "Cohesie (%)", "party_label": "Partij"},
                title="Cohesie bij hoofdelijke stemmingen",
            )
            bar_fig.update_layout(
                height=max(300, len(disc_df) * 35 + 80),
                showlegend=False,
                coloraxis_showscale=False,
                yaxis_title="",
            )
            st.plotly_chart(bar_fig, use_container_width=True)

            # Extremes table
            top3 = disc_df.nlargest(3, "discipline")[["party", "discipline_pct", "n_motions"]]
            bot3 = disc_df.nsmallest(3, "discipline")[["party", "discipline_pct", "n_motions"]]
            col_a, col_b = st.columns(2)
            with col_a:
                st.markdown("**Meest eensgezind**")
                st.dataframe(
                    top3.rename(columns={"party": "Partij", "discipline_pct": "Cohesie (%)", "n_motions": "Moties"}),
                    hide_index=True,
                    use_container_width=True,
                )
            with col_b:
                st.markdown("**Meest verdeeld**")
                st.dataframe(
                    bot3.rename(columns={"party": "Partij", "discipline_pct": "Cohesie (%)", "n_motions": "Moties"}),
                    hide_index=True,
                    use_container_width=True,
                )

• Step 8: Run all tests

uv run pytest tests/test_political_compass.py -v

Expected: all 5 tests PASS.

• Step 9: Commit

git add explorer.py tests/test_political_compass.py
git commit -m "feat: add voting discipline section below political compass"

---

Self-Review Checklist

• Spec coverage: Task 1 → SVD_THEMES axes 3–5. Task 2 → Y-axis arrows. Task 3 → discipline function + rendering. All three design requirements covered.
• Placeholder scan: No TBD or TODO. All code blocks are complete.
• Type consistency: compute_party_discipline returns pd.DataFrame with columns ["party", "n_motions", "discipline"] — referenced consistently in tests and rendering code. _window_to_dates returns tuple[str, str] — used as start_date, end_date in rendering.
• Test isolation: Tests monkeypatch duckdb.connect to return in-memory DB; tests add a minimal streamlit stub to avoid import errors. Both patterns match existing test style in the file.
• Edge case: _MIN_MOTIONS_FOR_DISCIPLINE = 5 guard ensures the section degrades gracefully for sparse windows. Empty DataFrame from compute_party_discipline is also handled.