# "Welk tweede kamerlid ben jij?" Implementation Plan **Goal:** Add a Streamlit quiz tab that interactively asks the user motion (vote) questions and narrows the set of 2026 MPs to find the best-matching MP. Implement two DB helpers (matching + discriminating-motion selection), the UI builder and tab wiring, and tests. Minimal viable changes only — no UX bells & whistles. **Design:** thoughts/shared/designs/2026-03-24-welk-tweede-kamerlid-ben-jij-design.md --- ## Dependency Graph ``` Batch 1 (parallel): 1.1 [foundation - no deps], 1.2 (plan file) [none] Batch 2 (parallel): 2.1 [explorer UI - depends: 1.1] Batch 3 (parallel): 3.1 [integration tests - depends: 1.1,2.1] ``` --- ## Summary of implementation decisions (gap-filling) - MotionDatabase.match_mps_for_votes: implement as a read-only DuckDB-backed method on the existing MotionDatabase class (database.py). It accepts user_votes: Dict[int, str] where keys are motion ids and values are UI vote tokens. I will implement vote normalization inside the method (mapping UI tokens to canonical DB tokens) to avoid touching other modules. Rationale: keeps surface changes minimal and avoids creating new modules. - MotionDatabase.choose_discriminating_motions: implement in the same file. For a small candidate set (expected << 200 MPs), fetch mp_votes for candidate MPs across candidate motions (excluding already-answered motion ids). Score candidate motions by information-entropy of vote distribution among remaining candidates (higher entropy = better split). Ties broken by controversy_score then motion id. - Explorer UI changes: add build_mp_quiz_tab(db_path) to explorer.py and wire it into the tabs list and fallback radio. Use st.session_state['mp_quiz_votes'] to store answers as mapping str(motion_id)->UI token. Use @st.cache_data on any expensive DB-calls in the UI layer. - Vote token normalization: UI will present choices: "Voor", "Tegen", "Onthouden", "Afwezig / Geen stem". The DB stores lowercase tokens like 'voor', 'tegen', 'onthouden', 'afwezig'. match_mps_for_votes will normalize case and a small set of variants (e.g., 'Geen stem' -> 'afwezig', 'Abstain' -> 'onthouden') — explicit list included in tests. --- ## BATCH 1: Foundation (parallel - N implementers) All tasks in this batch have NO dependencies and can run simultaneously. ### Task 1.1: Add DB helpers to MotionDatabase **File:** `database.py` (modify existing) **Test:** `tests/test_match_mps.py` **Depends:** none Description / Acceptance criteria: - Add two new public methods to MotionDatabase: - match_mps_for_votes(user_votes: Dict[int, str], limit: int = 50) -> List[Dict] - Returns an ordered list (desc by agreement_pct) of dicts with keys: mp_name, party, matched (int), overlap (int), agreement_pct (float 0-100). - Behavior: for each mp present in mp_votes for any of the provided motions compute: - overlap = number of motions where MP has a recorded vote AND the user provided a non-empty vote (i.e., not "Geen stem"). - matched = number of those overlaps where normalized(mp_vote) == normalized(user_vote). - agreement_pct = matched / overlap * 100 rounded to 1 decimal. MPs with overlap==0 are excluded from the returned list. - Ordering: agreement_pct desc, then matched desc, then mp_name asc. - choose_discriminating_motions(candidates: List[str], excluded_motion_ids: List[int], k: int = 1) -> List[int] - For the provided candidate mp_names, compute vote distributions per motion (voor/tegen/onthouden/afwezig) excluding motion ids in excluded_motion_ids. - Score each motion by Shannon entropy of the distribution among the candidate MPs (treating 'afwezig' as a separate bucket). Higher entropy preferred. - Return top-k motion ids as a list, tiebreakers: higher controversy_score (motions table) then lower motion id. Implementation notes & decisions: - Implement normalization inside these methods. Normalization mapping (DB vote -> canonical): map DB votes lowercased to one of {'voor','tegen','onthouden','afwezig'}. UI inputs (Voor/Tegen/Onthouden/Geen stem) normalized to these same tokens. - For performance, implement SQL queries that select mp_votes filtered by motion_id IN (...) and mp_name IN (candidates) and aggregate via GROUP BY mp_name and vote. For small candidate sets and a limited set of motion_ids this will be fast. If duckdb is not available, fall back to in-Python aggregates using the file-backed JSON format already present in MotionDatabase._init_database. - Add docstrings and basic parameter validation (raise ValueError for empty user_votes or empty candidates input). Tests will cover expected exceptions. Test outline (tests/test_match_mps.py): - Setup: create a temporary MotionDatabase using a temp db_path (MotionDatabase.reset_database() can be used if duckdb available; otherwise use file-backed mode). Insert a small set of motions and mp_votes via insert_motion / insert_mp_vote. Create at least 3 MPs with overlapping but distinct vote patterns across 4-6 motions. - Tests: 1) test_match_basic_counts: user_votes covering 3 motions returns expected matched/overlap/agreement_pct per MP. 2) test_match_excludes_zero_overlap: MPs with no recorded votes for provided motions are excluded. 3) test_choose_discriminating_motions_entropy_ranking: with a small candidate set, the chosen motion(s) split candidates as expected (assert returned motion id is one of known good splitters) 4) test_invalid_input: calling match_mps_for_votes with empty user_votes raises ValueError. Verify: `pytest -q tests/test_match_mps.py` Commit message: `feat(database): add match_mps_for_votes and choose_discriminating_motions` Estimated time: 3.0 - 4.5 hours --- ### Task 1.2: Add plan file (this document) **File:** `thoughts/shared/plans/2026-03-24-welk-tweede-kamerlid-ben-jij-plan.md` (this file) **Test:** none **Depends:** none Description: Add the implementation plan (this document) to the repo to provide step-by-step microtasks to implementers. No tests. Verify: visually review file in repo. No test run. Commit message: `docs(plans): add plan for 'Welk tweede kamerlid ben jij?'` Estimated time: 0.25 - 0.5 hours --- ## BATCH 2: Core UI (parallel - depends on Batch 1) All tasks in this batch assume the DB methods from Task 1.1 exist. ### Task 2.1: Add Streamlit quiz tab & wiring **File:** `explorer.py` (modify existing) **Test:** `tests/test_explorer_quiz.py` **Depends:** 1.1 Description / Acceptance criteria: - Add a function `build_mp_quiz_tab(db_path: str) -> None` placed near other build_*_tab functions (as described in the design, e.g., after build_svd_components_tab or near the top of the tab builders). The function must: - Render a short intro/instructions. - Load an initial pool of candidate motions using existing `load_motions_df(db_path)` and pick a seed set (top N by controversy_score). Decision: seed N = 8 (configurable constant in the function: SEED_MOTIONS = 8) — this is small and fast. - Present questions one at a time: show motion title + layman_explanation (if available) and a radio with choices: "Voor", "Tegen", "Onthouden", "Geen stem" and a "Skip"/"Niet zeker" optional button mapped to "Geen stem". Choice stored to `st.session_state['mp_quiz_votes']` as mapping with keys str(motion_id) -> UI token. - After each answer, call MotionDatabase.match_mps_for_votes(user_votes) to fetch ranked candidates and display a small DataFrame (top 10) with columns: MP name, party, matched, overlap, agreement_pct. Use st.dataframe. - If more than 1 candidate remains with top agreement_pct tied, call MotionDatabase.choose_discriminating_motions(candidates, excluded_motion_ids) to pick the next question to ask and continue until one unique MP remains or choose_discriminating_motions returns an empty list (tie / indistinguishable). Cap total questions at 20 (SESS_CAP = 20). - When unique MP is found (agreement_pct == 100 and overlap>0 and only one MP with highest agreement), show final MP summary (name, party) and their matching motions count. - Use caching: wrap any repeated DB lookups (e.g., load_motions_df already cached) and mark heavy updates via @st.cache_data where appropriate. Implementation notes & decisions: - Keep all UI state local to st.session_state with keys prefixed `mp_quiz_` to avoid collisions. - Normalize UI tokens before sending to DB helper (but DB methods will also normalize; duplication is defensive). - Keep the UI function self-contained in explorer.py (do not create new modules for this minimal MVP). Test outline (tests/test_explorer_quiz.py): - Use monkeypatching to inject a MotionDatabase mock into explorer module or run in a test DB using MotionDatabase with temp db_path. The test must be import-safe (explorer.py imports many heavy libs), so follow pattern used by existing tests/test_explorer_import.py: import the module and assert `build_mp_quiz_tab` exists and is callable. - Functional assertions: 1) test_builder_exists: import explorer, assert callable(build_mp_quiz_tab) 2) test_ui_state_update_simulation: simulate st.session_state by creating a fake session dict (use monkeypatch to set st.session_state to a dict-like object) and calling build_mp_quiz_tab with a small temp DB where motions and mp_votes are prepared. Assert that after calling the builder with pre-filled votes the DataFrame block would display ranked candidates (test inspects returned structure if builder returns it, or else monkeypatch MotionDatabase.match_mps_for_votes to verify it was called with expected mapping). Verification: `pytest -q tests/test_explorer_quiz.py` Commit message: `feat(ui): add 'Welk tweede kamerlid ben jij?' tab and wiring in explorer.py` Estimated time: 2.0 - 4.0 hours --- ## BATCH 3: Integration & Tests (parallel - depends on Batches 1+2) ### Task 3.1: Add integration test for quiz flow **File:** `tests/test_explorer_quiz_integration.py` **Test:** this file **Depends:** 1.1, 2.1 Description / Acceptance criteria: - Create an end-to-end-ish headless test that: - Sets up a temporary MotionDatabase instance (temp file path) and inserts a small controlled dataset: ~6 motions, 4 MPs with distinct votes. - Calls build_mp_quiz_tab via explorer with monkeypatched st.session_state (or with a minimal wrapper) and simulates a sequence of user answers by pre-populating st.session_state['mp_quiz_votes']. - Asserts that final candidate set matches expectations: either a unique MP (when answers match exactly one MP) or that the function properly identifies indistinguishable MPs (when two MPs have identical votes). Testing details & choices: - Avoid launching Streamlit server; tests only import explorer module and call the builder function in the same way other explorer tests do. Use monkeypatch to stub expensive functions (plotly, query_similar) where required. Verify: `pytest -q tests/test_explorer_quiz_integration.py` Commit message: `test(ui): add integration tests for mp quiz tab flow` Estimated time: 2.0 - 3.0 hours --- ## Verification & CI - Local verification commands (per task) use pytest. Example: - `pytest -q tests/test_match_mps.py` - `pytest -q tests/test_explorer_quiz.py` - `pytest -q tests/test_explorer_quiz_integration.py` - CI expectations: run full test suite. The new tests should be lightweight and use temporary DBs / monkeypatching to avoid depending on large production DB. --- ## Commit & PR Strategy - Work in a feature branch `feat/mp-quiz-2026-03-24`. - Make small focused commits per task (messages suggested above). Each micro-task should be one commit. - PR organization: - PR #1 (Batch 1): database.py changes + tests/test_match_mps.py — target only DB helpers and their unit tests. Keep this PR small so backend logic can be reviewed independently. - PR #2 (Batch 2): explorer.py UI builder + tests/test_explorer_quiz.py — depends on PR #1; rebase after PR #1 merges or open as stacked PR (base=feat/mp-quiz-2026-03-24). - PR #3 (Batch 3): integration+polish tests (tests/test_explorer_quiz_integration.py) and any small fixes discovered during integration testing. - Review checklist for each PR: - Tests covering edge cases (zero-overlap MPs, empty inputs) - DB queries use read_only DuckDB connections - UI uses st.session_state and @st.cache_data appropriately - No production DB writes, no schema changes --- ## Risks & Mitigations (short) - Performance: selecting motions across the entire motions table could be heavy. Mitigation: seed with top-N controversial motions and limit choose_discriminating_motions to motions that have mp_votes rows for the candidate MPs only. - Data quality: MPs with identical votes will remain indistinguishable — surface clearly to user. Tests include that scenario. --- ## Task checklist for implementers (copy/paste friendly) - [ ] Task 1.1: Modify database.py — implement match_mps_for_votes & choose_discriminating_motions. Add tests in tests/test_match_mps.py. (3.0–4.5h) - [ ] Task 1.2: Add this plan file. (0.25–0.5h) - [ ] Task 2.1: Modify explorer.py — add build_mp_quiz_tab and wire into tabs. Add tests in tests/test_explorer_quiz.py. (2.0–4.0h) - [ ] Task 3.1: Add integration test tests/test_explorer_quiz_integration.py to exercise quiz flow. (2.0–3.0h) --- If you run into ambiguous input normalization details or DB edge-cases, follow the choices documented above (explicit normalization mapping, exclude zero-overlap MPs, use entropy scoring). If you encounter a blocker (e.g. missing mp_votes data in test fixtures), create small test fixtures using MotionDatabase.insert_motion and insert_mp_vote in the test setup. Good luck — keep PRs small and tests fast.