# "Welk tweede kamerlid ben jij?" Implementation Plan

**Goal:** Add a Streamlit quiz tab that interactively asks the user motion (vote) questions and narrows the set of 2026 MPs to find the best-matching MP. Implement two DB helpers (matching + discriminating-motion selection), the UI builder and tab wiring, and tests. Minimal viable changes only — no UX bells & whistles.

**Design:** thoughts/shared/designs/2026-03-24-welk-tweede-kamerlid-ben-jij-design.md

---

## Dependency Graph

```
Batch 1 (parallel): 1.1 [foundation - no deps], 1.2 (plan file) [none]
Batch 2 (parallel): 2.1 [explorer UI - depends: 1.1]
Batch 3 (parallel): 3.1 [integration tests - depends: 1.1,2.1]
```

---

## Summary of implementation decisions (gap-filling)

- MotionDatabase.match_mps_for_votes: implement as a read-only DuckDB-backed method on the existing MotionDatabase class (database.py). It accepts user_votes: Dict[int, str] where keys are motion ids and values are UI vote tokens. I will implement vote normalization inside the method (mapping UI tokens to canonical DB tokens) to avoid touching other modules. Rationale: keeps surface changes minimal and avoids creating new modules.

- MotionDatabase.choose_discriminating_motions: implement in the same file. For a small candidate set (expected << 200 MPs), fetch mp_votes for candidate MPs across candidate motions (excluding already-answered motion ids). Score candidate motions by information-entropy of vote distribution among remaining candidates (higher entropy = better split). Ties broken by controversy_score then motion id.

- Explorer UI changes: add build_mp_quiz_tab(db_path) to explorer.py and wire it into the tabs list and fallback radio. Use st.session_state['mp_quiz_votes'] to store answers as mapping str(motion_id)->UI token. Use @st.cache_data on any expensive DB-calls in the UI layer.

- Vote token normalization: UI will present choices: "Voor", "Tegen", "Onthouden", "Afwezig / Geen stem". The DB stores lowercase tokens like 'voor', 'tegen', 'onthouden', 'afwezig'. match_mps_for_votes will normalize case and a small set of variants (e.g., 'Geen stem' -> 'afwezig', 'Abstain' -> 'onthouden') — explicit list included in tests.

---

## BATCH 1: Foundation (parallel - N implementers)
All tasks in this batch have NO dependencies and can run simultaneously.

### Task 1.1: Add DB helpers to MotionDatabase
**File:** `database.py` (modify existing)
**Test:** `tests/test_match_mps.py`
**Depends:** none

Description / Acceptance criteria:
- Add two new public methods to MotionDatabase:
  - match_mps_for_votes(user_votes: Dict[int, str], limit: int = 50) -> List[Dict]
    - Returns an ordered list (desc by agreement_pct) of dicts with keys: mp_name, party, matched (int), overlap (int), agreement_pct (float 0-100).
    - Behavior: for each mp present in mp_votes for any of the provided motions compute:
      - overlap = number of motions where MP has a recorded vote AND the user provided a non-empty vote (i.e., not "Geen stem").
      - matched = number of those overlaps where normalized(mp_vote) == normalized(user_vote).
      - agreement_pct = matched / overlap * 100 rounded to 1 decimal. MPs with overlap==0 are excluded from the returned list.
    - Ordering: agreement_pct desc, then matched desc, then mp_name asc.

  - choose_discriminating_motions(candidates: List[str], excluded_motion_ids: List[int], k: int = 1) -> List[int]
    - For the provided candidate mp_names, compute vote distributions per motion (voor/tegen/onthouden/afwezig) excluding motion ids in excluded_motion_ids.
    - Score each motion by Shannon entropy of the distribution among the candidate MPs (treating 'afwezig' as a separate bucket). Higher entropy preferred.
    - Return top-k motion ids as a list, tiebreakers: higher controversy_score (motions table) then lower motion id.

Implementation notes & decisions:
- Implement normalization inside these methods. Normalization mapping (DB vote -> canonical): map DB votes lowercased to one of {'voor','tegen','onthouden','afwezig'}. UI inputs (Voor/Tegen/Onthouden/Geen stem) normalized to these same tokens.
- For performance, implement SQL queries that select mp_votes filtered by motion_id IN (...) and mp_name IN (candidates) and aggregate via GROUP BY mp_name and vote. For small candidate sets and a limited set of motion_ids this will be fast. If duckdb is not available, fall back to in-Python aggregates using the file-backed JSON format already present in MotionDatabase._init_database.
- Add docstrings and basic parameter validation (raise ValueError for empty user_votes or empty candidates input). Tests will cover expected exceptions.

Test outline (tests/test_match_mps.py):
- Setup: create a temporary MotionDatabase using a temp db_path (MotionDatabase.reset_database() can be used if duckdb available; otherwise use file-backed mode). Insert a small set of motions and mp_votes via insert_motion / insert_mp_vote. Create at least 3 MPs with overlapping but distinct vote patterns across 4-6 motions.
- Tests:
  1) test_match_basic_counts: user_votes covering 3 motions returns expected matched/overlap/agreement_pct per MP.
  2) test_match_excludes_zero_overlap: MPs with no recorded votes for provided motions are excluded.
  3) test_choose_discriminating_motions_entropy_ranking: with a small candidate set, the chosen motion(s) split candidates as expected (assert returned motion id is one of known good splitters)
  4) test_invalid_input: calling match_mps_for_votes with empty user_votes raises ValueError.

Verify: `pytest -q tests/test_match_mps.py`

Commit message: `feat(database): add match_mps_for_votes and choose_discriminating_motions`

Estimated time: 3.0 - 4.5 hours

---

### Task 1.2: Add plan file (this document)
**File:** `thoughts/shared/plans/2026-03-24-welk-tweede-kamerlid-ben-jij-plan.md` (this file)
**Test:** none
**Depends:** none

Description: Add the implementation plan (this document) to the repo to provide step-by-step microtasks to implementers. No tests.

Verify: visually review file in repo. No test run.

Commit message: `docs(plans): add plan for 'Welk tweede kamerlid ben jij?'`

Estimated time: 0.25 - 0.5 hours

---

## BATCH 2: Core UI (parallel - depends on Batch 1)
All tasks in this batch assume the DB methods from Task 1.1 exist.

### Task 2.1: Add Streamlit quiz tab & wiring
**File:** `explorer.py` (modify existing)
**Test:** `tests/test_explorer_quiz.py`
**Depends:** 1.1

Description / Acceptance criteria:
- Add a function `build_mp_quiz_tab(db_path: str) -> None` placed near other build_*_tab functions (as described in the design, e.g., after build_svd_components_tab or near the top of the tab builders). The function must:
  - Render a short intro/instructions.
  - Load an initial pool of candidate motions using existing `load_motions_df(db_path)` and pick a seed set (top N by controversy_score). Decision: seed N = 8 (configurable constant in the function: SEED_MOTIONS = 8) — this is small and fast.
  - Present questions one at a time: show motion title + layman_explanation (if available) and a radio with choices: "Voor", "Tegen", "Onthouden", "Geen stem" and a "Skip"/"Niet zeker" optional button mapped to "Geen stem". Choice stored to `st.session_state['mp_quiz_votes']` as mapping with keys str(motion_id) -> UI token.
  - After each answer, call MotionDatabase.match_mps_for_votes(user_votes) to fetch ranked candidates and display a small DataFrame (top 10) with columns: MP name, party, matched, overlap, agreement_pct. Use st.dataframe.
  - If more than 1 candidate remains with top agreement_pct tied, call MotionDatabase.choose_discriminating_motions(candidates, excluded_motion_ids) to pick the next question to ask and continue until one unique MP remains or choose_discriminating_motions returns an empty list (tie / indistinguishable). Cap total questions at 20 (SESS_CAP = 20).
  - When unique MP is found (agreement_pct == 100 and overlap>0 and only one MP with highest agreement), show final MP summary (name, party) and their matching motions count.
  - Use caching: wrap any repeated DB lookups (e.g., load_motions_df already cached) and mark heavy updates via @st.cache_data where appropriate.

Implementation notes & decisions:
- Keep all UI state local to st.session_state with keys prefixed `mp_quiz_` to avoid collisions.
- Normalize UI tokens before sending to DB helper (but DB methods will also normalize; duplication is defensive).
- Keep the UI function self-contained in explorer.py (do not create new modules for this minimal MVP).

Test outline (tests/test_explorer_quiz.py):
- Use monkeypatching to inject a MotionDatabase mock into explorer module or run in a test DB using MotionDatabase with temp db_path. The test must be import-safe (explorer.py imports many heavy libs), so follow pattern used by existing tests/test_explorer_import.py: import the module and assert `build_mp_quiz_tab` exists and is callable.
- Functional assertions:
  1) test_builder_exists: import explorer, assert callable(build_mp_quiz_tab)
  2) test_ui_state_update_simulation: simulate st.session_state by creating a fake session dict (use monkeypatch to set st.session_state to a dict-like object) and calling build_mp_quiz_tab with a small temp DB where motions and mp_votes are prepared. Assert that after calling the builder with pre-filled votes the DataFrame block would display ranked candidates (test inspects returned structure if builder returns it, or else monkeypatch MotionDatabase.match_mps_for_votes to verify it was called with expected mapping).

Verification: `pytest -q tests/test_explorer_quiz.py`

Commit message: `feat(ui): add 'Welk tweede kamerlid ben jij?' tab and wiring in explorer.py`

Estimated time: 2.0 - 4.0 hours

---

## BATCH 3: Integration & Tests (parallel - depends on Batches 1+2)

### Task 3.1: Add integration test for quiz flow
**File:** `tests/test_explorer_quiz_integration.py`
**Test:** this file
**Depends:** 1.1, 2.1

Description / Acceptance criteria:
- Create an end-to-end-ish headless test that:
  - Sets up a temporary MotionDatabase instance (temp file path) and inserts a small controlled dataset: ~6 motions, 4 MPs with distinct votes.
  - Calls build_mp_quiz_tab via explorer with monkeypatched st.session_state (or with a minimal wrapper) and simulates a sequence of user answers by pre-populating st.session_state['mp_quiz_votes'].
  - Asserts that final candidate set matches expectations: either a unique MP (when answers match exactly one MP) or that the function properly identifies indistinguishable MPs (when two MPs have identical votes).

Testing details & choices:
- Avoid launching Streamlit server; tests only import explorer module and call the builder function in the same way other explorer tests do. Use monkeypatch to stub expensive functions (plotly, query_similar) where required.

Verify: `pytest -q tests/test_explorer_quiz_integration.py`

Commit message: `test(ui): add integration tests for mp quiz tab flow`

Estimated time: 2.0 - 3.0 hours

---

## Verification & CI

- Local verification commands (per task) use pytest. Example:
  - `pytest -q tests/test_match_mps.py`
  - `pytest -q tests/test_explorer_quiz.py`
  - `pytest -q tests/test_explorer_quiz_integration.py`

- CI expectations: run full test suite. The new tests should be lightweight and use temporary DBs / monkeypatching to avoid depending on large production DB.

---

## Commit & PR Strategy

- Work in a feature branch `feat/mp-quiz-2026-03-24`.
- Make small focused commits per task (messages suggested above). Each micro-task should be one commit.
- PR organization:
  - PR #1 (Batch 1): database.py changes + tests/test_match_mps.py — target only DB helpers and their unit tests. Keep this PR small so backend logic can be reviewed independently.
  - PR #2 (Batch 2): explorer.py UI builder + tests/test_explorer_quiz.py — depends on PR #1; rebase after PR #1 merges or open as stacked PR (base=feat/mp-quiz-2026-03-24).
  - PR #3 (Batch 3): integration+polish tests (tests/test_explorer_quiz_integration.py) and any small fixes discovered during integration testing.

- Review checklist for each PR:
  - Tests covering edge cases (zero-overlap MPs, empty inputs)
  - DB queries use read_only DuckDB connections
  - UI uses st.session_state and @st.cache_data appropriately
  - No production DB writes, no schema changes

---

## Risks & Mitigations (short)

- Performance: selecting motions across the entire motions table could be heavy. Mitigation: seed with top-N controversial motions and limit choose_discriminating_motions to motions that have mp_votes rows for the candidate MPs only.
- Data quality: MPs with identical votes will remain indistinguishable — surface clearly to user. Tests include that scenario.

---

## Task checklist for implementers (copy/paste friendly)

- [ ] Task 1.1: Modify database.py — implement match_mps_for_votes & choose_discriminating_motions. Add tests in tests/test_match_mps.py. (3.0–4.5h)
- [ ] Task 1.2: Add this plan file. (0.25–0.5h)
- [ ] Task 2.1: Modify explorer.py — add build_mp_quiz_tab and wire into tabs. Add tests in tests/test_explorer_quiz.py. (2.0–4.0h)
- [ ] Task 3.1: Add integration test tests/test_explorer_quiz_integration.py to exercise quiz flow. (2.0–3.0h)

---

If you run into ambiguous input normalization details or DB edge-cases, follow the choices documented above (explicit normalization mapping, exclude zero-overlap MPs, use entropy scoring). If you encounter a blocker (e.g. missing mp_votes data in test fixtures), create small test fixtures using MotionDatabase.insert_motion and insert_mp_vote in the test setup.

Good luck — keep PRs small and tests fast. 
