# Trajectory Plots Not Showing — Debugging Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Identify why trajectory plots are invisible or empty in the Streamlit Explorer UI, then fix the root cause. **Architecture:** Systematic step-by-step pipeline trace from UI → DB. Each stage has explicit "what should I see" checkpoints so we can pinpoint exactly where data becomes invisible. **Tech Stack:** Streamlit, Plotly, DuckDB, Python ≥3.13, uv --- ## Debugging Pipeline (Stage-by-Stage Checkpoints) ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ STAGE 0: UI Layer — what does the user actually see? │ │ explorer.py → build_trajectories_tab() │ │ → Is the tab visible? Empty chart? Error message? No chart at all? │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ STAGE 1: positions_by_window — are MP positions loaded? │ │ load_positions(db_path, "annual") │ │ → Expected: 12 windows, ~150-200 MPs per window │ │ → Check: _last_trajectories_diagnostics["stage"] │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ STAGE 2: party_map — are MP→party mappings loaded? │ │ load_party_map(db_path) │ │ → Expected: ~1036 entries │ │ → Check: party_map is non-empty dict │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ STAGE 3: party centroids — are party means computed? │ │ compute_party_centroids() / compute_party_coords() │ │ → Expected: CDA, D66, VVD, PVV, SP, GroenLinks-PvdA centroids exist │ │ → Check: plottable_parties > 0 │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ STAGE 4: select_trajectory_plot_data — does it return traces? │ │ → Expected: fig with 3-6 colored scatter traces, trace_count > 0 │ │ → Check: banner_text is None (no fallback), trace_count ≥ 3 │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ STAGE 5: Plotly render — is the figure rendered in the browser? │ │ st.plotly_chart(fig, use_container_width=True) │ │ → Expected: visible chart with colored party lines │ │ → Check: browser DOM, no JS errors │ └─────────────────────────────────────────────────────────────────────────┘ ``` --- ## Task 1: Instrument the app to print real-time pipeline state **Files:** - Modify: `explorer.py` (add print statements at each stage) - Test: Run `uv run streamlit run explorer.py` with `EXPLORER_DEBUG_TRAJECTORIES=1` - [ ] **Step 1: Add stage-0 checkpoint at top of `build_trajectories_tab`** Read `explorer.py` lines 1601-1650. Add a print statement at the start of `build_trajectories_tab`: ```python print(f"[TRAJ DEBUG] build_trajectories_tab called — db_path={db_path}, window_size={window_size}") ``` - [ ] **Step 2: Add stage-1 checkpoint after `load_positions`** Read `explorer.py` lines 1605-1610. After the call to `load_positions`, add: ```python positions_by_window, axis_def = load_positions(db_path, window_size) print(f"[TRAJ DEBUG] load_positions → {len(positions_by_window)} windows, " f"total MPs={sum(len(v) for v in positions_by_window.values())}") ``` - [ ] **Step 3: Add stage-2 checkpoint after `load_party_map`** Read `explorer.py` lines 1638-1642. After the call to `load_party_map`, add: ```python party_map = load_party_map(db_path) print(f"[TRAJ DEBUG] load_party_map → {len(party_map)} entries, " f"sample={list(party_map.items())[:3]}") ``` - [ ] **Step 4: Add stage-3 checkpoint after centroid computation** Read `explorer.py` lines 1641-1670. After the inline centroid loop, add: ```python all_parties = sorted(set(party_map.get(mp) for MPs in positions_by_window.values() for mp in MPs) - {None, "Unknown"}) print(f"[TRAJ DEBUG] all_parties (raw from party_map) → {len(all_parties)} parties: {all_parties[:10]}") ``` - [ ] **Step 5: Add stage-4 checkpoint before `st.plotly_chart`** Read `explorer.py` around line 2105. Before the `st.plotly_chart` call, add: ```python print(f"[TRAJ DEBUG] About to render plotly chart — trace_count={trace_count}, " f"banner={banner_text}, fig has {len(fig.data)} traces") ``` - [ ] **Step 6: Run the app and capture all debug output** ```bash EXPLORER_DEBUG_TRAJECTORIES=1 uv run streamlit run explorer.py 2>&1 | grep TRAJ ``` Expected output (all stages should print): ``` [TRAJ DEBUG] build_trajectories_tab called — db_path=..., window_size=annual [TRAJ DEBUG] load_positions → 12 windows, total MPs=... [TRAJ DEBUG] load_party_map → 1036 entries, sample=[(...), (...), (...) ] [TRAJ DEBUG] all_parties (raw from party_map) → N parties: [...] [TRAJ DEBUG] About to render plotly chart — trace_count=N, banner=None, fig has N traces ``` **If any stage is missing or shows 0/empty, that's the bug location. Document which stage fails and proceed to the corresponding fix task.** - [ ] **Step 7: Commit** ```bash git add explorer.py git commit -m "chore: add TRAJ DEBUG print checkpoints to build_trajectories_tab" ``` --- ## Task 2: Fix each failure mode Based on Task 1 output, one of these will be the culprit: ### Failure Mode A: `positions_by_window` is empty (Stage 1) **Symptom:** `load_positions → 0 windows` **Root causes to check:** - `get_uniform_dim_windows` returns `[]` (no dim-50 windows in DB) - `compute_2d_axes` silently fails on all windows - DB path is wrong or `data/motions.db` is missing **Fix:** - [ ] Run: `uv run python -c "from explorer import get_uniform_dim_windows; print(get_uniform_dim_windows('data/motions.db'))"` - [ ] If empty: query DB directly — `uv run duckdb data/motions.db "SELECT COUNT(*) FROM svd_vectors WHERE entity_type='mp'"` and check dimension distribution - [ ] If `compute_2d_axes` fails: add try/except with print at `explorer.py:584` - [ ] If DB path wrong: fix `run_app()` to resolve relative path ### Failure Mode B: `party_map` is empty (Stage 2) **Symptom:** `load_party_map → 0 entries` **Root causes:** - `mp_metadata` and `mp_votes` tables are empty or missing - DuckDB connection fails - DB path points to wrong file **Fix:** - [ ] Run: `uv run python -c "from analysis.visualize import _load_party_map; print(len(_load_party_map('data/motions.db')))"` - [ ] If 0: query `SELECT COUNT(*) FROM mp_metadata`, `SELECT COUNT(*) FROM mp_votes` - [ ] If tables missing: run data pipeline to populate them - [ ] If DuckDB fails to import: check `pip install duckdb` in the uv environment ### Failure Mode C: `all_parties` is empty (Stage 3) **Symptom:** `all_parties (raw from party_map) → 0 parties` **Root causes:** - All MP names in `positions_by_window` have no match in `party_map` (name mismatch) - Every MP maps to `"Unknown"` or `None` **Fix:** - [ ] Run: `uv run python -c "from explorer import load_positions, load_party_map; pw = load_positions('data/motions.db', 'annual')[0]; pm = load_party_map('data/motions.db'); sample_mps = list(pw[list(pw.keys())[0]].keys())[:5]; print({mp: pm.get(mp, 'NO MATCH') for mp in sample_mps})"` - [ ] If name mismatches: investigate `_strip_paren` fallback logic in `compute_party_coords` (explorer_helpers.py:165-170) - [ ] If too many mismatches: add name normalization (strip titles, standardize suffixes) - [ ] Commit fix with test ### Failure Mode D: `trace_count == 0` (Stage 4) **Symptom:** `About to render plotly chart — trace_count=0` or `banner != None` **Root causes:** - All party centroids are NaN (every MP position is NaN) - `compute_party_coords` filters out all parties (NaN/Inf in all positions) - `select_trajectory_plot_data` falls back to MP trajectories but MP fallback also fails **Fix:** - [ ] Add debug print inside `compute_party_coords`: `print(f"[TRAJ DEBUG] compute_party_coords window={window_id} → {len(party_coords)} parties: {list(party_coords.keys())[:5]}")` - [ ] Check if NaN comes from `compute_2d_axes` output (PCA on svd_vectors) - [ ] Run: `uv run python -c "from explorer import load_positions; pw = load_positions('data/motions.db', 'annual')[0]; win = list(pw.values())[0]; sample = list(win.items())[:3]; print({k: v for k, v in sample})"` — if all values are `(nan, nan)`, the PCA step is producing NaN - [ ] If PCA produces NaN: check `analysis/political_axis.py:compute_2d_axes` for the specific window's SVD vectors ### Failure Mode E: Chart not visible in browser (Stage 5) **Symptom:** All stages pass but chart is blank in browser **Root causes:** - Plotly `fig` is empty (no traces added to figure) - Streamlit `st.plotly_chart` suppressed by CSS/JS error - Container width is 0 (layout issue) **Fix:** - [ ] Add debug print: `print(f"[TRAJ DEBUG] st.plotly_chart called with fig.data={[(t.mode, len(t.x), len(t.y)) for t in fig.data]}")` - [ ] Check browser console for JavaScript errors (Plotly.js errors) - [ ] Check if `use_container_width=True` causes issues — try `use_container_width=False` - [ ] Add `st.write(fig)` as alternative to `st.plotly_chart` for debugging ### Failure Mode F: All stages pass, chart still shows blank **Symptom:** `trace_count > 0` but chart looks empty to user **Root causes:** - All traces are transparent/white-on-white - X/Y axes have huge range and all data is in a tiny corner - Party lines overlap completely (all parties at same position) **Fix:** - [ ] Print axis ranges: `print(f"[TRAJ DEBUG] xaxis range={[fig.layout.xaxis.range] if fig.layout.xaxis.range else 'auto'}, yaxis range={[fig.layout.yaxis.range] if fig.layout.yaxis.range else 'auto'}")` - [ ] Check if centroids are all at `(0, 0)` — run: `uv run python -c "from explorer import load_positions, load_party_map; from explorer_helpers import compute_party_coords; ..."` - [ ] Check if PARTY_COLOURS assignment is broken (all traces same color) - [ ] Verify window ordering is correct (chronological left-to-right) --- ## Task 3: Write regression test **Files:** - Create: `tests/test_trajectories_pipeline_integration.py` - [ ] **Step 1: Write integration test** ```python """Integration test: full trajectory pipeline produces non-empty plot.""" from explorer import load_positions, load_party_map from explorer_helpers import compute_party_centroids from explorer import select_trajectory_plot_data def test_trajectory_pipeline_produces_traces(): db_path = "data/motions.db" window_size = "annual" positions_by_window, _ = load_positions(db_path, window_size) party_map = load_party_map(db_path) windows = list(positions_by_window.keys()) centroids, mp_positions = compute_party_centroids(positions_by_window, party_map, windows) fig, trace_count, banner = select_trajectory_plot_data( positions_by_window, party_map, windows, selected_parties=list(centroids.keys())[:6], smooth_alpha=0.35, ) assert trace_count > 0, f"Expected traces but got trace_count={trace_count}, banner={banner}" assert banner is None, f"Expected no fallback banner but got: {banner}" assert len(fig.data) == trace_count ``` - [ ] **Step 2: Run the test** ```bash uv run pytest tests/test_trajectories_pipeline_integration.py -v ``` Expected: PASS - [ ] **Step 3: Commit** ```bash git add tests/test_trajectories_pipeline_integration.py git commit -m "test: add trajectory pipeline integration test" ``` --- ## Execution Order 1. **Task 1 first** — Run the instrumented app and capture which stage fails 2. **Task 2** — Fix the specific failure mode based on Task 1 output 3. **Task 3** — Write regression test once the fix is confirmed **Estimated time:** 15-30 minutes for Task 1 (identifying the stage), 10-30 minutes for Task 2 fix (depends on which mode), 5 minutes for Task 3.