docs(design): add fix-missing-trajectories design

thoughts/shared/designs/2026-03-30-fix-missing-trajectories-design.md

@@ -0,0 +1,102 @@
+---
+date: 2026-03-30
+topic: "fix-missing-trajectories"
+status: draft
+---
+
+## Problem Statement
+
+We're seeing empty/absent party trajectories in the Explorer "Partij Trajectories" tab despite compute_2d_axes producing windows and many parties having centroids. The UI shows no visible traces for selected parties in some runs, making the feature unreliable for end users.
+
+**Root hypothesis:** either (A) selected parties have only missing/None centroid values at plot time, (B) a runtime exception (eg float(None)) aborts trace creation silently, or (C) label/party normalization mismatch filters out traces. We need a low-risk, diagnostic-first fix to reveal which of these is happening and restore visible traces quickly.
+
+## Constraints
+
+- Preserve public function names and locations: **compute_2d_axes, classify_axes, load_positions, _build_party_axis_figure, build_trajectories_tab, build_compass_tab, _spline_smooth**.
+- Avoid large refactors; prefer reversible, minimal changes that surface diagnostics.
+- Do not expose internal modal tokens ("As 1"/"As 2") to end users; use axis_classifier.display_label_for_modal(...) or choose_trajectory_title() where appropriate.
+- Visual traces should remain smoothed; hover must include raw centroid values for auditability.
+
+## Chosen Approach (what we'll implement)
+
+I'm choosing a **minimal triage-first approach**: add precise diagnostics and defensive conversions around plotting, so we either restore visible traces immediately or produce deterministic diagnostics that reveal the real data mismatch.
+
+**Why:** low risk, fastest feedback loop. This will either fix simple runtime errors (safe float conversion, exceptions while adding traces) or provide clear evidence that deeper normalization changes are required.
+
+**Key changes:**
+- Add a small helper: **safe_float(x)** — converts numeric-like values to floats, maps None/NaN/invalid -> float('nan') without raising.
+- In build_trajectories_tab/_build_party_axis_figure:
+  - Wrap per-party fig.add_trace(...) in try/except and log the exception with party id/name to the DEBUG expander instead of aborting the whole plot.
+  - Emit per-selected-party diagnostics into the existing DEBUG expander: number of raw centroids, counts of non-NaN coordinates, example first 5 raw xs/ys, and lengths per window.
+  - Replace direct float(...) casts on raw centroid values used in hover/customdata with safe_float.
+  - Ensure per-MP fallback plotting path still exists and can be forced via EXPLORER_FORCE_SHOW_TRAJECTORIES for diagnosis.
+- Add unit tests for safe_float and targeted integration tests that assert traces are created when centroids contain NaNs and when party_map exists.
+
+## Alternatives Considered
+
+1) Full normalization sweep: align party centroids to global windows (fill missing with NaN) and accept parties with at least one non-NaN value.
+   - Pros: robust long-term fix, canonical data shape.
+   - Cons: larger change surface, higher risk, slower to validate in production data.
+
+2) Refactor plotting pipeline to use a normalized DataFrame (rows=windows, cols=parties) and build traces from that canonical shape.
+   - Pros: clearer data flow, easier testing.
+   - Cons: larger refactor, touches many modules.
+
+I considered both but rejected them for immediate work because we need quick deterministic diagnostics to determine if these larger efforts are warranted.
+
+## Architecture (high-level)
+
+**Inputs:** positions_by_window (from compute_2d_axes), party_map, selected_parties.
+
+**Flow:**
+- compute_2d_axes -> positions_by_window
+- load_positions / helpers -> party-centroid dicts keyed by party
+- build_trajectories_tab calls _build_party_axis_figure to build per-party traces
+- _build_party_axis_figure uses smoothing helpers (_spline_smooth) to produce visible traces and also builds hover customdata with raw centroid values (smoothed coords for the trace, raw values in customdata)
+
+**Intervention points:** build_trajectories_tab and _build_party_axis_figure (small helper additions and safe conversion), plus tests and diagnostic output in the DEBUG expander.
+
+## Components and Responsibilities
+
+- **safe_float helper:** convert inputs to float or return float('nan') safely. Centralized to avoid repeated float(None) errors.
+- **Diagnostic emitter:** small utility used by build_trajectories_tab to format and write per-party diagnostic rows to the DEBUG expander.
+- **Plotly trace wrapper:** per-party try/except around fig.add_trace that writes exception details to diagnostics instead of failing silently.
+- **Unit + integration tests:** verify hover customdata creation, safe_float behaviour, trajectories rendered with partial centroids, and UI label mapping does not emit "As 1"/"As 2".
+
+## Data Flow (detailed)
+
+- compute_2d_axes produces windows (time labels) and canonical positions_by_window.
+- load_positions consumes positions_by_window and returns a mapping party -> list of centroids (one per window) where centroids may contain None/NaN for missing windows.
+- build_trajectories_tab selects parties and for each party calls _build_party_axis_figure which:
+  - extracts raw xs_raw, ys_raw arrays aligned to windows
+  - computes smoothed xs_plot, ys_plot via _spline_smooth
+  - builds Plotly trace using xs_plot/ys_plot for the line and includes xs_raw/ys_raw in customdata with safe_float conversion
+  - adds the trace inside a try/except and emits any exception + raw samples to debug
+
+## Error Handling
+
+- Use safe_float to prevent float(None) and similar runtime TypeErrors when building hover/customdata.
+- Use per-party try/except to avoid a single-party failure blanking the whole chart; log the error and continue plotting other parties.
+- Show structured diagnostics in the existing DEBUG expander with these fields: party name, windows_count, raw_centroid_count, non_nan_count, sample_raw_xs, sample_raw_ys, exception (if any).
+
+## Testing Strategy
+
+- Unit tests:
+  - safe_float: None -> nan, '1.23' -> 1.23 (if strings are expected), invalid -> nan
+  - UI label helpers: axis_classifier.display_label_for_modal(...) and choose_trajectory_title() do not return raw "As 1"/"As 2"
+
+- Integration tests (lightweight):
+  - Build a synthetic positions_by_window with some None / NaN holes and assert _build_party_axis_figure returns a Plotly trace object (or equivalent structure) and that customdata contains numeric/NaN values not exceptions.
+  - Test that build_trajectories_tab's DEBUG expander receives the expected diagnostic entries for a party with missing centroids.
+
+- Manual verification steps (later): run full Streamlit with duckdb/plotly installed and open Explorer -> Trajectories to confirm traces are visible for typical parties and inspect the DEBUG expander.
+
+## Open Questions
+
+- Are there other UI locations still exposing raw modal labels? We should sweep the repo and tests already added help with this, but it may not be exhaustive.
+- Do we want safe_float to try to coerce numeric strings? My proposal is **no coercion** (only pass-through numeric types and map others -> nan) unless tests show string encodings exist in centroid data.
+- If diagnostics show that many parties are missing centroids entirely, we'll need the full normalization sweep (alternative #1).
+
+---
+
+I'm proceeding to create the design doc. Interrupt if you want changes.