You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
102 lines
7.2 KiB
102 lines
7.2 KiB
---
|
|
date: 2026-03-30
|
|
topic: "fix-missing-trajectories"
|
|
status: draft
|
|
---
|
|
|
|
## Problem Statement
|
|
|
|
We're seeing empty/absent party trajectories in the Explorer "Partij Trajectories" tab despite compute_2d_axes producing windows and many parties having centroids. The UI shows no visible traces for selected parties in some runs, making the feature unreliable for end users.
|
|
|
|
**Root hypothesis:** either (A) selected parties have only missing/None centroid values at plot time, (B) a runtime exception (eg float(None)) aborts trace creation silently, or (C) label/party normalization mismatch filters out traces. We need a low-risk, diagnostic-first fix to reveal which of these is happening and restore visible traces quickly.
|
|
|
|
## Constraints
|
|
|
|
- Preserve public function names and locations: **compute_2d_axes, classify_axes, load_positions, _build_party_axis_figure, build_trajectories_tab, build_compass_tab, _spline_smooth**.
|
|
- Avoid large refactors; prefer reversible, minimal changes that surface diagnostics.
|
|
- Do not expose internal modal tokens ("As 1"/"As 2") to end users; use axis_classifier.display_label_for_modal(...) or choose_trajectory_title() where appropriate.
|
|
- Visual traces should remain smoothed; hover must include raw centroid values for auditability.
|
|
|
|
## Chosen Approach (what we'll implement)
|
|
|
|
I'm choosing a **minimal triage-first approach**: add precise diagnostics and defensive conversions around plotting, so we either restore visible traces immediately or produce deterministic diagnostics that reveal the real data mismatch.
|
|
|
|
**Why:** low risk, fastest feedback loop. This will either fix simple runtime errors (safe float conversion, exceptions while adding traces) or provide clear evidence that deeper normalization changes are required.
|
|
|
|
**Key changes:**
|
|
- Add a small helper: **safe_float(x)** — converts numeric-like values to floats, maps None/NaN/invalid -> float('nan') without raising.
|
|
- In build_trajectories_tab/_build_party_axis_figure:
|
|
- Wrap per-party fig.add_trace(...) in try/except and log the exception with party id/name to the DEBUG expander instead of aborting the whole plot.
|
|
- Emit per-selected-party diagnostics into the existing DEBUG expander: number of raw centroids, counts of non-NaN coordinates, example first 5 raw xs/ys, and lengths per window.
|
|
- Replace direct float(...) casts on raw centroid values used in hover/customdata with safe_float.
|
|
- Ensure per-MP fallback plotting path still exists and can be forced via EXPLORER_FORCE_SHOW_TRAJECTORIES for diagnosis.
|
|
- Add unit tests for safe_float and targeted integration tests that assert traces are created when centroids contain NaNs and when party_map exists.
|
|
|
|
## Alternatives Considered
|
|
|
|
1) Full normalization sweep: align party centroids to global windows (fill missing with NaN) and accept parties with at least one non-NaN value.
|
|
- Pros: robust long-term fix, canonical data shape.
|
|
- Cons: larger change surface, higher risk, slower to validate in production data.
|
|
|
|
2) Refactor plotting pipeline to use a normalized DataFrame (rows=windows, cols=parties) and build traces from that canonical shape.
|
|
- Pros: clearer data flow, easier testing.
|
|
- Cons: larger refactor, touches many modules.
|
|
|
|
I considered both but rejected them for immediate work because we need quick deterministic diagnostics to determine if these larger efforts are warranted.
|
|
|
|
## Architecture (high-level)
|
|
|
|
**Inputs:** positions_by_window (from compute_2d_axes), party_map, selected_parties.
|
|
|
|
**Flow:**
|
|
- compute_2d_axes -> positions_by_window
|
|
- load_positions / helpers -> party-centroid dicts keyed by party
|
|
- build_trajectories_tab calls _build_party_axis_figure to build per-party traces
|
|
- _build_party_axis_figure uses smoothing helpers (_spline_smooth) to produce visible traces and also builds hover customdata with raw centroid values (smoothed coords for the trace, raw values in customdata)
|
|
|
|
**Intervention points:** build_trajectories_tab and _build_party_axis_figure (small helper additions and safe conversion), plus tests and diagnostic output in the DEBUG expander.
|
|
|
|
## Components and Responsibilities
|
|
|
|
- **safe_float helper:** convert inputs to float or return float('nan') safely. Centralized to avoid repeated float(None) errors.
|
|
- **Diagnostic emitter:** small utility used by build_trajectories_tab to format and write per-party diagnostic rows to the DEBUG expander.
|
|
- **Plotly trace wrapper:** per-party try/except around fig.add_trace that writes exception details to diagnostics instead of failing silently.
|
|
- **Unit + integration tests:** verify hover customdata creation, safe_float behaviour, trajectories rendered with partial centroids, and UI label mapping does not emit "As 1"/"As 2".
|
|
|
|
## Data Flow (detailed)
|
|
|
|
- compute_2d_axes produces windows (time labels) and canonical positions_by_window.
|
|
- load_positions consumes positions_by_window and returns a mapping party -> list of centroids (one per window) where centroids may contain None/NaN for missing windows.
|
|
- build_trajectories_tab selects parties and for each party calls _build_party_axis_figure which:
|
|
- extracts raw xs_raw, ys_raw arrays aligned to windows
|
|
- computes smoothed xs_plot, ys_plot via _spline_smooth
|
|
- builds Plotly trace using xs_plot/ys_plot for the line and includes xs_raw/ys_raw in customdata with safe_float conversion
|
|
- adds the trace inside a try/except and emits any exception + raw samples to debug
|
|
|
|
## Error Handling
|
|
|
|
- Use safe_float to prevent float(None) and similar runtime TypeErrors when building hover/customdata.
|
|
- Use per-party try/except to avoid a single-party failure blanking the whole chart; log the error and continue plotting other parties.
|
|
- Show structured diagnostics in the existing DEBUG expander with these fields: party name, windows_count, raw_centroid_count, non_nan_count, sample_raw_xs, sample_raw_ys, exception (if any).
|
|
|
|
## Testing Strategy
|
|
|
|
- Unit tests:
|
|
- safe_float: None -> nan, '1.23' -> 1.23 (if strings are expected), invalid -> nan
|
|
- UI label helpers: axis_classifier.display_label_for_modal(...) and choose_trajectory_title() do not return raw "As 1"/"As 2"
|
|
|
|
- Integration tests (lightweight):
|
|
- Build a synthetic positions_by_window with some None / NaN holes and assert _build_party_axis_figure returns a Plotly trace object (or equivalent structure) and that customdata contains numeric/NaN values not exceptions.
|
|
- Test that build_trajectories_tab's DEBUG expander receives the expected diagnostic entries for a party with missing centroids.
|
|
|
|
- Manual verification steps (later): run full Streamlit with duckdb/plotly installed and open Explorer -> Trajectories to confirm traces are visible for typical parties and inspect the DEBUG expander.
|
|
|
|
## Open Questions
|
|
|
|
- Are there other UI locations still exposing raw modal labels? We should sweep the repo and tests already added help with this, but it may not be exhaustive.
|
|
- Do we want safe_float to try to coerce numeric strings? My proposal is **no coercion** (only pass-through numeric types and map others -> nan) unless tests show string encodings exist in centroid data.
|
|
- If diagnostics show that many parties are missing centroids entirely, we'll need the full normalization sweep (alternative #1).
|
|
|
|
---
|
|
|
|
I'm proceeding to create the design doc. Interrupt if you want changes.
|
|
|