You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
79 lines
4.1 KiB
79 lines
4.1 KiB
---
|
|
title: "Trajectories diagnostic script produced false alarm due to mocked empty data"
|
|
date: 2026-03-31
|
|
module: explorer
|
|
problem_type: workflow_issue
|
|
component: explorer
|
|
severity: medium
|
|
symptoms:
|
|
- "Diagnostic JSON showed party_map_count: 0 for all scenarios"
|
|
- "Trajectories appeared broken based on diagnostic output"
|
|
- "Mindmodel anti-pattern flagged compute_party_coords party_map mismatch"
|
|
root_cause: logic_error
|
|
resolution_type: process_fix
|
|
tags:
|
|
- trajectories
|
|
- diagnostics
|
|
- false-alarm
|
|
- mocking
|
|
- testing
|
|
---
|
|
|
|
# Trajectories Diagnostic Script Produced False Alarm
|
|
|
|
## Problem
|
|
|
|
The `scripts/diagnose_trajectories_cli.py` diagnostic script reported `party_map_count: 0` across all scenarios, suggesting that party trajectories in the Explorer were broken. This triggered an investigation, a mindmodel anti-pattern flag, and multiple design docs for fixing "missing trajectories."
|
|
|
|
## Symptoms
|
|
|
|
- `thoughts/shared/diagnostics/2026-03-31-trajectories-diagnostics.json` showed `party_map_count: 0` in every scenario
|
|
- The mindmodel generation process flagged `explorer_helpers.py:compute_party_coords` as a top anti-pattern (party_map key/value mismatch hypothesis)
|
|
- Multiple implementation plans were drafted to add fallback rendering, MP-level trajectories, and debug instrumentation
|
|
|
|
## Root Cause
|
|
|
|
The diagnostic script itself was the bug. It artificially passed `load_party_map_ret={}` (empty dict) in **all** scenarios, creating a false alarm that had no relation to production behavior.
|
|
|
|
When tested with real data:
|
|
- `party_map` has **1,036 entries** (not 0)
|
|
- `select_trajectory_plot_data` returns `trace_count=6` with real data
|
|
- Annual view shows CDA, D66, VVD traces; quarterly view shows 6 party traces
|
|
- Trajectories work correctly — no production bug exists
|
|
|
|
The anti-pattern detected (`compute_party_coords` party_map mismatch) was also a false alarm: `svd_vectors` entity_ids are ALL MP names, never party names (no `entity_type='party'` rows exist in the database).
|
|
|
|
## What Didn't Work
|
|
|
|
- Trusting the diagnostic script output without validating against real data
|
|
- Investigating based on the JSON artifact alone
|
|
- The `2026-03-31-trajectories-diagnostics.json` was created by a script that passes `load_party_map_ret={}` artificially
|
|
|
|
## Solution
|
|
|
|
**No production code changes needed** — trajectories work correctly. The fix is to the diagnostic script and the process:
|
|
|
|
1. **Fix `scripts/diagnose_trajectories_cli.py`** to use real data paths (`data/motions.db`) and real `load_party_map` / `load_positions` calls instead of mocking everything to empty
|
|
2. **Re-run the fixed diagnostic script** to produce a correct `trajectories-diagnostics.json` artifact
|
|
3. **Remove the incorrect anti-pattern** from the mindmodel manifest (the party_map mismatch hypothesis does not apply since no party-level entity_ids exist in `svd_vectors`)
|
|
|
|
## Prevention
|
|
|
|
1. **Never trust a diagnostic that mocks data to empty without a real-data validation step**
|
|
2. **Always compare diagnostic output against a known-good baseline** — run the same checks with real data before concluding there's a bug
|
|
3. **Diagnostic scripts should use real data paths by default** — if mocking is needed for unit tests, keep it in tests, not in diagnostic CLI scripts
|
|
4. **Verify DB state directly** before investigating based on intermediary artifacts:
|
|
```python
|
|
# Quick sanity check
|
|
from explorer import load_positions, load_party_map
|
|
positions_by_window, _ = load_positions("data/motions.db", "annual")
|
|
party_map = load_party_map("data/motions.db")
|
|
assert len(party_map) > 0, "party_map is empty — investigate data pipeline"
|
|
```
|
|
5. **Add an integration test** that calls `select_trajectory_plot_data` with real DB data and asserts `trace_count > 0`
|
|
|
|
## Related
|
|
|
|
- `docs/solutions/best-practices/refactoring-streamlit-data-loading.md` — Data loading patterns for explorer
|
|
- `thoughts/shared/plans/2026-03-31-debug-trajectories-not-showing.md` — Original debugging plan (based on false alarm)
|
|
- `thoughts/shared/designs/2026-03-31-diagnose-no-plot-trajectories-design.md` — Original design doc (based on false alarm)
|
|
|