feat: agent-native refactor, SVD consistency fixes, UX cleanup, mobile support
- Refactor agent_tools to atomic primitives (24 tools, delete workflows) - Fix SVD component score inconsistency between single-window and trajectory views (same PCA basis, same flip handling, same active-MP filter for current_parliament) - Fix Dutch spelling: Huidig parliament -> Huidig parlement - Remove all decorative emojis from UI (app.py, explorer.py, analysis tabs) - Add dark theme matching sgeboers.nl (mint accent on dark background) - Remove browser tab favicon and Streamlit chrome (deploy button, running status) - Remove trajectories debug UI and EMA settings (hardcoded smooth_alpha=0.35) - Switch layout to centered for mobile readability - Add responsive CSS for mobile (touch targets, font sizing, overflow prevention) - Update AGENTS.md and SYSTEM_PROMPT.md with active tool instructions - Add compound docs for SVD consistency bug - Update tests: 214 passed, 3 skipped
This commit is contained in:
+147
@@ -0,0 +1,147 @@
|
||||
---
|
||||
title: SVD component scores inconsistent between single-window and trajectory views
|
||||
date: "2026-05-04"
|
||||
category: logic-errors
|
||||
module: analysis
|
||||
problem_type: logic_error
|
||||
component: service_object
|
||||
severity: high
|
||||
symptoms:
|
||||
- "Party position numbers differ between Enkel venster and Tijdtraject views for the same SVD component and window"
|
||||
- "Displayed values have opposite signs for flipped components even when underlying data is identical"
|
||||
root_cause: logic_error
|
||||
resolution_type: code_fix
|
||||
tags:
|
||||
- svd
|
||||
- pca
|
||||
- alignment
|
||||
- visualization
|
||||
- data-consistency
|
||||
---
|
||||
|
||||
# SVD component scores inconsistent between single-window and trajectory views
|
||||
|
||||
## Problem
|
||||
|
||||
In the parliamentary explorer's "SVD Components" tab, party position numbers differed between the "Enkel venster" (single window) view and the "Tijdtraject" (time trajectory) view for the SAME component and SAME window. Users comparing a specific year across the two views saw inconsistent numerical scores.
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Selecting component 2, window "2023" in single-window shows a party at +0.42, but the trajectory view at the same point shows that party at a different value (e.g. -0.15)
|
||||
- Signs invert for certain components when `theme["flip"]` is `True`
|
||||
- The mismatch occurs even though both views claim to show the same underlying SVD component
|
||||
- **After initial fixes:** most years aligned, but "Huidig parlement" still showed different values between the two views
|
||||
- "Huidig parlement" was misspelled as "Huidig parliament" in the window selector label
|
||||
|
||||
## What Didn't Work
|
||||
|
||||
Initial suspicion that the difference came from Procrustes alignment or data caching issues. Checking whether `load_party_scores_all_windows_aligned()` vs `load_party_scores_all_windows()` was the culprit. However, both views were already using the same alignment path.
|
||||
|
||||
The real cause was subtler: the trajectory view was computing PCA over a different set of windows than the single-window view, and then ignoring the flip flag entirely.
|
||||
|
||||
## Solution
|
||||
|
||||
### Fix 1: Align trajectory PCA computation with single-window computation
|
||||
|
||||
In `analysis/explorer_data.py`, function `_get_aligned_trajectory_scores()`:
|
||||
|
||||
```python
|
||||
def _get_aligned_trajectory_scores(
|
||||
db_path: str, windows: List[str], n_components: int = 10
|
||||
) -> Dict[str, Dict[str, List[float]]]:
|
||||
from analysis.political_axis import compute_nd_axes
|
||||
|
||||
all_uniform_windows = get_uniform_dim_windows(db_path)
|
||||
scores_by_window, _ = compute_nd_axes(
|
||||
db_path, window_ids=all_uniform_windows, n_components=n_components
|
||||
)
|
||||
# ... rest filters to requested windows
|
||||
```
|
||||
|
||||
**Change:** Compute PCA on **all** uniform-dim windows (matching `get_aligned_party_scores`), then filter to the requested windows. Previously, `_get_aligned_trajectory_scores()` passed only a subset of windows (excluding `_current_year`) to `compute_nd_axes()`, which produced different principal components, global mean, and flip signs.
|
||||
|
||||
### Fix 2: Apply theme flip in trajectory rendering
|
||||
|
||||
In `analysis/tabs/_rendering.py`, function `_render_svd_time_trajectory()`:
|
||||
|
||||
```python
|
||||
idx = comp_sel - 1
|
||||
flip = theme.get("flip", False)
|
||||
# ...
|
||||
for window in sorted_windows:
|
||||
scores_by_party = party_scores_by_window.get(window, {})
|
||||
for party in selected_parties:
|
||||
scores = scores_by_party.get(party, [])
|
||||
if scores and len(scores) > idx:
|
||||
try:
|
||||
score = float(scores[idx])
|
||||
if flip:
|
||||
score = -score
|
||||
party_trajectories.setdefault(party, []).append((window, score))
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
```
|
||||
|
||||
**Change:** Added flip application to negate scores when `theme.get("flip", False)` is `True`. `_render_party_axis_chart_1d()` already did this, but `_render_svd_time_trajectory()` completely ignored the flip flag.
|
||||
|
||||
### Fix 3: Filter current_parliament to active MPs in trajectory view
|
||||
|
||||
In `analysis/explorer_data.py`, function `_get_aligned_trajectory_scores()`:
|
||||
|
||||
```python
|
||||
party_map = load_party_map(db_path)
|
||||
active_mps = load_active_mps(db_path)
|
||||
|
||||
result: Dict[str, Dict[str, List[float]]] = {}
|
||||
for window in windows:
|
||||
window_scores = scores_by_window.get(window, {})
|
||||
if not window_scores:
|
||||
continue
|
||||
|
||||
# For current_parliament, match single-window view by filtering to
|
||||
# only MPs who are still seated (active). Historical windows include
|
||||
# all MPs present in that window.
|
||||
if window == "current_parliament":
|
||||
window_scores = {
|
||||
mp: sc for mp, sc in window_scores.items() if mp in active_mps
|
||||
}
|
||||
|
||||
party_vecs: Dict[str, List[np.ndarray]] = {}
|
||||
# ... aggregate by party as before
|
||||
```
|
||||
|
||||
**Change:** Added `active_mps = load_active_mps(db_path)` and filtered `window_scores` to only active MPs when `window == "current_parliament"`. The single-window view (`get_aligned_party_scores()`) already did this filtering, but the trajectory view averaged ALL MPs (including those who had left parliament), producing different party means.
|
||||
|
||||
### Fix 4: Correct Dutch spelling of window label
|
||||
|
||||
In `analysis/tabs/components.py`:
|
||||
|
||||
```python
|
||||
def _svd_window_label(w: str) -> str:
|
||||
if w == "current_parliament":
|
||||
return "Huidig parlement" # was "Huidig parliament"
|
||||
return w
|
||||
```
|
||||
|
||||
**Change:** Fixed misspelling of Dutch word "parlement" (was "parliament").
|
||||
|
||||
## Why This Works
|
||||
|
||||
1. **Same PCA basis**: `compute_nd_axes()` computes global PCA across all provided windows. When the single-window view used all uniform-dim windows and the trajectory view used a subset, the resulting components, mean centering, and variance explained were different. Passing the same `window_ids` to `compute_nd_axes()` guarantees identical PCA bases.
|
||||
|
||||
2. **Same flip handling**: The single-window view negates scores when `flip=True`. The trajectory view now does the same, ensuring both views display numerically identical values for the same (window, component, party) tuple.
|
||||
|
||||
3. **Same MP population for current_parliament**: The single-window view filtered `current_parliament` to only active (still-seated) MPs before computing party means. The trajectory view now applies the same filter, so party averages are computed over the identical set of MPs.
|
||||
|
||||
## Prevention
|
||||
|
||||
- When multiple views display the same underlying SVD/PCA data, ensure they all call `compute_nd_axes()` with the **identical** set of window IDs.
|
||||
- Never apply visual transformations (like `theme["flip"]` or `active_mps` filtering) in one view but omit them in another — keep rendering logic symmetric across all views for the same data.
|
||||
- Add a regression test that asserts `get_aligned_party_scores(window, comp)` equals `_get_aligned_trajectory_scores([window], comp)[window]` for sampled windows and components, including `current_parliament`.
|
||||
- Document that `compute_nd_axes()` is a global operation over its input windows; any subset produces a different coordinate frame.
|
||||
- When special-casing `current_parliament` (e.g. active-MP filtering), apply the same logic in every code path that processes that window — single-window, trajectory, compass, and exports.
|
||||
|
||||
## Related Issues
|
||||
|
||||
- `docs/solutions/ui-bugs/svd-axis-pole-labels-incorrect-after-flip.md` — related flip-handling bug in the same SVD Components tab
|
||||
- `docs/solutions/ui-bugs/svd-compass-components-position-inconsistency.md` — related alignment inconsistency between compass and components tab
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
title: "SVD time trajectory shows different scores than single-window view for same component and window"
|
||||
date: 2026-05-04
|
||||
module: analysis
|
||||
problem_type: ui_bug
|
||||
component: analysis
|
||||
symptoms:
|
||||
- "Party position numbers in Tijdtraject view differ from Enkel venster view for the exact same component and window"
|
||||
- "Same party has opposite sign in trajectory vs single-window when theme flip is active"
|
||||
- "Inconsistent numerical values break user trust in the SVD Components tab"
|
||||
root_cause: logic_error
|
||||
resolution_type: code_fix
|
||||
severity: high
|
||||
tags:
|
||||
- svd
|
||||
- pca
|
||||
- time-trajectory
|
||||
- parliamentary-explorer
|
||||
- alignment
|
||||
- flip
|
||||
---
|
||||
|
||||
# SVD Time Trajectory Shows Different Scores Than Single-Window View
|
||||
|
||||
## Problem
|
||||
|
||||
In the parliamentary explorer's "SVD Components" tab, party position numbers differed between the "Enkel venster" (single window) view and the "Tijdtraject" (time trajectory) view for the SAME component and SAME window. Users comparing a specific year across the two views saw inconsistent numerical scores.
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Selecting component 2, window "2023-2024" in single-window shows PVV at +0.42, but the trajectory view at the same point shows PVV at -0.15
|
||||
- Signs invert for certain components when `theme["flip"]` is `True`
|
||||
- The mismatch occurs even though both views claim to show the same underlying SVD component
|
||||
|
||||
## What Didn't Work
|
||||
|
||||
Initial suspicion that the difference came from Procrustes alignment or data caching issues. Checking whether `load_party_scores_all_windows_aligned()` vs `load_party_scores_all_windows()` was the culprit. However, both views were already using the same alignment path.
|
||||
|
||||
The real cause was subtler: the trajectory view was computing PCA over a different set of windows than the single-window view, and then ignoring the flip flag entirely.
|
||||
|
||||
## Solution
|
||||
|
||||
### Fix 1: Align trajectory PCA computation with single-window computation
|
||||
|
||||
In `analysis/explorer_data.py`, function `_get_aligned_trajectory_scores()`:
|
||||
|
||||
```python
|
||||
def _get_aligned_trajectory_scores(
|
||||
db_path: str, windows: List[str], n_components: int = 10
|
||||
) -> Dict[str, Dict[str, List[float]]]:
|
||||
from analysis.political_axis import compute_nd_axes
|
||||
|
||||
all_uniform_windows = get_uniform_dim_windows(db_path)
|
||||
scores_by_window, _ = compute_nd_axes(
|
||||
db_path, window_ids=all_uniform_windows, n_components=n_components
|
||||
)
|
||||
# ... rest filters to requested windows
|
||||
```
|
||||
|
||||
**Change:** Compute PCA on **all** uniform-dim windows (matching `get_aligned_party_scores`), then filter to the requested windows. Previously, `_get_aligned_trajectory_scores()` passed only a subset of windows (excluding `_current_year`) to `compute_nd_axes()`, which produced different principal components, global mean, and flip signs.
|
||||
|
||||
### Fix 2: Apply theme flip in trajectory rendering
|
||||
|
||||
In `analysis/tabs/_rendering.py`, function `_render_svd_time_trajectory()`:
|
||||
|
||||
```python
|
||||
idx = comp_sel - 1
|
||||
flip = theme.get("flip", False)
|
||||
# ...
|
||||
for window in sorted_windows:
|
||||
scores_by_party = party_scores_by_window.get(window, {})
|
||||
for party in selected_parties:
|
||||
scores = scores_by_party.get(party, [])
|
||||
if scores and len(scores) > idx:
|
||||
try:
|
||||
score = float(scores[idx])
|
||||
if flip:
|
||||
score = -score
|
||||
party_trajectories.setdefault(party, []).append((window, score))
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
```
|
||||
|
||||
**Change:** Added flip application to negate scores when `theme.get("flip", False)` is `True`. `_render_party_axis_chart_1d()` already did this, but `_render_svd_time_trajectory()` completely ignored the flip flag.
|
||||
|
||||
## Why This Works
|
||||
|
||||
1. **Same PCA basis**: `compute_nd_axes()` computes global PCA across all provided windows. When the single-window view used all uniform-dim windows and the trajectory view used a subset, the resulting components, mean centering, and variance explained were different. Passing the same `window_ids` to `compute_nd_axes()` guarantees identical PCA bases.
|
||||
|
||||
2. **Same flip handling**: The single-window view negates scores when `flip=True`. The trajectory view now does the same, ensuring both views display numerically identical values for the same (window, component, party) tuple.
|
||||
|
||||
## Prevention
|
||||
|
||||
- When multiple views display the same underlying SVD/PCA data, ensure they all call `compute_nd_axes()` with the **identical** set of window IDs
|
||||
- Never apply visual transformations (like `theme["flip"]`) in one view but omit them in another — keep rendering logic symmetric
|
||||
- Add a regression test that asserts `get_aligned_party_scores(window, comp)` equals `_get_aligned_trajectory_scores([window], comp)[window]` for sampled windows and components
|
||||
- Document that `compute_nd_axes()` is a global operation over its input windows; any subset produces a different coordinate frame
|
||||
|
||||
## Related Files
|
||||
|
||||
- `analysis/explorer_data.py` — `_get_aligned_trajectory_scores()` fix (all uniform windows)
|
||||
- `analysis/tabs/_rendering.py` — `_render_svd_time_trajectory()` fix (flip application)
|
||||
- `analysis/political_axis.py` — `compute_nd_axes()` global PCA logic
|
||||
|
||||
## Related Issues
|
||||
|
||||
- Builds on `docs/solutions/ui-bugs/svd-compass-components-position-inconsistency.md` (consistent alignment for components 1-2)
|
||||
- Builds on `docs/solutions/ui-bugs/svd-axis-pole-labels-incorrect-after-flip.md` (runtime flip mechanism)
|
||||
Reference in New Issue
Block a user