Previously the st.plotly_chart call was wrapped in 'except Exception: pass'
which silently swallowed all rendering errors. The user would see no chart
and no error message.
Now:
- Exception message is shown via st.error()
- Diagnostics JSON is shown when debug is enabled (EXPLORER_DEBUG_TRAJECTORIES=1
or UI checkbox), even when trace_count > 0
This reveals the actual root cause when the chart fails to render.
- Lock x_label/y_label to Links-Rechts / Progressief-Conservatief after
classify_axes; Procrustes sign-fixing in compute_2d_axes already ensures
the correct orientation so the heuristic _should_swap_axes call is removed
- Remove visual error bars from party axis chart; 95% CI is now shown in
hover text (party: score, N=n, 95%-BI: [low, high]) to keep the 1D
scatter clean
- Remove show_ci checkbox and parameter — CI is always accessible on hover
- Update tests to match new hover format and absence of error_x
- Extract shared helper that both load_party_axis_scores and
load_party_mp_vectors delegate to, eliminating ~40 lines of
duplicated DB query + vector parsing code
- Remove dead code in load_party_axis_scores that queried mp_metadata
twice (first without ORDER BY, then again with ORDER BY, overwriting)
- Fix _cached_bootstrap_cis parameter: remove _ prefix so Streamlit
actually hashes the input dict instead of caching with no key
- Add load_party_mp_vectors() to return raw per-MP SVD vectors by party
- Extract _build_party_axis_figure() as pure function for testability
- Modify _render_party_axis_chart to accept bootstrap_data and delegate
to the new builder
- When bootstrap_data present: show error_x bars, diamond markers for
N=1 parties, and N=count in hover text
- Wire up bootstrap computation in build_svd_components_tab via cached
_cached_bootstrap_cis wrapper
- Add 6 tests covering figure construction, bootstrap rendering, flip
behavior, and importability
- PC2: rename 'maatschappelijke verantwoordelijkheid' to 'institutioneel
progressivisme' (less normatively loaded), rewrite explanation with actual
party scores (CU=-59, SGP=-25, VVD=-15 — all strongly negative, not
'near the middle'), update pole descriptions
- PC3: remove speculative motivation claim about PVV, state factual
observation that PVV/SP/PvdD/GL-PvdA vote alike despite opposing PC1
- PC7-PC10: add '(indicatief)' to labels — these axes explain <4% EVR
and may be below noise level
- PC7: add explicit fragility warning in explanation
- PC8: clarify DENK/SP negative scores mean active opposition voting,
not lack of focus; note Volt N=1 unreliability
- Scree plot: soften claim that later axes are 'meaningful'
- Re-ran generate_svd_json.py for current_parliament window (100 rows, 10 components)
- Computed party centroid scores per axis from 150 matched MPs
- Updated all 10 SVD_THEMES entries with accurate labels, Dutch explanations
and correct positive/negative pole party attributions
- Key findings: PC1=rechts-links, PC2=populistisch nationalisme vs mainstream,
PC3=verzorgingsstaat vs bezuinigingen, PC6=klimaat & energie,
PC8=Europese defensie-integratie
- Added axis_analysis_data.json and party_svd_scores.json as analysis artifacts
Previously load_scree_data computed L2-norms per dimension on current_parliament
vectors only, giving ~11% for PC1. This was inconsistent with the compass which
uses all windows + Procrustes alignment and gets PC1=24.1%.
Added compute_svd_spectrum() helper to political_axis.py that reuses the same
alignment pipeline. load_scree_data now delegates to it. _render_scree_plot
no longer re-normalizes (inputs are already EVR percentages). Hover label
updated to 'verklaarde variantie'.
Replace hardcoded 'Links-Rechts' / 'Progressief-Conservatief' axis labels
with values from classify_axes(). Add per-year interpretation caption when
axis quality score is below the 0.65 correlation threshold.
- Rename app to 'Motief: de stematlas' in Home.py
- Remove PCA variance caption from compass tab
- Hardcode db_path and window_size; remove sidebar inputs
- Change trajectories default to [CDA, D66, VVD]
- Move quiz to pages/1_Stemwijzer.py; wrap in st.form
- Remove quiz tab from main explorer
- Add pytest dev dep + fix test fixtures (_load_mp_vectors_for_window)
- Add test_pca_axis_orientation with proper PCA variance dominance
The mp_votes table contains both party-aggregate rows (e.g. 'PVV', 'NSC')
and individual MP rows (e.g. 'Aardema, M.'). Running SVD on both together
creates a block-diagonal vote matrix where party codes and individual MPs
occupy disjoint SVD dimensions — causing dim 0 to be zero for all 421 MPs.
Fix: _build_expanded_rows() converts every party-level vote to individual
MP votes using mp_metadata date ranges (active MPs on motion date). Motions
that already have individual MP records are kept as-is. A party name mapping
handles NSC/Nieuw Sociaal Contract and other canonical name variants.
Results for current_parliament: 517 individual MPs, all 8732 motions covered,
dim 0 std=23.1 (was 0.0 for all MPs). PVV/NSC/BBB on positive end, SP/GL/PvdD
on negative end — matches expected left-right political axis.
All 11 annual windows (2016-2026) re-run with the new pipeline.
- load_party_axis_scores now loads individual MP vectors and averages
per party (same data source as the political compass), so SVD axis
rankings are consistent between the two tabs. Previously it used
party-aggregate rows which gave structurally 0 signal on dim 3.
- Re-add component 4 (dim 3) to SVD_THEMES and revert comp_options
filter — with individual MPs averaged, dim 3 now shows real party
separation (the 'publieke voorzieningen vs marktwerking' axis).
- Scree plot y-axis now shows percentage of total variance instead of
raw L2-norms; hover also updated to show '% van totaal'.
Dim 3 captures within-party individual disagreement (MPs splitting
from party lines). Party-aggregate votes are structurally 0 on this
dimension, so the axis chart showed all parties at ~0 with no
discrimination. Confirmed via dry-run: identical to existing data.
Also filter comp_options to only show components with a defined theme,
so component 4 is hidden from the selectbox entirely.
current_parliament has two separate SVD data spaces mixed together.
Party vectors (entity_id without comma) carry the between-party signal
in dims 0-15. Individual MP vectors only have signal in dim 3 and
dims 16-49 (within-party variance). The axis chart uses party vectors,
so the scree must too.
Previous query used DISTINCT ON without ordering by dim, picking arbitrary
(often non-50) dim per window. Rewritten to find the dominant dim per window
(highest count) and include only windows where dominant dim = 50 with >= 10
entities. This surfaces annual windows 2016/2018/2019/2022-2026 that were
previously excluded due to mixed-dim rows from multiple pipeline runs.
- load_positions annual mode now selects actual annual window_ids
('2022', '2023', etc.) instead of Q4 quarterly approximations,
with current_parliament appended as the most-recent anchor
- Sidebar radio defaults to 'Per jaar' (annual) instead of quarterly
- Dutch labels for window size radio: 'Per jaar' / 'Per kwartaal'
- Add import re (needed for strip_paren deduplication)
- Deduplicate MPs with parenthetical first-name variants (e.g. 'Dijk, J.P.'
and 'Dijk, J.P. (Jimmy)') by stripping parens and averaging positions;
fixes 22 duplicate groups across all windows
- Replace select_slider with selectbox for time window control
- Remove non-functional 'Toon namen' checkbox and its usage
- Remove 'Min. MPs per partij' slider and party-size filter
- Add 'Weergave' radio toggle: Kamerleden (individual MPs) vs Partijen
(party centroids computed as mean x/y per party)
- Fix axis ranges: x [-1, 1], y [-0.6, 0.6]
- Party view shows party abbreviation as dot label and hover shows MP count
- Use all individual MPs (not party aggregates) for L2-norm computation;
party-aggregated vectors have near-zero values on some dims due to
Procrustes alignment, producing spurious zeros
- Sort importances descending so scree plot is properly monotonic
- Relabel x-axis as 'Rang' since dim ordering after Procrustes alignment
no longer matches original singular value order
- Add Scatter line trace connecting bar tops for elbow visibility
- Add load_scree_data() cached loader computing L2-norm of party scores
per SVD dimension as a proxy for component importance
- Add _render_scree_plot() rendering a bar chart of the first 15 components
- Insert scree plot + Dutch explanation at the top of build_svd_components_tab
- Clean up _render_party_axis_chart: remove tick numbers, axis line, grid,
and zero-line from the x-axis (pole labels remain as chart title)
Adds new SVD window 'current_parliament' covering 8732 motions and 451 MPs
(vs 7424 motions in old '2025' window, adding ~1300 motions from 2025-Q4+).
Updates explorer.py to query the new window. Regenerates top_svd_top_motions.json.
Also clarifies axis 3 explanation noting FVD's anti-American positioning.
Removes the raw_title[:80] cap on expander labels so full titles show.
Adds scripts/generate_svd_json.py to regenerate top_svd_top_motions.json
from any SVD window after a recompute.
Axes 4 and 5 had inverted sign conventions relative to actual party votes.
Diagnostic confirmed SP/PvdD scored negative on axis 4 (free trade motions)
and FVD scored negative on axis 5 (secular motions), opposite to their
voting behaviour. Fix: swap positive_pole/negative_pole for both axes and
set correct flip direction so progressive parties appear on the left.
- Add ChristenUnie colour alias and CURRENT_PARLIAMENT_PARTIES frozenset (15 parties)
- Add load_party_axis_scores() — queries party SVD vectors from window=2025, cached
- Add _render_party_axis_chart() — 1D Plotly scatter of party positions per axis
- Restructure build_svd_components_tab: replace session-state button/detail-pane with
inline st.expander per motion, split into pos/neg pole columns, batch DB query for
all 10 motions including voting_results, rendered via _render_voting_results
Smoke-tested: 15 parties loaded, all 10 axis-1 motions returned with voting data.
Replace draft SVD_THEMES with themes produced by per-axis analysis of all
10 unique top motions (zero cross-axis overlap, window=2025). Each axis now
has a detailed Dutch-language explanation, positive_pole and negative_pole
labels, displayed as colour-coded columns in the UI.
- Regenerated top_svd_top_motions.json for window=2025 with strict
cross-axis deduplication: 100 unique motions across 10 axes (10 per
axis, zero overlap), sorted by absolute SVD score
- Added SVD_THEMES dict to build_svd_components_tab with Dutch-language
theme label and political-polarisation explanation for each of the 10
axes (e.g. 'Confessioneel-conservatief vs. seculier-progressief')
- Selectbox now shows 'As N — <theme>' instead of bare component number
- Each selected axis shows an info banner with the full explanation
- Motion list buttons show ▲/▼ to indicate positive/negative SVD loading
- Translated UI strings to Dutch for consistency
Root causes:
- Seed selection sorted by controversy_score across all 28k motions, but
only 282 have individual MP vote records. Top controversial motions only
have party-level votes, so match_mps_for_votes always returned empty.
- global_db singleton was used for match/discriminate instead of the db_path
passed to the tab builder.
Fixes:
- Add MotionDatabase.get_motions_with_individual_votes(k) which queries
motions with comma-formatted mp_name votes, ordered by controversy_score
- Replace broken seed logic in build_mp_quiz_tab with this new method
- Replace global_db usages with a local MotionDatabase(db_path) instance
- Guard against motion IDs present in votes but absent from motions DataFrame
