The SVD explorer shows *where* parties and motions sit on axes at a point in time, but doesn't reveal *how the semantic content of motions on each axis evolves*. Users can see that axis 1 separates right-wing from left-wing motions, but can't answer questions like:
- Did "right-wing" motions on axis 1 become more extreme over time, or did the framing shift?
- Are the SVD axes themselves stable across windows, or do they capture different dimensions in different periods?
- How did individual parties' motion sponsorship patterns shift along axes over time?
This analysis would add a temporal dimension to the SVD analysis, revealing how political discourse evolved.
## Requirements
**Axis Stability**
- R1. Compute cosine similarity between SVD component vectors (or motion projection patterns) across all annual windows (2016-2024)
- R2. Generate a stability heatmap showing which axes are comparable across time (e.g., axis 1 in 2020 ≈ axis 1 in 2024)
- R3. Detect axis reordering — cases where axis N in window A ≈ axis M in window B where N ≠ M
- R4. Flag unstable axes where no clear match exists across windows (may indicate the dimension dissolved or merged)
**Semantic Drift**
- R5. For each stable axis, compute the average fused embedding centroid of top N motions per window (ranked by absolute SVD loading on that axis)
- R6. Track semantic drift over time using cosine distance between consecutive window centroids
- R7. Identify inflection points where drift accelerated (statistical change-point detection or threshold-based)
- R8. For each inflection point, show example motions before/after to illustrate framing shifts
**Party-Level Drift**
- R9. For each party, compute their voting centroid per window along each stable axis (weighted average of motions they voted "voor" on)
- R10. Track how parties move along axes over time (party trajectory plots)
- R11. Detect cross-ideological voting — cases where left-wing parties vote "voor" on right-wing motions (or vice versa), and how this pattern changes over time
- R12. Show concrete examples: motions where parties voted against their ideological alignment, with before/after comparisons across windows
**Output**
- R13. Script produces a markdown report with embedded charts (static, shareable)
- R14. Report includes: axis stability heatmap, semantic drift timelines, party trajectory plots, inflection point analysis with motion examples
- R15. Script is parameterized: `--db`, `--windows`, `--top-n`, `--output` for flexibility
## Success Criteria
- Axis stability heatmap clearly shows which of the 10 SVD components are comparable across available annual windows (2016-2024)
- Semantic drift timelines reveal and characterize drift patterns; where inflection points exist, identifies at least 2-3 per affected axis
- Party-level analysis reveals cross-ideological voting patterns — cases where parties vote against their expected position on the axis
- Report is readable and actionable — a political analyst could use it to understand discourse evolution
- Script runs in under 2 minutes on the full dataset
## Scope Boundaries
- Annual windows only (2016-2024) — quarterly windows are too sparse for meaningful drift analysis. Note: 2025-2026 data may be incomplete; script should validate coverage before analysis.
- Fused embeddings (SVD + text) for semantic analysis, not raw SVD vectors alone
- No UI/explorer integration in this phase — script + report only
- No statistical significance testing beyond basic change-point detection (keep it interpretable)
## Key Decisions
- **Fused embeddings for semantic drift, SVD vectors for axis stability:** Fused embeddings (50 SVD + 2560 text dims) capture both voting patterns and semantic content, making them better for drift analysis. SVD component vectors are used for axis stability comparison (comparing vector directions, not semantic content). Note: SVD component vectors (V^T matrix) are not currently stored — axis stability must be computed indirectly.
- **Annual windows only:** Quarterly windows have too few motions (some have <100)forstablecentroidcomputation
- **Cosine similarity for axis stability:** Standard metric for comparing vector directions; interpretable (1.0 = identical, 0.0 = orthogonal)
- **Centroid-based drift:** Average embedding of top motions per window is more stable than individual motion tracking and captures the axis's semantic center
## Dependencies / Assumptions
- SVD component vectors (V^T matrix) must be stored during SVD pipeline runs — currently not persisted. A new table or metadata field is needed for axis stability comparison
- Motion text is available in `motions` table for example extraction
- Party metadata is available in `mp_metadata`; motions can be linked to parties via voting records (`mp_votes`), not sponsorship data
## Outstanding Questions
### Resolve Before Planning
- [Affects R7][Technical] What change-point detection method? Simple threshold on drift rate (recommended — simpler and more interpretable, no new dependencies).
- [Affects R9][Technical] How to link motions to parties? Use voting records from `mp_votes` (party voted "voor" on motion). This measures party alignment patterns, not sponsorship.
### Deferred to Planning
- [Affects R5][Needs research] What's the optimal N for top motions per axis? Too few = noisy, too many = dilutes the signal. Should test N=10, 20, 50.
- [Affects R13][Technical] Charting library: matplotlib recommended for static markdown embedding (Plotly already in stack but produces HTML; matplotlib produces static images suitable for markdown).
- [Affects R11][Needs research] What threshold counts as "cross-ideological voting"? Provisional: a party voting "voor" on motions where the canonical opposite-wing parties have high absolute loadings. E.g., left-wing party voting "voor" on motions where PVV/FVD/JA21/SGP have high positive scores.
## Results (2026-04-05)
### Key Finding: Stability and Overtone Shift are Independent Phenomena
Analysis of 9 annual windows (2016-2026) revealed that **axis stability** and **overtone shift** measure fundamentally different aspects of SVD axes over time:
| **Axis Stability** | Structural consistency of which motions/embeddings load on axis | 0.70-0.83 for stable axes | The same semantic pattern persists |
| **Overtone Shift** | How motion content evolves over time | 1.30-1.97 cosine distance | The specific topics change substantially |
**Critical insight**: Axes can be structurally stable (parties vote similarly across years) while their semantic content drifts dramatically (different motions define the axis).
### Axis Stability Results
| Axis | Stability | Status | Interpretation |
|------|-----------|--------|----------------|
| 1 | 0.83 | ✅ Stable | Coalition vs opposition pattern consistent |
This analysis explores the relationship between **axis stability** (structural consistency of SVD components over time) and **overtone shift** (semantic drift of motion content within those stable axes). The key finding is that these are **independent phenomena**: axes can be structurally stable (same parties voting similarly) while their semantic content drifts dramatically.
## Key Finding: Stability vs. Semantic Content are Independent
| Phenomenon | What it Measures | Typical Value | Interpretation |
**Critical observation**: ALL stable axes show high overtone shift (1.3-1.97), with no inflection points detected. This indicates **gradual, continuous semantic drift** rather than sudden shifts.
## Interpretation Framework
### The "Axis Stability" Metric
Axis stability uses **Lasso regression** to learn the semantic signature of each axis:
```
SVD_score ~ fused_embedding
```
The learned weight vector (2610 dimensions) represents which embedding dimensions are most predictive of an axis score. Stability is measured by comparing these weight vectors across windows using:
- **Cosine similarity** of full weight vectors
- **Jaccard similarity** of top-100 weighted dimensions
Why Lasso (alpha=0.1)? The L1 regularization produces sparse weight vectors, concentrating on the most important semantic dimensions. This makes cross-window comparison more robust than dense Ridge regression.
### The "Overtone Shift" Metric
Overtone shift computes **semantic gravity** — the weighted mean fused embedding of all motions on an axis:
The cosine distance between gravity vectors of consecutive windows measures how the "center of mass" of motion content moves. High shift values (1.3-1.9) indicate the motion topics that define each axis change substantially over time.
## Implications for Interpretation
### For Users of the Stemwijzer
1. **Axis labels are temporally bounded** — The label "Rechts kabinetsbeleid versus links oppositiebeleid" accurately describes the 2016-2026 period, but the specific motions that exemplify this axis have changed.
2. **Cross-temporal comparison is valid structurally but not semantically** — Party positions along Axis 1 are comparable across years (stable structure), but the meaning of extreme positions has shifted (high overtone).
3. **Axis 6 (Migration/Culture)** is an exception — Low stability (0.35) suggests this axis may have fundamentally changed meaning or composition over the period.
### For Analysts Studying Parliamentary Evolution
1. **Coalition/opposition as a dimension is remarkably stable** — Despite changes in coalition composition (Rutte III, Rutte IV, Schoof), the first axis consistently captures this dynamic.
2. **Policy content evolves within stable voting patterns** — What constitutes "coalition policy" in 2026 differs semantically from 2016, even if the voting alignment remains.
3. **The 2022-2023 period may be significant** — Gap in windows (2020-2021) coincides with COVID and government crises, potentially affecting overtone patterns.
## Methodological Notes
### Why Lasso (alpha=0.1)?
Three alternatives were evaluated and rejected:
| Approach | Problem |
|----------|---------|
| Jaccard similarity of top-N motion IDs | Motions are unique per window — 0% overlap |
| Cosine similarity of embedding centroids | Near-zero similarity due to varying embedding dimensions |
| Ridge regression weights | Dense weights less interpretable; Lasso concentrates signal |
- **Stability**: Results are less sensitive to embedding dimension changes
### Dimension Alignment Challenge
Fused embeddings have varying dimensions across windows (typically 768-2610). All comparisons use **minimum common dimension** alignment to ensure valid cosine similarity computation.
### Inflection Point Detection
Inflection points are defined as shift/drift rates exceeding 2× median rate. The absence of detected inflection points suggests **gradual, continuous drift** rather than sudden semantic shifts — consistent with how policy debates evolve incrementally.
## Recommendations
### For Stemwijzer Maintenance
1. **Re-run overtone analysis after SVD recomputation** — Current themes may drift further from the underlying data
# SVD Axis Stability and Overtone Shift are Independent Phenomena
## Key Insight
When analyzing SVD axes across time windows, **axis stability** and **overtone shift** measure fundamentally different phenomena:
| Phenomenon | What it Measures | How to Compute |
|------------|-----------------|----------------|
| **Axis Stability** | Whether the same motions/embeddings load on an axis | Lasso regression: `SVD_score ~ fused_embedding`, compare weight vectors via cosine similarity + Jaccard |
| **Overtone Shift** | How motion content evolves over time | Semantic gravity (weighted mean embedding) tracking via cosine distance |
**The implication**: An axis can be "stable" (parties vote similarly across years) while its semantic content drifts dramatically (different motions define the axis).
## Evidence
Analysis of 9 annual windows (2016-2026) revealed:
- **9 of 10 axes are stable** (similarity > 0.7)
- **All stable axes show high overtone shift** (1.3-1.97 cosine distance)
- **No inflection points detected** — drift is gradual, not sudden
| Overtone Shift | 1.47 avg, 1.97 max | Motion content defining "coalition" vs "opposition" has changed substantially |
This means: PVV, VVD, NSC, BBB consistently vote together against SP, GL-PvdA, PvdD across all windows — but the specific motions that exemplify "coalition policy" in 2026 are semantically different from 2016.
## Why This Matters
1. **Axis labels are temporally bounded** — "Rechts kabinetsbeleid versus links oppositiebeleid" accurately describes 2016-2026, but the underlying motions have evolved.
2. **Cross-temporal comparison is valid structurally but not semantically** — Party positions are comparable; motion content is not.
3. **Axis 6 (Migration/Culture)** is an exception — Low stability (0.35) suggests fundamental change in how this dimension is structured.
## How to Analyze This
Use `scripts/motion_drift.py`:
```bash
uv run python scripts/motion_drift.py --db data/motions.db --output reports/drift
```
The script computes:
- **Axis stability**: Lasso regression weights compared across windows
- **Overtone shift**: Semantic gravity tracking
- **Inflection points**: Sudden drift detection
- **Party trajectories**: How parties move along stable axes
## Prevention
When updating SVD themes:
1. Run `scripts/motion_drift.py` to check current overtone shift levels
2. Verify that theme descriptions match current motion content, not historical content
3. Monitor Axis 6 specifically for stability issues
4. Consider temporal weighting in visualizations — recent windows better represent current semantics
## Related
- `scripts/motion_drift.py` — Analysis script
- `docs/research/2026-04-05-svd-overtone-shift-deep-dive.md` — Deep analysis
