You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
241 lines
13 KiB
241 lines
13 KiB
---
|
|
title: "Refine axis stability with regression weights and overtone shift"
|
|
type: refactor
|
|
status: active
|
|
date: 2026-04-05
|
|
origin: docs/brainstorms/2026-04-05-motion-semantic-drift-over-time-requirements.md
|
|
---
|
|
|
|
# Refine Axis Stability with Regression Weights and Overtone Shift
|
|
|
|
## Overview
|
|
|
|
Replace the current axis stability computation (party-based sign consistency) with a regression-based approach that measures whether the *semantic features* defining each SVD axis remain stable across windows. Add overtone shift analysis to detect when motion content changes even if party ordering stays the same.
|
|
|
|
## Problem Frame
|
|
|
|
The current stability metric only checks whether left/right parties score on the expected side of each axis. This misses two important questions:
|
|
1. **Axis stability**: Does axis 1 capture the same underlying theme in 2019 and 2024? (e.g., "social vs individual" should be stable even if specific motions change)
|
|
2. **Overtone shift**: Are motions on axis 1 becoming more about migration and less about economics over time, even if PVV still scores higher than SP?
|
|
|
|
The current approach found zero stable axes because it measured party sign consistency, not semantic stability.
|
|
|
|
## Requirements Trace
|
|
|
|
- R1. Compute semantic stability via Ridge regression weights across windows (replaces party sign consistency)
|
|
- R2. Generate stability heatmap showing which axes are semantically comparable across time
|
|
- R3. Detect axis reordering — cases where axis N in window A ≈ axis M in window B
|
|
- R4. Flag unstable axes where semantic signature changes significantly
|
|
- R5. For each stable axis, compute semantic gravity (weighted mean fused embedding) per window
|
|
- R6. Track overtone shift: how semantic gravity moves across windows
|
|
- R7. Identify inflection points where overtone shift accelerated
|
|
- R8. Show example motions and top shifting dimensions at inflection points
|
|
- R9-R12. Party voting analysis (unchanged from existing implementation)
|
|
- R13-R15. Output and parameterization (unchanged)
|
|
|
|
## Scope Boundaries
|
|
|
|
- Refine existing `scripts/motion_drift.py` — no new script
|
|
- Keep party voting analysis and report generation (already working)
|
|
- Annual windows only; quarterly too sparse
|
|
- Ridge regression with scikit-learn (already in dependencies)
|
|
|
|
## Context & Research
|
|
|
|
### Existing Code
|
|
|
|
- `scripts/motion_drift.py` — current implementation with party-based fallback stability
|
|
- `analysis/clustering.py` — UMAP + KMeans infrastructure (not directly used but shows pattern)
|
|
- `scikit-learn>=1.8.0` — already in `pyproject.toml`, provides `Ridge`
|
|
|
|
### Key Technical Decisions
|
|
|
|
- **Ridge regression per axis per window**: Fit `SVD_score ~ fused_embedding` for each axis. The weight vector (2610 dims) is the semantic signature. Compare via cosine similarity across windows.
|
|
- **Semantic gravity for overtone shift**: Weighted mean fused embedding of all motions, weighted by absolute SVD score on the axis. Track how gravity moves across windows.
|
|
- **Top-K dimensions for interpretation**: Extract top-50 dimensions by absolute regression weight. Project gravity onto these to identify which semantic features are shifting.
|
|
- **Party-based fallback kept**: For windows with too few motions for regression (< 50), fall back to party sign consistency.
|
|
|
|
## Open Questions
|
|
|
|
### Resolved During Planning
|
|
|
|
- **Regression type**: Ridge (L2 regularization) — handles 2610-dim vectors without overfitting, already available via scikit-learn.
|
|
- **Alpha (regularization strength)**: Default 1.0, parameterized via `--regression-alpha`. Will test 0.1, 1.0, 10.0 during execution.
|
|
- **Top-K dimensions for interpretation**: K=50 — enough to capture semantic signal without noise.
|
|
- **Overtone shift metric**: Cosine distance between semantic gravity points across consecutive windows. Threshold for inflection: 2× median shift rate.
|
|
|
|
### Deferred to Implementation
|
|
|
|
- Optimal alpha for Ridge regression — will test against real data and pick value that gives most interpretable weight vectors
|
|
- Whether to normalize fused embeddings before regression (likely yes, since SVD dims are ~1-100 scale and text dims are ~0-1)
|
|
|
|
## High-Level Technical Design
|
|
|
|
> *This illustrates the intended approach and is directional guidance for review, not implementation specification.*
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Refined Axis Stability + Overtone Shift │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ 1. Per-Axis Ridge Regression (per window) │
|
|
│ ├── For each SVD axis k: │
|
|
│ │ X = fused_embeddings (n_motions × 2610) │
|
|
│ │ y = SVD scores on axis k (n_motions) │
|
|
│ │ w_k = Ridge.fit(X, y).coef_ (2610-dim weight vector) │
|
|
│ └── Output: weight_vectors[window][axis] │
|
|
│ │
|
|
│ 2. Stability Matrix │
|
|
│ ├── For each axis k, compute cosine similarity of w_k │
|
|
│ │ across all window pairs │
|
|
│ └── Output: stability_matrix[window][window][axis] │
|
|
│ │
|
|
│ 3. Overtone Shift │
|
|
│ ├── For each axis k and window: │
|
|
│ │ gravity_k = weighted_mean(fused_embeddings, │
|
|
│ │ weights=abs(SVD_scores_k)) │
|
|
│ │ shift_k = cosine_distance(gravity_k[t], gravity_k[t+1]) │
|
|
│ └── Output: shift_series[axis] = [shift values per window] │
|
|
│ │
|
|
│ 4. Interpretation │
|
|
│ ├── Top-50 dimensions per axis (by |weight|) │
|
|
│ ├── Project gravity onto top dimensions to see shifts │
|
|
│ └── Report: "Axis 1 stable (0.82), overtone shift (0.45) │
|
|
│ — migration framing gained +0.31, economic -0.22" │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Implementation Units
|
|
|
|
- [ ] **Unit 1: Add Ridge regression-based stability computation**
|
|
|
|
**Goal:** Replace `compute_axis_stability()` with regression-based version.
|
|
|
|
**Requirements:** R1, R2, R3, R4
|
|
|
|
**Dependencies:** None (replaces existing function)
|
|
|
|
**Files:**
|
|
- Modify: `scripts/motion_drift.py` (replace `compute_axis_stability`)
|
|
- Modify: `tests/test_motion_drift.py` (update stability tests)
|
|
|
|
**Approach:**
|
|
- New `compute_axis_stability()` function:
|
|
- For each window, load motion scores + fused embeddings
|
|
- For each axis k (1-10), fit Ridge regression: `score_k ~ fused_embedding`
|
|
- Normalize features before fitting (StandardScaler on fused embeddings)
|
|
- Extract weight vector w_k (2610 dims)
|
|
- Compute pairwise cosine similarity of w_k across windows
|
|
- Return stability matrix, stable/reordered/unstable axes
|
|
- Keep `_compute_stability_fallback()` for windows with < 50 motions
|
|
- Add `--regression-alpha` CLI argument (default 1.0)
|
|
|
|
**Patterns to follow:**
|
|
- `sklearn.linear_model.Ridge` — standard usage: `Ridge(alpha=alpha).fit(X, y)`
|
|
- `sklearn.preprocessing.StandardScaler` — normalize features before regression
|
|
|
|
**Test scenarios:**
|
|
- Happy path: regression produces weight vectors with cosine similarity in [-1, 1]
|
|
- Happy path: synthetic data with known semantic signatures recovers stable axes
|
|
- Edge case: window with < 50 motions falls back to party-based method
|
|
- Edge case: all motions have same score on axis (degenerate case)
|
|
- Integration: run against real data, verify stability values are non-zero
|
|
|
|
**Verification:**
|
|
- Stability matrix has correct shape (n_windows × n_windows × n_components)
|
|
- At least some axes show stability > 0.5 on real data
|
|
- Fallback triggers correctly for sparse windows
|
|
|
|
- [ ] **Unit 2: Add overtone shift analysis**
|
|
|
|
**Goal:** Compute semantic gravity trajectories and detect overtone shifts.
|
|
|
|
**Requirements:** R5, R6, R7, R8
|
|
|
|
**Dependencies:** Unit 1 (needs regression weight vectors for top-K dimension interpretation; shift computation itself is independent)
|
|
|
|
**Files:**
|
|
- Create: `compute_overtone_shift()` function in `scripts/motion_drift.py`
|
|
- Modify: `scripts/motion_drift.py` (call overtone shift in main)
|
|
- Modify: `tests/test_motion_drift.py` (add overtone shift tests)
|
|
|
|
**Approach:**
|
|
- New `compute_overtone_shift(db_path, stable_axes, windows, top_k=50)` function:
|
|
- For each stable axis and window:
|
|
- Load motion scores and fused embeddings
|
|
- Compute semantic gravity: weighted mean of fused embeddings, weights = abs(SVD scores)
|
|
- Extract top-K dimensions by absolute regression weight
|
|
- Project gravity onto top-K dimensions
|
|
- Compute cosine distance between consecutive window gravity points
|
|
- Detect inflection points: shift > 2× median shift rate
|
|
- For each inflection, identify top shifting dimensions and example motions
|
|
- Return shift series, inflection points, dimension-level analysis
|
|
|
|
**Test scenarios:**
|
|
- Happy path: overtone shift returns shift series for each stable axis
|
|
- Happy path: synthetic data with known shift detects inflection point
|
|
- Edge case: axis with only 2 windows returns shift but no inflection points
|
|
- Edge case: monotonic shift returns no inflection points
|
|
- Integration: run against real data, verify shift values are plausible
|
|
|
|
**Verification:**
|
|
- Shift series has correct length (n_windows - 1 per axis)
|
|
- Inflection points (if any) include dimension-level analysis
|
|
- Top shifting dimensions are reported with direction and magnitude
|
|
|
|
- [ ] **Unit 3: Update report generation with new metrics**
|
|
|
|
**Goal:** Update report to show both stability and overtone shift per axis.
|
|
|
|
**Requirements:** R13, R14
|
|
|
|
**Dependencies:** Units 1, 2
|
|
|
|
**Files:**
|
|
- Modify: `scripts/motion_drift.py` (`_generate_report` function)
|
|
- Modify: `tests/test_motion_drift.py` (update report tests)
|
|
|
|
**Approach:**
|
|
- Update `_generate_report()` to include:
|
|
- Stability heatmap (regression weight similarity)
|
|
- Overtone shift timeline per axis (line chart with inflection markers)
|
|
- For each stable axis: stability score + overtone shift magnitude
|
|
- Top shifting dimensions table: dimension index, direction, magnitude
|
|
- Example motions at inflection points
|
|
- Keep existing party voting analysis section unchanged
|
|
|
|
**Test scenarios:**
|
|
- Happy path: report includes both stability and overtone shift sections
|
|
- Happy path: all charts generated and embedded
|
|
- Edge case: no stable axes → report notes this, skips overtone shift
|
|
|
|
**Verification:**
|
|
- Report contains stability heatmap, shift timelines, and dimension analysis
|
|
- All PNG files exist in output directory
|
|
|
|
## System-Wide Impact
|
|
|
|
- **Interaction graph:** Replaces `compute_axis_stability()` — callers (main function) unchanged API
|
|
- **Unchanged invariants:** Party voting analysis, report structure, CLI interface
|
|
- **New dependency:** None — scikit-learn already in dependencies
|
|
|
|
## Risks & Dependencies
|
|
|
|
| Risk | Likelihood | Impact | Mitigation |
|
|
|------|-----------|--------|------------|
|
|
| Ridge regression overfits with 2610 features | Medium | Medium | Use Ridge (L2 regularization), test multiple alpha values, validate with cross-validation |
|
|
| Fused embeddings have different dimensions across windows | Low | Low | Already handled — truncate to min dimension |
|
|
| Regression takes too long on full dataset | Medium | Low | 9 windows × 10 axes = 90 Ridge fits. Each fit on ~3000×2610 matrix ~0.1s with sklearn. Total ~9s. Acceptable. |
|
|
| Weight vectors are hard to interpret | Medium | Low | Focus on top-50 dimensions, report direction and magnitude clearly |
|
|
|
|
## Documentation / Operational Notes
|
|
|
|
- Updated script: `scripts/motion_drift.py` — new stability metric, new overtone shift analysis
|
|
- Report output: markdown with stability heatmap, shift timelines, dimension analysis
|
|
- Existing report sections (party voting) unchanged
|
|
|
|
## Sources & References
|
|
|
|
- **Origin document:** [docs/brainstorms/2026-04-05-motion-semantic-drift-over-time-requirements.md](docs/brainstorms/2026-04-05-motion-semantic-drift-over-time-requirements.md)
|
|
- Related code: `scripts/motion_drift.py` (existing implementation), `analysis/clustering.py` (UMAP/KMeans patterns)
|
|
- Ridge regression: `sklearn.linear_model.Ridge`
|
|
|