fix: SVD tab now uses raw SVD values for ALL components 1-10

Previously, components 1-2 in the SVD tab used Procrustes-aligned PCA
coordinates (from load_positions), which meant the SVD tab showed PCA
dimensions of the 50D aligned space rather than the actual raw SVD
components. This was a fundamental inconsistency — the SVD tab's component 2
showed completely different party ordering than the raw SVD component 2.

Changes:
- explorer.py: Unified all components 1-10 to use raw SVD values via
  load_party_axis_scores_for_window(). Removed the separate
  load_positions() path for components 1-2. Now all components use the
  same data source (50D vectors from svd_vectors table).
- explorer.py: Updated flip computation to cover ALL components 1-10
  (was range 3-11 for components 3-10 only). The compute_flip_direction
  function correctly determines sign for each component.
- explorer.py: Unified rendering to always use _render_party_axis_chart_1d
  (was _render_party_axis_chart for components 1-2 using 2D coords).
- explorer.py: Unified trajectory to always use load_party_scores_all_windows.
- analysis/config.py: Updated component 1 label (simplified explanation,
  removed coalition-specific policy references).
- analysis/config.py: Updated component 2 label to "Nationalistisch versus
  kosmopolitisch" matching raw SVD data (PVV/FVD at positive extreme,
  Volt/DENK/GL-PvdA at negative extreme).
- tests: Updated test assertions to match new labels.
- scripts/validate_svd_themes.py: Verified all components pass right-wing
  alignment check, config flip consistency, and theme pole consistency.

Fixes the core inconsistency: SVD tab component 2 now uses the same raw
SVD data as components 3-10, with consistent party ordering and labels.
The compass remains a separate PCA-based visualization.
This commit is contained in:
2026-04-13 21:51:21 +02:00
parent 3d69375c01
commit 467b0d1be1
7 changed files with 177 additions and 235 deletions
+51 -2
View File
@@ -193,6 +193,43 @@ def check_canonical_right_on_right(
return divergences
def check_config_flip_consistency(
party_avg_vectors: Dict[str, List[float]],
themes: Dict[int, Dict[str, str]],
canonical_right: frozenset,
canonical_left: frozenset,
num_components: int = 10,
) -> List[Dict]:
"""Check that config flip values match computed flip directions.
The runtime uses compute_flip_direction() to determine flips, but config
also stores a flip value. If they disagree, the config is stale or wrong.
"""
from analysis.svd_labels import compute_flip_direction
scores_dict = {
p: v
for p, v in party_avg_vectors.items()
if p in canonical_right or p in canonical_left
}
mismatches = []
for comp in range(1, num_components + 1):
config_flip = themes.get(comp, {}).get("flip", False)
computed_flip = compute_flip_direction(comp, scores_dict)
if config_flip != computed_flip:
mismatches.append(
{
"component": comp,
"issue": "config_flip_mismatch",
"config_flip": config_flip,
"computed_flip": computed_flip,
}
)
return mismatches
def check_theme_consistency(
party_positions: Dict[str, Dict[int, float]],
themes: Dict[int, Dict[str, str]],
@@ -247,13 +284,19 @@ def main() -> int:
args.components,
)
# Check 2: Theme pole label consistency
# Check 2: Config flip vs computed flip consistency
logger.info("Checking config flip vs computed flip consistency")
flip_mismatches = check_config_flip_consistency(
party_avg_vectors, themes, canonical_right, canonical_left, args.components
)
# Check 3: Theme pole label consistency
logger.info("Checking theme pole label consistency")
theme_divergences = check_theme_consistency(
party_positions, themes, canonical_right, canonical_left
)
all_divergences = canonical_divergences + theme_divergences
all_divergences = canonical_divergences + flip_mismatches + theme_divergences
if all_divergences:
print(f"\n{'=' * 60}")
@@ -282,10 +325,16 @@ def main() -> int:
print(f" Found right: {d['right_found']}")
print(f" Found left: {d['left_found']}")
elif d["issue"] == "config_flip_mismatch":
print(f" Config flip: {d['config_flip']}")
print(f" Computed flip: {d['computed_flip']}")
print(f" → Update SVD_THEMES[{comp}]['flip'] to {d['computed_flip']}")
return 1
else:
print("\n✓ All SVD themes match actual party positions")
print(" - Canonical right-wing parties on right side of all axes")
print(" - Config flip values match computed flip directions")
print(" - Theme pole labels consistent with party positions")
return 0