fix: SVD tab now uses raw SVD values for ALL components 1-10
Previously, components 1-2 in the SVD tab used Procrustes-aligned PCA coordinates (from load_positions), which meant the SVD tab showed PCA dimensions of the 50D aligned space rather than the actual raw SVD components. This was a fundamental inconsistency — the SVD tab's component 2 showed completely different party ordering than the raw SVD component 2. Changes: - explorer.py: Unified all components 1-10 to use raw SVD values via load_party_axis_scores_for_window(). Removed the separate load_positions() path for components 1-2. Now all components use the same data source (50D vectors from svd_vectors table). - explorer.py: Updated flip computation to cover ALL components 1-10 (was range 3-11 for components 3-10 only). The compute_flip_direction function correctly determines sign for each component. - explorer.py: Unified rendering to always use _render_party_axis_chart_1d (was _render_party_axis_chart for components 1-2 using 2D coords). - explorer.py: Unified trajectory to always use load_party_scores_all_windows. - analysis/config.py: Updated component 1 label (simplified explanation, removed coalition-specific policy references). - analysis/config.py: Updated component 2 label to "Nationalistisch versus kosmopolitisch" matching raw SVD data (PVV/FVD at positive extreme, Volt/DENK/GL-PvdA at negative extreme). - tests: Updated test assertions to match new labels. - scripts/validate_svd_themes.py: Verified all components pass right-wing alignment check, config flip consistency, and theme pole consistency. Fixes the core inconsistency: SVD tab component 2 now uses the same raw SVD data as components 3-10, with consistent party ordering and labels. The compass remains a separate PCA-based visualization.
This commit is contained in:
+18
-24
@@ -66,35 +66,29 @@ PARTY_COLOURS: Dict[str, str] = {
|
|||||||
|
|
||||||
SVD_THEMES: dict[int, dict[str, str]] = {
|
SVD_THEMES: dict[int, dict[str, str]] = {
|
||||||
1: {
|
1: {
|
||||||
"label": "Rechts kabinetsbeleid versus links oppositiebeleid",
|
"label": "Rechts versus links (economisch en migratiebeleid)",
|
||||||
"explanation": (
|
"explanation": (
|
||||||
"Deze as scheidt het rechts kabinetsbeleid van links oppositiebeleid. "
|
"Deze as scheidt rechts kabinetsbeleid van links oppositiebeleid. "
|
||||||
"Aan de positieve kant staan moties die passen bij het kabinetsbeleid: "
|
"Aan de positieve kant staan PVV, NSC, SGP en BBB. "
|
||||||
"Eurofighter Typhoons, defensie-uitgaven naar 3% bbp, F-35 reservedelen, "
|
"Aan de negatieve kant staan PvdD, GroenLinks-PvdA en DENK. "
|
||||||
"marine-steun aan Rode Zee en asielrestricties. "
|
"Deze as weerspiegelt de klassieke links-rechts verdeling op economisch en migratiebeleid."
|
||||||
"PVV, VVD, NSC en BBB scoren sterk positief. "
|
|
||||||
"Aan de negatieve kant staan moties uit de oppositie: "
|
|
||||||
"zorgbuurthuizen voor ouderen, boycot van Israël, sancties, en internationale "
|
|
||||||
"klimaatsamenwerking. GroenLinks-PvdA, SP, PvdD en Volt scoren negatief. "
|
|
||||||
"Deze as weerspiegelt de coalitie-oppositie dynamiek."
|
|
||||||
),
|
),
|
||||||
"positive_pole": "Rechts: PVV, VVD, NSC, BBB, JA21 — kabinetsbeleid, defensie en restricties",
|
"positive_pole": "Rechts: PVV, NSC, SGP, BBB — kabinetsbeleid, defensie en restricties",
|
||||||
"negative_pole": "Links: GroenLinks-PvdA, SP, PvdD, Volt, DENK — oppositie, zorg en multilateraal",
|
"negative_pole": "Links: PvdD, GroenLinks-PvdA, DENK — oppositie, zorg en multilateraal",
|
||||||
"flip": False,
|
"flip": False,
|
||||||
},
|
},
|
||||||
2: {
|
2: {
|
||||||
"label": "PVV/FVD-populisme versus mainstream-partijen",
|
"label": "Nationalistisch versus kosmopolitisch",
|
||||||
"explanation": (
|
"explanation": (
|
||||||
"Deze as scheidt het PVV/FVD-populisme van het overige parliament. "
|
"Deze as meet een onafhankelijke culturele dimensie: nationalistisch-populistisch "
|
||||||
"Alleen PVV en FVD scoren positief; alle andere partijen scoren negatief. "
|
"tegenover kosmopolitisch-mainstream. Aan de positieve kant staan PVV en FVD. "
|
||||||
"Positieve moties: Syriërs terugsturen, geen geld aan Jordanië, tijdelijke "
|
"Aan de negatieve kant staan Volt, GroenLinks-PvdA, DENK en SP. "
|
||||||
"bescherming Oekraïne beëindigen, uitstappen uit WHO en klimaatakkoorden. "
|
"Deze as is onafhankelijk van links-rechts (as 1) en scheidt partijen "
|
||||||
"Negatieve moties: digitale toegankelijkheid Caribisch Nederland, ethiekprogramma "
|
"op hun houding tegenover nationale identiteit, EU-samenwerking en de "
|
||||||
"Defensie, zorg voor slachtoffers bombardement Hawija, internationale klimaatsamenwerking. "
|
"etnisch-culturele dimensie."
|
||||||
"Dit is geen links-rechts verdeling maar een populistisch vs. mainstream onderscheid."
|
|
||||||
),
|
),
|
||||||
"positive_pole": "PVV en FVD — soevereiniteit en anti-establishment",
|
"positive_pole": "Nationalistisch/populistisch — PVV, FVD: nationale identiteit en soevereiniteit",
|
||||||
"negative_pole": "Overige partijen: VVD, CDA, SGP, ChristenUnie, GroenLinks-PvdA, D66, Volt, BBB",
|
"negative_pole": "Kosmopolitisch/mainstream — Volt, GL-PvdA, DENK, SP: EU en internationale samenwerking",
|
||||||
"flip": False,
|
"flip": False,
|
||||||
},
|
},
|
||||||
3: {
|
3: {
|
||||||
@@ -126,7 +120,7 @@ SVD_THEMES: dict[int, dict[str, str]] = {
|
|||||||
),
|
),
|
||||||
"positive_pole": "D66, CDA, JA21 — moties met brede steun",
|
"positive_pole": "D66, CDA, JA21 — moties met brede steun",
|
||||||
"negative_pole": "NSC, BBB — moties met andere focus",
|
"negative_pole": "NSC, BBB — moties met andere focus",
|
||||||
"flip": True,
|
"flip": False,
|
||||||
},
|
},
|
||||||
5: {
|
5: {
|
||||||
"label": "Christelijk-sociaal en gemeenschapswaarden versus progressieve individuele rechten",
|
"label": "Christelijk-sociaal en gemeenschapswaarden versus progressieve individuele rechten",
|
||||||
@@ -211,7 +205,7 @@ SVD_THEMES: dict[int, dict[str, str]] = {
|
|||||||
),
|
),
|
||||||
"positive_pole": "Pragmatisch-bestuurlijk: SGP, ChristenUnie, DENK, SP — concrete oplossingen",
|
"positive_pole": "Pragmatisch-bestuurlijk: SGP, ChristenUnie, DENK, SP — concrete oplossingen",
|
||||||
"negative_pole": "Systeemhervorming: D66, JA21, PVV — idealistische beleidsposities",
|
"negative_pole": "Systeemhervorming: D66, JA21, PVV — idealistische beleidsposities",
|
||||||
"flip": True,
|
"flip": False,
|
||||||
},
|
},
|
||||||
10: {
|
10: {
|
||||||
"label": "Kritisch op overheidsbemoeienis versus pro-regulering (indicatief)",
|
"label": "Kritisch op overheidsbemoeienis versus pro-regulering (indicatief)",
|
||||||
|
|||||||
+45
-144
@@ -953,8 +953,9 @@ def _build_party_axis_figure(
|
|||||||
|
|
||||||
pos_pole = theme.get("positive_pole", "")
|
pos_pole = theme.get("positive_pole", "")
|
||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
left_label = theme.get("left_pole", pos_pole if flip else neg_pole)
|
# Labels always from poles: negative_pole = LEFT, positive_pole = RIGHT
|
||||||
right_label = theme.get("right_pole", neg_pole if flip else pos_pole)
|
left_label = neg_pole
|
||||||
|
right_label = pos_pole
|
||||||
|
|
||||||
fig.update_layout(
|
fig.update_layout(
|
||||||
height=160,
|
height=160,
|
||||||
@@ -1073,8 +1074,9 @@ def _render_party_axis_chart_1d(
|
|||||||
# Determine pole labels based on flip
|
# Determine pole labels based on flip
|
||||||
pos_pole = theme.get("positive_pole", "")
|
pos_pole = theme.get("positive_pole", "")
|
||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
left_label = theme.get("left_pole", pos_pole if flip else neg_pole)
|
# Labels always from poles: negative_pole = LEFT, positive_pole = RIGHT
|
||||||
right_label = theme.get("right_pole", neg_pole if flip else pos_pole)
|
left_label = neg_pole
|
||||||
|
right_label = pos_pole
|
||||||
|
|
||||||
# Update layout with same format as components 1-2
|
# Update layout with same format as components 1-2
|
||||||
fig.update_layout(
|
fig.update_layout(
|
||||||
@@ -1238,10 +1240,9 @@ def _render_svd_time_trajectory(
|
|||||||
# Determine pole labels based on theme (use reference flip from current_parliament)
|
# Determine pole labels based on theme (use reference flip from current_parliament)
|
||||||
pos_pole = theme.get("positive_pole", "")
|
pos_pole = theme.get("positive_pole", "")
|
||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
# Use the theme's flip value (computed from current_parliament) for label orientation
|
# Labels always from poles: negative_pole = LEFT, positive_pole = RIGHT
|
||||||
reference_flip = theme.get("flip", False)
|
left_label = neg_pole
|
||||||
left_label = theme.get("left_pole", pos_pole if reference_flip else neg_pole)
|
right_label = pos_pole
|
||||||
right_label = theme.get("right_pole", neg_pole if reference_flip else pos_pole)
|
|
||||||
|
|
||||||
# Y-axis labels
|
# Y-axis labels
|
||||||
y_labels = {}
|
y_labels = {}
|
||||||
@@ -2532,76 +2533,9 @@ def build_svd_components_tab(db_path: str) -> None:
|
|||||||
motions = comp_map.get(comp_sel, [])
|
motions = comp_map.get(comp_sel, [])
|
||||||
|
|
||||||
# Party axis chart
|
# Party axis chart
|
||||||
# Default party scores already loaded earlier for sidebar controls
|
# Default party scores already loaded earlier for sidebar controls.
|
||||||
# For components 1 and 2, prefer MP-centroid values from the Procrustes-aligned
|
# ALL components 1-10 use raw (non-aligned) SVD vectors.
|
||||||
# positions_by_window so the compass matches the trajectories (MP-mean centroids).
|
# The compass uses Procrustes-aligned PCA — separate visualization.
|
||||||
if comp_sel in (1, 2):
|
|
||||||
try:
|
|
||||||
positions_by_window, axis_def = load_positions(db_path)
|
|
||||||
if axis_def is None:
|
|
||||||
axis_def = {}
|
|
||||||
|
|
||||||
# Window selector for components 1-2 (same windows as Political Compass)
|
|
||||||
year_windows = sorted(
|
|
||||||
w for w in positions_by_window if w != "current_parliament"
|
|
||||||
)
|
|
||||||
has_current = "current_parliament" in positions_by_window
|
|
||||||
windows = year_windows + (["current_parliament"] if has_current else [])
|
|
||||||
|
|
||||||
def _window_label(w: str) -> str:
|
|
||||||
if w == "current_parliament":
|
|
||||||
return "Huidig parlement"
|
|
||||||
return w
|
|
||||||
|
|
||||||
with col1:
|
|
||||||
window = st.selectbox(
|
|
||||||
"Jaar",
|
|
||||||
options=windows,
|
|
||||||
index=len(windows) - 1, # default: current_parliament
|
|
||||||
format_func=_window_label,
|
|
||||||
)
|
|
||||||
|
|
||||||
pos = positions_by_window.get(window, {})
|
|
||||||
|
|
||||||
# build party -> list of MP x/y coords
|
|
||||||
party_map = load_party_map(db_path)
|
|
||||||
per_party_coords: dict = {}
|
|
||||||
party_mp_counts: dict = {} # Track MP count per party
|
|
||||||
for ent, (x, y) in pos.items():
|
|
||||||
party = party_map.get(ent)
|
|
||||||
if party is None:
|
|
||||||
continue
|
|
||||||
per_party_coords.setdefault(party, []).append((x, y))
|
|
||||||
party_mp_counts[party] = party_mp_counts.get(party, 0) + 1
|
|
||||||
|
|
||||||
# construct party_scores mapping: prefer MP centroid [x,y], fallback to default vector
|
|
||||||
party_scores = {}
|
|
||||||
for party in set(
|
|
||||||
list(per_party_coords.keys()) + list(party_scores_default.keys())
|
|
||||||
):
|
|
||||||
coords = per_party_coords.get(party)
|
|
||||||
if coords:
|
|
||||||
xs = [c[0] for c in coords]
|
|
||||||
ys = [c[1] for c in coords]
|
|
||||||
party_scores[party] = [float(np.mean(xs)), float(np.mean(ys))]
|
|
||||||
else:
|
|
||||||
# fallback: use the default single-window SVD mean vector
|
|
||||||
party_scores[party] = party_scores_default.get(party, [])
|
|
||||||
|
|
||||||
except Exception:
|
|
||||||
# On any error, fall back to the old behaviour
|
|
||||||
logger.exception(
|
|
||||||
"Failed to derive party centroids from positions_by_window; falling back to load_party_axis_scores"
|
|
||||||
)
|
|
||||||
party_scores = party_scores_default
|
|
||||||
# For fallback, compute MP counts from party_mp_vectors
|
|
||||||
party_mp_counts = (
|
|
||||||
{p: len(v) for p, v in party_mp_vectors.items()}
|
|
||||||
if party_mp_vectors
|
|
||||||
else {}
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# Components 3-10: use SVD vectors with year selection
|
|
||||||
# Get available windows from svd_vectors
|
# Get available windows from svd_vectors
|
||||||
available_windows = get_uniform_dim_windows(db_path)
|
available_windows = get_uniform_dim_windows(db_path)
|
||||||
year_windows = sorted(w for w in available_windows if w != "current_parliament")
|
year_windows = sorted(w for w in available_windows if w != "current_parliament")
|
||||||
@@ -2610,7 +2544,7 @@ def build_svd_components_tab(db_path: str) -> None:
|
|||||||
|
|
||||||
def _svd_window_label(w: str) -> str:
|
def _svd_window_label(w: str) -> str:
|
||||||
if w == "current_parliament":
|
if w == "current_parliament":
|
||||||
return "Huidig parlement"
|
return "Huidig parliament"
|
||||||
return w
|
return w
|
||||||
|
|
||||||
with col1:
|
with col1:
|
||||||
@@ -2628,40 +2562,46 @@ def build_svd_components_tab(db_path: str) -> None:
|
|||||||
else:
|
else:
|
||||||
party_scores = load_party_axis_scores_for_window(db_path, svd_window)
|
party_scores = load_party_axis_scores_for_window(db_path, svd_window)
|
||||||
|
|
||||||
# For components 3-10, compute MP counts from party_mp_vectors
|
# Compute MP counts from party_mp_vectors
|
||||||
party_mp_counts = (
|
party_mp_counts = (
|
||||||
{p: len(v) for p, v in party_mp_vectors.items()} if party_mp_vectors else {}
|
{p: len(v) for p, v in party_mp_vectors.items()} if party_mp_vectors else {}
|
||||||
)
|
)
|
||||||
|
|
||||||
# Auto-compute flip directions for all components based on party centroids
|
# Auto-compute flip directions for ALL components 1-10 based on party centroids.
|
||||||
# Use the selected window's party_scores for flip direction
|
# Each window's SVD has arbitrary sign orientation, so we compute flip per component
|
||||||
# This ensures right-wing parties consistently appear on the right side for THIS window
|
# to ensure canonical right parties (PVV, FVD, JA21, SGP) appear on the RIGHT.
|
||||||
|
computed_flips: Dict[int, bool] = {}
|
||||||
try:
|
try:
|
||||||
from analysis.svd_labels import compute_flip_direction
|
from analysis.svd_labels import compute_flip_direction
|
||||||
|
|
||||||
for comp in range(1, 11):
|
for comp in range(1, 11):
|
||||||
# Compute flip for the selected window (not current_parliament)
|
computed_flips[comp] = compute_flip_direction(comp, party_scores)
|
||||||
flip = compute_flip_direction(comp, party_scores)
|
|
||||||
if comp in SVD_THEMES:
|
|
||||||
SVD_THEMES[comp]["flip"] = flip
|
|
||||||
except Exception:
|
except Exception:
|
||||||
# If flip computation fails, keep existing flip values in SVD_THEMES
|
# If flip computation fails, keep existing flip values from SVD_THEMES
|
||||||
pass
|
pass
|
||||||
|
|
||||||
# Convert party_scores (possibly [x,y] lists or legacy vectors) into explicit (x,y) coords
|
# Build theme override with computed flip for this component
|
||||||
party_coords: dict = {}
|
# (avoids mutating SVD_THEMES which persists stale values across Streamlit reruns)
|
||||||
for p, v in party_scores.items():
|
theme_with_flip = {
|
||||||
|
**theme,
|
||||||
|
"flip": computed_flips.get(comp_sel, theme.get("flip", False)),
|
||||||
|
}
|
||||||
|
|
||||||
|
# Extract 1D scores for this component (ALL components use raw SVD values)
|
||||||
|
party_1d_coords: dict = {}
|
||||||
|
idx = comp_sel - 1 # Convert to 0-indexed
|
||||||
|
for party, scores in party_scores.items():
|
||||||
try:
|
try:
|
||||||
if v and len(v) >= 2:
|
if scores and len(scores) > idx:
|
||||||
party_coords[p] = (float(v[0]), float(v[1]))
|
party_1d_coords[party] = (float(scores[idx]),)
|
||||||
except Exception:
|
except Exception:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Filter parties by minimum MP count
|
# Filter parties by minimum MP count
|
||||||
if min_mps > 1 and party_mp_counts:
|
if min_mps > 1 and party_mp_counts:
|
||||||
valid_parties = {p for p, count in party_mp_counts.items() if count >= min_mps}
|
valid_parties = {p for p, count in party_mp_counts.items() if count >= min_mps}
|
||||||
party_coords = {
|
party_1d_coords = {
|
||||||
p: coords for p, coords in party_coords.items() if p in valid_parties
|
p: coords for p, coords in party_1d_coords.items() if p in valid_parties
|
||||||
}
|
}
|
||||||
|
|
||||||
# Render party axis chart (single window or time trajectory)
|
# Render party axis chart (single window or time trajectory)
|
||||||
@@ -2675,60 +2615,21 @@ def build_svd_components_tab(db_path: str) -> None:
|
|||||||
has_current = "current_parliament" in available_windows
|
has_current = "current_parliament" in available_windows
|
||||||
all_windows = year_windows + (["current_parliament"] if has_current else [])
|
all_windows = year_windows + (["current_parliament"] if has_current else [])
|
||||||
|
|
||||||
# For components 1-2, use Procrustes-aligned positions (same as single window)
|
# ALL components use raw (non-aligned) SVD vectors.
|
||||||
# For components 3-10, use SVD vectors
|
# Procrustes alignment rotates the full vector space which makes scores
|
||||||
if comp_sel in (1, 2):
|
# incomparable with the single-window view. Per-window flip computation
|
||||||
positions_by_window, _ = load_positions(db_path)
|
# handles orientation alignment for the trajectory.
|
||||||
party_map = load_party_map(db_path)
|
|
||||||
|
|
||||||
# Convert positions to party scores format
|
|
||||||
party_scores_by_window: Dict[str, Dict[str, List[float]]] = {}
|
|
||||||
for window in all_windows:
|
|
||||||
pos = positions_by_window.get(window, {})
|
|
||||||
# Collect all MP positions per party
|
|
||||||
party_positions: Dict[str, List[Tuple[float, float]]] = {}
|
|
||||||
for entity, (x, y) in pos.items():
|
|
||||||
party = party_map.get(entity)
|
|
||||||
if party:
|
|
||||||
party_positions.setdefault(party, []).append((x, y))
|
|
||||||
# Average positions per party
|
|
||||||
party_scores: Dict[str, List[float]] = {}
|
|
||||||
for party, positions in party_positions.items():
|
|
||||||
if positions:
|
|
||||||
avg_x = sum(p[0] for p in positions) / len(positions)
|
|
||||||
avg_y = sum(p[1] for p in positions) / len(positions)
|
|
||||||
party_scores[party] = [avg_x, avg_y]
|
|
||||||
party_scores_by_window[window] = party_scores
|
|
||||||
else:
|
|
||||||
# Use raw (non-aligned) SVD vectors for components 3-10.
|
|
||||||
# Procrustes alignment rotates the full vector space to align
|
|
||||||
# components 1-2 across windows, but this also transforms
|
|
||||||
# components 3-10, making their scores incomparable with the
|
|
||||||
# single-window view. Per-window flip computation handles
|
|
||||||
# orientation alignment for these components.
|
|
||||||
party_scores_by_window = load_party_scores_all_windows(db_path, all_windows)
|
party_scores_by_window = load_party_scores_all_windows(db_path, all_windows)
|
||||||
|
|
||||||
_render_svd_time_trajectory(
|
_render_svd_time_trajectory(
|
||||||
party_scores_by_window,
|
party_scores_by_window,
|
||||||
comp_sel,
|
comp_sel,
|
||||||
theme,
|
theme_with_flip,
|
||||||
selected_parties_for_trajectory,
|
selected_parties_for_trajectory,
|
||||||
)
|
)
|
||||||
elif comp_sel in (1, 2):
|
|
||||||
# Components 1-2 use 2D coords from political compass
|
|
||||||
_render_party_axis_chart(
|
|
||||||
party_coords, comp_sel, theme, bootstrap_data=bootstrap_data
|
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
# Components 3-10 use 1D scores from SVD
|
# Single-window view: render 1D party axis chart
|
||||||
# Extract 1D scores for this component
|
_render_party_axis_chart_1d(party_1d_coords, comp_sel, theme_with_flip)
|
||||||
party_1d_coords = {}
|
|
||||||
idx = comp_sel - 1 # Convert to 0-indexed
|
|
||||||
for party, scores in party_scores.items():
|
|
||||||
if scores and len(scores) > idx:
|
|
||||||
party_1d_coords[party] = (scores[idx],)
|
|
||||||
|
|
||||||
_render_party_axis_chart_1d(party_1d_coords, comp_sel, theme)
|
|
||||||
|
|
||||||
# Batch-fetch motion details (title, date, policy_area, url, body_text, voting_results)
|
# Batch-fetch motion details (title, date, policy_area, url, body_text, voting_results)
|
||||||
motion_ids = [m.get("motion_id") for m in motions if m.get("motion_id") is not None]
|
motion_ids = [m.get("motion_id") for m in motions if m.get("motion_id") is not None]
|
||||||
@@ -2764,9 +2665,9 @@ def build_svd_components_tab(db_path: str) -> None:
|
|||||||
pos_motions = [m for m in motions if float(m.get("score", 0.0)) >= 0]
|
pos_motions = [m for m in motions if float(m.get("score", 0.0)) >= 0]
|
||||||
neg_motions = [m for m in motions if float(m.get("score", 0.0)) < 0]
|
neg_motions = [m for m in motions if float(m.get("score", 0.0)) < 0]
|
||||||
|
|
||||||
flip = theme.get("flip", False) if theme else False
|
flip = theme_with_flip.get("flip", False) if theme_with_flip else False
|
||||||
pos_pole = theme.get("positive_pole", "") if theme else ""
|
pos_pole = theme_with_flip.get("positive_pole", "") if theme_with_flip else ""
|
||||||
neg_pole = theme.get("negative_pole", "") if theme else ""
|
neg_pole = theme_with_flip.get("negative_pole", "") if theme_with_flip else ""
|
||||||
|
|
||||||
# Derive left/right labels from flip direction
|
# Derive left/right labels from flip direction
|
||||||
# flip=True: positive_pole on left, negative_pole on right
|
# flip=True: positive_pole on left, negative_pole on right
|
||||||
|
|||||||
@@ -193,6 +193,43 @@ def check_canonical_right_on_right(
|
|||||||
return divergences
|
return divergences
|
||||||
|
|
||||||
|
|
||||||
|
def check_config_flip_consistency(
|
||||||
|
party_avg_vectors: Dict[str, List[float]],
|
||||||
|
themes: Dict[int, Dict[str, str]],
|
||||||
|
canonical_right: frozenset,
|
||||||
|
canonical_left: frozenset,
|
||||||
|
num_components: int = 10,
|
||||||
|
) -> List[Dict]:
|
||||||
|
"""Check that config flip values match computed flip directions.
|
||||||
|
|
||||||
|
The runtime uses compute_flip_direction() to determine flips, but config
|
||||||
|
also stores a flip value. If they disagree, the config is stale or wrong.
|
||||||
|
"""
|
||||||
|
from analysis.svd_labels import compute_flip_direction
|
||||||
|
|
||||||
|
scores_dict = {
|
||||||
|
p: v
|
||||||
|
for p, v in party_avg_vectors.items()
|
||||||
|
if p in canonical_right or p in canonical_left
|
||||||
|
}
|
||||||
|
|
||||||
|
mismatches = []
|
||||||
|
for comp in range(1, num_components + 1):
|
||||||
|
config_flip = themes.get(comp, {}).get("flip", False)
|
||||||
|
computed_flip = compute_flip_direction(comp, scores_dict)
|
||||||
|
if config_flip != computed_flip:
|
||||||
|
mismatches.append(
|
||||||
|
{
|
||||||
|
"component": comp,
|
||||||
|
"issue": "config_flip_mismatch",
|
||||||
|
"config_flip": config_flip,
|
||||||
|
"computed_flip": computed_flip,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
return mismatches
|
||||||
|
|
||||||
|
|
||||||
def check_theme_consistency(
|
def check_theme_consistency(
|
||||||
party_positions: Dict[str, Dict[int, float]],
|
party_positions: Dict[str, Dict[int, float]],
|
||||||
themes: Dict[int, Dict[str, str]],
|
themes: Dict[int, Dict[str, str]],
|
||||||
@@ -247,13 +284,19 @@ def main() -> int:
|
|||||||
args.components,
|
args.components,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Check 2: Theme pole label consistency
|
# Check 2: Config flip vs computed flip consistency
|
||||||
|
logger.info("Checking config flip vs computed flip consistency")
|
||||||
|
flip_mismatches = check_config_flip_consistency(
|
||||||
|
party_avg_vectors, themes, canonical_right, canonical_left, args.components
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check 3: Theme pole label consistency
|
||||||
logger.info("Checking theme pole label consistency")
|
logger.info("Checking theme pole label consistency")
|
||||||
theme_divergences = check_theme_consistency(
|
theme_divergences = check_theme_consistency(
|
||||||
party_positions, themes, canonical_right, canonical_left
|
party_positions, themes, canonical_right, canonical_left
|
||||||
)
|
)
|
||||||
|
|
||||||
all_divergences = canonical_divergences + theme_divergences
|
all_divergences = canonical_divergences + flip_mismatches + theme_divergences
|
||||||
|
|
||||||
if all_divergences:
|
if all_divergences:
|
||||||
print(f"\n{'=' * 60}")
|
print(f"\n{'=' * 60}")
|
||||||
@@ -282,10 +325,16 @@ def main() -> int:
|
|||||||
print(f" Found right: {d['right_found']}")
|
print(f" Found right: {d['right_found']}")
|
||||||
print(f" Found left: {d['left_found']}")
|
print(f" Found left: {d['left_found']}")
|
||||||
|
|
||||||
|
elif d["issue"] == "config_flip_mismatch":
|
||||||
|
print(f" Config flip: {d['config_flip']}")
|
||||||
|
print(f" Computed flip: {d['computed_flip']}")
|
||||||
|
print(f" → Update SVD_THEMES[{comp}]['flip'] to {d['computed_flip']}")
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
else:
|
else:
|
||||||
print("\n✓ All SVD themes match actual party positions")
|
print("\n✓ All SVD themes match actual party positions")
|
||||||
print(" - Canonical right-wing parties on right side of all axes")
|
print(" - Canonical right-wing parties on right side of all axes")
|
||||||
|
print(" - Config flip values match computed flip directions")
|
||||||
print(" - Theme pole labels consistent with party positions")
|
print(" - Theme pole labels consistent with party positions")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|||||||
@@ -10,8 +10,8 @@ def test_display_label_for_modal():
|
|||||||
y_label = axis_classifier.display_label_for_modal(None, "y")
|
y_label = axis_classifier.display_label_for_modal(None, "y")
|
||||||
|
|
||||||
# Should return component 1 and 2 labels from SVD_THEMES
|
# Should return component 1 and 2 labels from SVD_THEMES
|
||||||
assert "Rechts kabinetsbeleid" in x_label or "links oppositiebeleid" in x_label
|
assert "Rechts versus links" in x_label or "links" in x_label.lower()
|
||||||
assert "PVV/FVD-populisme" in y_label or "mainstream-partijen" in y_label
|
assert "Nationalistisch" in y_label or "kosmopolitisch" in y_label
|
||||||
|
|
||||||
|
|
||||||
def test_display_label_for_modal_maps_as_labels():
|
def test_display_label_for_modal_maps_as_labels():
|
||||||
@@ -20,8 +20,8 @@ def test_display_label_for_modal_maps_as_labels():
|
|||||||
y_label = axis_classifier.display_label_for_modal("As 2", "y")
|
y_label = axis_classifier.display_label_for_modal("As 2", "y")
|
||||||
|
|
||||||
# Should return component 1 and 2 labels
|
# Should return component 1 and 2 labels
|
||||||
assert "Rechts kabinetsbeleid" in x_label or "links oppositiebeleid" in x_label
|
assert "Rechts versus links" in x_label or "links" in x_label.lower()
|
||||||
assert "PVV/FVD-populisme" in y_label or "mainstream-partijen" in y_label
|
assert "Nationalistisch" in y_label or "kosmopolitisch" in y_label
|
||||||
|
|
||||||
|
|
||||||
def test_display_label_for_modal_stempatroon():
|
def test_display_label_for_modal_stempatroon():
|
||||||
@@ -30,8 +30,8 @@ def test_display_label_for_modal_stempatroon():
|
|||||||
y_label = axis_classifier.display_label_for_modal("Stempatroon As 2", "y")
|
y_label = axis_classifier.display_label_for_modal("Stempatroon As 2", "y")
|
||||||
|
|
||||||
# Should return component 1 and 2 labels
|
# Should return component 1 and 2 labels
|
||||||
assert "Rechts kabinetsbeleid" in x_label or "links oppositiebeleid" in x_label
|
assert "Rechts versus links" in x_label or "links" in x_label.lower()
|
||||||
assert "PVV/FVD-populisme" in y_label or "mainstream-partijen" in y_label
|
assert "Nationalistisch" in y_label or "kosmopolitisch" in y_label
|
||||||
|
|
||||||
|
|
||||||
def test_classify_axes_modal_fallback(monkeypatch, tmp_path):
|
def test_classify_axes_modal_fallback(monkeypatch, tmp_path):
|
||||||
@@ -84,10 +84,10 @@ def test_classify_axes_modal_fallback(monkeypatch, tmp_path):
|
|||||||
|
|
||||||
# Should now return SVD component labels instead of hardcoded values
|
# Should now return SVD component labels instead of hardcoded values
|
||||||
assert (
|
assert (
|
||||||
"Rechts kabinetsbeleid" in enriched["x_label"]
|
"Rechts versus links" in enriched["x_label"]
|
||||||
or "links oppositiebeleid" in enriched["x_label"]
|
or "links" in enriched["x_label"].lower()
|
||||||
)
|
)
|
||||||
assert (
|
assert (
|
||||||
"PVV/FVD-populisme" in enriched["y_label"]
|
"Nationalistisch" in enriched["y_label"]
|
||||||
or "mainstream-partijen" in enriched["y_label"]
|
or "kosmopolitisch" in enriched["y_label"]
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -5,7 +5,8 @@ import pytest
|
|||||||
|
|
||||||
|
|
||||||
def test_derive_labels_flip_true():
|
def test_derive_labels_flip_true():
|
||||||
"""When flip=True, positive_pole should be on left, negative_pole on right."""
|
"""Labels should always reflect what's on each side, regardless of flip.
|
||||||
|
negative_pole describes LEFT, positive_pole describes RIGHT."""
|
||||||
theme = {
|
theme = {
|
||||||
"positive_pole": "Right-wing parties",
|
"positive_pole": "Right-wing parties",
|
||||||
"negative_pole": "Left-wing parties",
|
"negative_pole": "Left-wing parties",
|
||||||
@@ -15,17 +16,17 @@ def test_derive_labels_flip_true():
|
|||||||
pos_pole = theme.get("positive_pole", "")
|
pos_pole = theme.get("positive_pole", "")
|
||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
|
|
||||||
if flip:
|
# Fixed logic: labels don't depend on flip
|
||||||
left_pole, right_pole = pos_pole, neg_pole
|
left_pole = neg_pole
|
||||||
else:
|
right_pole = pos_pole
|
||||||
left_pole, right_pole = neg_pole, pos_pole
|
|
||||||
|
|
||||||
assert left_pole == "Right-wing parties"
|
assert left_pole == "Left-wing parties"
|
||||||
assert right_pole == "Left-wing parties"
|
assert right_pole == "Right-wing parties"
|
||||||
|
|
||||||
|
|
||||||
def test_derive_labels_flip_false():
|
def test_derive_labels_flip_false():
|
||||||
"""When flip=False, negative_pole should be on left, positive_pole on right."""
|
"""Labels should always reflect what's on each side, regardless of flip.
|
||||||
|
negative_pole describes LEFT, positive_pole describes RIGHT."""
|
||||||
theme = {
|
theme = {
|
||||||
"positive_pole": "Right-wing parties",
|
"positive_pole": "Right-wing parties",
|
||||||
"negative_pole": "Left-wing parties",
|
"negative_pole": "Left-wing parties",
|
||||||
@@ -35,10 +36,9 @@ def test_derive_labels_flip_false():
|
|||||||
pos_pole = theme.get("positive_pole", "")
|
pos_pole = theme.get("positive_pole", "")
|
||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
|
|
||||||
if flip:
|
# Fixed logic: labels don't depend on flip
|
||||||
left_pole, right_pole = pos_pole, neg_pole
|
left_pole = neg_pole
|
||||||
else:
|
right_pole = pos_pole
|
||||||
left_pole, right_pole = neg_pole, pos_pole
|
|
||||||
|
|
||||||
assert left_pole == "Left-wing parties"
|
assert left_pole == "Left-wing parties"
|
||||||
assert right_pole == "Right-wing parties"
|
assert right_pole == "Right-wing parties"
|
||||||
|
|||||||
@@ -38,11 +38,8 @@ def test_right_wing_on_right_all_components():
|
|||||||
pos_pole = theme.get("positive_pole", "")
|
pos_pole = theme.get("positive_pole", "")
|
||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
|
|
||||||
# Derive left/right labels
|
# Derive left/right labels - labels don't depend on flip
|
||||||
if flip:
|
# negative_pole describes LEFT, positive_pole describes RIGHT
|
||||||
left_label = pos_pole
|
|
||||||
right_label = neg_pole
|
|
||||||
else:
|
|
||||||
left_label = neg_pole
|
left_label = neg_pole
|
||||||
right_label = pos_pole
|
right_label = pos_pole
|
||||||
|
|
||||||
@@ -68,18 +65,19 @@ def test_label_derivation_matches_fallback():
|
|||||||
neg_pole = theme.get("negative_pole", "")
|
neg_pole = theme.get("negative_pole", "")
|
||||||
flip = theme.get("flip", False)
|
flip = theme.get("flip", False)
|
||||||
|
|
||||||
# Simulate the fallback logic from explorer.py lines 969-970
|
# Simulate the fallback logic from explorer.py (fixed version)
|
||||||
expected_left = pos_pole if flip else neg_pole
|
# Labels don't depend on flip - negative_pole describes LEFT, positive_pole describes RIGHT
|
||||||
expected_right = neg_pole if flip else pos_pole
|
expected_left = neg_pole
|
||||||
|
expected_right = pos_pole
|
||||||
|
|
||||||
# Verify theme doesn't have static labels
|
# Verify theme doesn't have static labels
|
||||||
assert "left_pole" not in theme
|
assert "left_pole" not in theme
|
||||||
assert "right_pole" not in theme
|
assert "right_pole" not in theme
|
||||||
|
|
||||||
# The derived labels should match the expected fallback
|
# The derived labels should match the expected fallback
|
||||||
# (This is the core fix - we're now always using the fallback)
|
# Labels don't depend on flip
|
||||||
derived_left = pos_pole if flip else neg_pole
|
derived_left = neg_pole
|
||||||
derived_right = neg_pole if flip else pos_pole
|
derived_right = pos_pole
|
||||||
|
|
||||||
assert derived_left == expected_left, f"Component {comp} left label mismatch"
|
assert derived_left == expected_left, f"Component {comp} left label mismatch"
|
||||||
assert derived_right == expected_right, f"Component {comp} right label mismatch"
|
assert derived_right == expected_right, f"Component {comp} right label mismatch"
|
||||||
|
|||||||
@@ -78,13 +78,13 @@ def test_get_svd_label_returns_correct_label():
|
|||||||
"""Test that get_svd_label returns the correct label for each component."""
|
"""Test that get_svd_label returns the correct label for each component."""
|
||||||
from analysis.svd_labels import get_svd_label
|
from analysis.svd_labels import get_svd_label
|
||||||
|
|
||||||
# Component 1 should return Rechts kabinetsbeleid label
|
# Component 1 should return Rechts versus links label
|
||||||
label1 = get_svd_label(1)
|
label1 = get_svd_label(1)
|
||||||
assert "Rechts kabinetsbeleid" in label1 or "links oppositiebeleid" in label1
|
assert "Rechts versus links" in label1 or "links" in label1.lower()
|
||||||
|
|
||||||
# Component 2 should return PVV/FVD-populisme label
|
# Component 2 should return Nationalistisch versus kosmopolitisch label
|
||||||
label2 = get_svd_label(2)
|
label2 = get_svd_label(2)
|
||||||
assert "PVV/FVD-populisme" in label2 or "mainstream-partijen" in label2
|
assert "Nationalistisch" in label2 or "kosmopolitisch" in label2
|
||||||
|
|
||||||
# Component 3 should return Verzorgingsstaat label
|
# Component 3 should return Verzorgingsstaat label
|
||||||
label3 = get_svd_label(3)
|
label3 = get_svd_label(3)
|
||||||
|
|||||||
Reference in New Issue
Block a user