Commit Graph
296 Commits
Author SHA1 Message Date
sgeboers bb8ce65ec9 overton analysis explorer page push 2026-06-18 22:01:57 +02:00
sgeboers 19e8d5b8ba feat(overton): add category domain decomposition with interactive charts and TDD tests
Populated the right_wing_motions.category column (previously 100% NULL across
3,030 motions) via parallel subagent classification — 80 agents derived a
10-category taxonomy and classified all motions in minutes.

Adds to the Overton QMD report:
- Plotly dropdown filter on Chart 1 to toggle between policy categories
- Chart 7: category delta bar chart (pre/post centrist support per domain)
- Chart 8: quarterly domain trajectories for the 5 largest categories
- Domain Decomposition narrative section

Also fixes a Streamlit tab crash (m.text -> m.body_text) and adds TDD tests.
2026-06-15 21:49:37 +02:00
sgeboers a5624f65bc style: humanize reports and translate Streamlit UI to English
Streamlit tabs: translated all Dutch text to English in overton.py,
compass.py (Overton expander + voting discipline), trajectories.py.

Humanization (all 3 report surfaces):
- Removed 60+ em dashes, replaced with periods/commas/colons
- Removed AI vocabulary (crucial, pivotal, underscores, landscape)
- Removed excessive boldface, rule-of-three, -ing padding
- Replaced 'serves as'/'stands as' with 'is'
- Preserved all data, numbers, tables, code blocks exactly
2026-06-14 23:39:34 +02:00
sgeboers 3192a1a2bf fix(overton): correct 3 critical data errors and 3 high-severity caveats
CRIT-1: Stylistic extremity direction reversed — both dimensions declined
  (stijl 1.875→1.744, not increased +0.097). Holistic moderation, not divergence.
CRIT-2: Masking rate corrected from 36.1% to 9.7% (S≤2, M≥4 on full dataset).
  Original 36.8% was from 117-motion manual audit, not extrapolatable.
CRIT-3: Material impact values harmonized to motion-level means (2.79→2.45).
  Old values (2.78→2.43) used undisclosed mean-of-yearly-means aggregation.
HIGH-1: Migration domain provenance caveat — category column is NULL,
  analysis relies on title keyword matching (approximate boundaries).
HIGH-2: 2026-Q2 bounce caveat — n=44, bimodal distribution, sensitive to
  composition (many consensus defense motions among CS=1.0 items).
HIGH-3: Non-right-wing control group corrected — CS rose 58%→62% (+3.5pp),
  not 'flat at 49%'. Surge was disproportionate for right-wing content.

Also: fixed 6-party centrist definition (line 30) to 4-party,
removed 'did not shift rightward' phrasing, added Phase 4 synthesis
reminder to build_all_reports.py.
2026-06-14 22:45:12 +02:00
sgeboers e00ad6283f fix(overton): update verdict — migration acceptance durable, non-migration temporary
Key finding from 2026 data:
- 2026-Q2 CS bounced back to 0.523 (from 0.334 in Q1)
- Migration CS (0.395) now EXCEEDS non-migration (0.368) for first time
- Multiple 2026-Q2 migration motions got unanimous centrist support (CS=1.00)

Updated verdict across synthesis, QMD, and HTML:
- Non-migration acceptance was a temporary electoral shock response
- Migration acceptance is durable and growing as debate intensifies
- The Overton shift is domain-specific, not uniformly temporary

Also fixed hashline formatting corruption in synthesis file.
2026-06-08 19:42:57 +02:00
sgeboers d05ec40584 fix(overton): address 6 substantive audit findings across all report surfaces
- NSC sensitivity: only 3.1% of CS surge attributable to NSC inclusion
- Submitter parsing: opposition-only d=0.85 likely inflated (~0.65-0.75)
- Mechanism taxonomy: relabeled as exploratory (kappa=0.41, 50.5% agreement)
- Causal claims: softened from 'refuted' to 'less consistent with data'
- 2026 reversion: material moderation persisted but shock was primary driver
- Placeholder contamination: r=0.43 drops to r=0.34 without 6K (1,1) defaults
2026-06-08 19:11:48 +02:00
sgeboers 8826346190 fix(overton): reframe verdict and migration as gateway domain
- Verdict now says 'window widened' (not 'did not shift') — centrist
  support surged for right-wing motions while staying flat for left-wing
- Migration reframed from 'one exception' to 'gateway domain' — where
  acceptance expanded most genuinely and right-wing parties learned
  frames they applied elsewhere
- Explorer Overton tab: added migration gateway section with pre/post
  metrics, full motion text (no truncation), 100-motion browser
- Explorer Kompas tab: updated Overton context to lead with the shift
- Explorer Trajectories tab: Dutch-language Overton annotation
- Synthesis, QMD, HTML report, STATUS.md all updated consistently
2026-06-07 22:29:29 +02:00
sgeboers 5706e86777 feat(overton): coherent narrative architecture — Quarto article, Explorer Overton tab, report cleanup
- U1: Remove stale findings_report.md and blog_post.html, add cross-reference
  headers to all 13 appendix reports, switch HTML report to canonical 4-party
  centrist definition
- U2: Create Quarto narrative spine (overton_window.qmd) with 9 sections and
  6 interactive Plotly charts. Includes 'About Stemwijzer' platform section.
- U3: Add Overton tab to Explorer (centrist support trend, right-wing motion
  browser, explore-further links). Add Overton context expander to Kompas tab
  and 2024 breakpoint annotation to Trajectories tab.
- U4: Create build_all_reports.py master regeneration script (3-phase,
  dependency-ordered, --skip-llm support)
- U5: Update README with Research section, create reports/overton_window/README.md
  reading guide, update STATUS.md with broader platform framing

Plan: docs/plans/2026-06-06-001-overton-coherent-narrative-plan.md
282 tests pass.
2026-06-07 22:02:04 +02:00
sgeboers a3154f72df refactor: extract shared helpers to common.py, fix bugs, add TDD tests
- Created analysis/right_wing/common.py with all shared helpers:
  Constants: CANONICAL_CENTRIST, COALITION, BREAK_YEAR, etc.
  Functions: _conn, cohens_d, build_party_name_map, parse_lead_submitter,
  motion_passed, quarter_sort_key, find_inflection_point

- Fixed bugs:
  1. ai_provider.py: requests.Timeout now caught alongside ConnectionError
  2. voting_margin.py: Removed walrus operator misuse, fixed Mann-Whitney test

- Updated 13 consuming files to import from common.py

- Added 35 TDD tests in tests/right_wing/test_common.py

- 282 tests pass (was 247)
2026-05-31 23:41:29 +02:00
sgeboers 0183bbc8a3 docs(solutions): compound extended Overton analysis methodology 2026-05-31 23:18:50 +02:00
sgeboers 28b24084f6 docs: add Overton analysis to README and AGENTS conventions 2026-05-31 23:14:51 +02:00
sgeboers 364c312076 fix(right-wing): update DB with latest motions, fix DROP TABLE bug, score all missing 2D
- Fetched 276 new motions from Tweede Kamer API (2026-04-23 to 2026-05-31)
- Fixed classify_motions.py: DROP TABLE → CREATE TABLE IF NOT EXISTS
- Restored derived columns (centrist_support_strict, category, etc.) via migration
- Scored 180 missing motions in extremity_scores_2d (now 3,049 total, 0 missing)
- Re-ran temporal trajectory with updated data (inflection: 2024-Q2)
2026-05-31 22:21:35 +02:00
sgeboers 2d5b28fe1b feat(overton): coalition coding fix + regenerate breakpoint analysis 2026-05-31 20:11:03 +02:00
sgeboers 7ff3fec992 fix(blog): correct centrist definition, contextualize metrics, add verdict at end 2026-05-31 20:10:14 +02:00
sgeboers f8aca6be66 docs(blog): Add Overton window analysis blog post
578-line HTML blog post with CSS-only charts, metrics, timeline,
and full narrative covering all findings from the analysis.
2026-05-31 20:00:14 +02:00
sgeboers d34d43a888 feat(overton): improvements and extensions — party differentiation, voting margin, SVD viz, mechanism validation, predictive model
U1: JA21 drives moderation effect (+0.203 CS shift, only party with volume+support gains)
U2: Coalition coding split at July 2024 — opposition effect confirmed (d=0.85 vs 0.87)
U3: Voting margin (ρ=0.812 with centrist support) is far superior to pass rate
U4: SVD trajectory confirms spatial divergence — centrists moved left (Δx=-0.30), right stationary
U5: Mechanism classification Cohen's κ=0.41 (moderate) — taxonomy needs revision
U6: Predictive model AUC-ROC=0.81 — submitter party and category are strongest predictors
2026-05-31 19:41:22 +02:00
sgeboers 7df961ba83 feat(overton): address 7 critical gaps in Overton window analysis
U1: Temporal trajectory — quarterly granularity reveals immediate
electoral jump at 2024-Q1 (+0.180), peak at 2024-Q4 (0.648), reversion
to 0.334 by 2026-Q1.

U2: 2D extremity temporal — single-score masks divergence. Material
impact decreased (-0.146) while stylistic increased (+0.097).
Wilcoxon p=0.002 confirms systematic divergence.

U3: Systematic mechanism classification — 150 motions. Consensus
framing confirmed (24% high-CS vs 8% low-CS, p=0.014). Post-2024
high-CS dominated by procedural (32%), consensus (24%), targeted
restriction (17%).

U4: Causal timing — shift is electorally driven (after Nov 2023 PVV
election, before Jul 2024 Schoof cabinet). Rules out coalition
dynamics, gradual learning, European contagion.

U5: Left-wing response — barely changed (21.3%→20.2%, -1.1pp).
Centrist shift (d=+1.89) is 18.3x larger than left hardening
(d=-0.75). Volt is only left party that softened (+12.9pp).

U6: Success correlation — significant trend (p<0.001) but success
premium only +3.2%, ceiling effect at 96%+ limits practical meaning.

U7: Synthesis update — integrated all findings, updated verdict
to note electoral-cycle effect and 2026-Q1 reversion.
2026-05-26 23:33:13 +02:00
sgeboers ff7665e86c docs(solutions): domain decomposition reveals hidden Overton variance
Aggregate centrist support surge masks two distinct mechanisms:
strategic moderation (85%) and migration acceptance expansion (15%).
Party-level: CDA+ChristenUnie drive the shift, D66 barely moves.
Compounding note for future Overton/domain analysis work.
2026-05-25 01:10:53 +02:00
sgeboers 1e06c46bd9 docs(overton): add domain decomposition, MP-level granularity, anti-institutional analysis
- Non-migration: strategic moderation (impact down, volume up)
- Migration: acceptance expansion (M=5 support 0→19%, impact stable)
- MP-level: CDA+ChristenUnie drive shift, D66 barely moves
- Anti-institutional: abolition→contestation (nexit=0, judiciary up)
- Updated verdict with migration as sole acceptance exception
2026-05-25 01:10:03 +02:00
sgeboers 23aa70133f docs(overton): revise verdict — window did NOT shift right, right-wing moderated
Material impact declined post-2024 (2.78→2.43, M>=4 share 23.7%→11.3%).
Right-wing strategic moderation: more motions, milder content, better
framing. The Overton window did not expand — right-wing proposals
shifted into the existing window. 'Acceptance through moderation'
replaces 'acceptance without conversion.'
2026-05-25 00:40:19 +02:00
sgeboers cea1468f15 docs: caveat SVD spatial positions reflect voting patterns, not policy content
SVD axes capture agreement structure — centrists 'moving left' means
voting patterns diverged from right-wing, not that parties changed
ideology. 'Acceptance without conversion' is a behavioral claim.
Documented as best-practice learning.
2026-05-25 00:37:34 +02:00
sgeboers eada678c0c docs(overton): Add Overton window shift synthesis report
Three-indicator verdict: window widened (acceptance without conversion)
- Centrist support: 0.251→0.507, d=+0.65 (d=+0.85 opposition-only)
- SVD: centrists moved LEFT spatially while voting more with right
- 2D extremity: r=0.47, material 0.83 above stylistic (2,869 motions)
- Mechanisms: consensus framing (33%), institutional (21%), welfare (17%)
2026-05-25 00:34:24 +02:00
sgeboers 80c68c0112 fix(right-wing): match store_scores column names and value order to DB schema 2026-05-25 00:01:31 +02:00
sgeboers 84ec44e468 docs(report): add mechanism analysis findings
- Classified 24 post-2024 right-wing motions with CS>=0.5
- Dominant mechanisms: consensus framing (33%), institutional (21%), welfare (17%)
- Only 1 targeted restriction, zero system dismantling
- Right-wing gains centrist support through repackaging, not conversion
- Confirms acceptance-without-conversion dynamic at the mechanism level
2026-05-24 23:29:14 +02:00
sgeboers 91325aa1f7 docs(report): document preliminary 2D extremity findings
- Pearson r=0.45 between stylistic and material impact (separable)
- Material impact averages 0.85 points above stylistic
- 36.8% of motions mask high-impact policy behind restrained language
- Original single-score conflates language vs substance
- Mark U4 mechanism analysis as in progress
2026-05-24 23:22:58 +02:00
sgeboers b6612d834a docs(plan): subagent-based two-dimensional extremity rescoring plan 2026-05-24 23:13:54 +02:00
sgeboers bf37f84a8b feat(extremity): two-dimensional rescoring with subagent pipeline
- Project-local skill .opencode/skills/score-extremity/ for subagent dispatch
- Orchestrator extremity_rescore_2d.py with load_skill/sample/format/validate/store
- 16 TDD tests covering all orchestrator functions
- 117 motions scored by deepseek v4 flash subagents (12 parallel batches)
- Pearson r=0.45 between stylistic and material dimensions — separable
- Key finding: 36.8% of motions use restrained language for consequential policies
- 2d_extremity_correlation_report.md documents distribution, divergence patterns,
  and implications for the Overton acceptance-without-conversion narrative
2026-05-24 23:13:42 +02:00
sgeboers 10fc002ef9 feat(skill): add score-extremity project-local skill
Two-dimensional scoring via subagents:
- Stijl-extremiteit: stylistic/inflammatory language (1-5)
- Materiele impact: substantive rights/policy impact (1-5)
Defines prompt template, output JSON schema, and batch subagent workflow.
2026-05-24 22:52:50 +02:00
sgeboers be007165b1 fix(right-wing): add resume support to extremity and sentiment scorers
Use CREATE TABLE IF NOT EXISTS and skip already-scored motions
to allow resuming interrupted batch runs.
2026-05-24 22:33:44 +02:00
sgeboers ec18fe0540 chore: remove reports/ from .gitignore, delete stray backfill.py
reports/ contains tracked analysis deliverables (drift reports, Overton
window charts) that were being force-added. Removing the ignore rule
matches project usage.
2026-05-24 22:20:52 +02:00
sgeboers 711a410df3 chore: simplify Overton scripts, update README, add stemwijzer.db to gitignore
- Extracted EXTREMITY_BUCKET_ORDER constant and _extremity_bucket() helper (4 duplications removed)
- Merged two-pass query loop in compute_yearly_baseline into single pass
- Removed unused import (mticker), dead code (year_titles_map), 12 obvious comments
- Extracted _fmt_axis() helper in SVD drift script
- Updated README analysis/ description to include right-wing motion analysis
2026-05-24 22:19:21 +02:00
sgeboers 7b5f97e177 docs(solutions): document Overton window shift analysis methodology
Captures the 7-step methodology developed through multiple analysis
iterations: strict centrist definition, Procrustes-aligned SVD with
anchor-party sign validation, centrist support fraction over pass rate,
acceptance-without-conversion test, opposition-only filter as coalition
control, extremity-stratified tolerance analysis, and LLM score auditing.
2026-05-24 22:11:56 +02:00
sgeboers 2a081ade25 fix(overton): strict centrist definition + left support analysis
- Reclassified centrist to {D66, CDA, CU, NSC} — removing VVD/BBB
  which are center-right coalition partners
- Added centrist_support_strict (0.251→0.507, d=+0.65), center_right_support,
  and left_support_mp columns via migration script
- Figure 1 now shows center-right (VVD/BBB) support as orange dashed line
- New Figure 3: bar chart of left-party support for right-wing motions
  (0.268→0.202, left opposition hardened)
- New report Section 6 covering left-wing support trends
- All analysis now uses strict centrist definition throughout
2026-05-09 00:45:38 +02:00
sgeboers e478235c84 fix(overton): correct SVD axis interpretation, drop pass rate, synthesis rewrite
- SVD axis 2 sign corrected: negative = nationalist (PVV -0.56, FVD -0.36), positive = kosmopolitisch (Volt +0.27). Centrists moved LEFT on both axes while right-wing moved further right culturally (+0.146 gap). 'Acceptance without conversion' named as unifying interpretation.
- U1: Figure 1 merged to single panel, pass rate removed, 5 centrist_support lines
- U2: Pass rate columns dropped from all breakpoint tables, PR narrative cut
- U3: Findings report rewritten: SVD section replaced, synthesis restructured into 3 tiers, extremity LLM bias qualified
- U4: Axis labels and sign convention added to svd_stability_report.md
- Added centrist_support_mp column (MP-weighted, correlates 0.998 with party-level)
2026-05-09 00:21:46 +02:00
sgeboers 76b499cdc0 feat(analysis): Overton window breakpoint analysis with opposition control and SVD drift
Quantify 2024 breakpoint in centrist support (d=+0.68 overall, d=+0.85 opposition-only),
domain decomposition, extremity-stratified pass rates, and manual LLM audit (75% agreement).
SVD center drift aborted due to axis instability (9/10 consecutive window pairs fail stability threshold).
2026-05-08 23:14:34 +02:00
sgeboers d170444bda feat(analysis): add migration anti-democratic overlap analysis 2026-05-08 22:56:38 +02:00
sgeboers fbf92c82cf feat(right-wing): dual-scoring extremity/sentiment + derived categories
Extremity Scorer (U4 enhanced):
- Now scores BOTH original motion text AND layman explanation separately
- Schema: text_score, text_explanation, layman_score, layman_explanation
- Text scores: 1→7, 2→33, 3→5, 4→5 (mild-to-moderate)
- Layman scores: 1→12, 2→20, 3→17, 4→1 (slightly milder)

Sentiment Analysis (U5 enhanced):
- Now scores BOTH original motion text AND layman explanation separately
- Schema: text_score, text_explanation, layman_score, layman_explanation
- Text sentiment avg: 0.294 (slightly positive)
- Layman sentiment avg: 0.416 (more positive - summaries tone down hostility)

Category Derivation (new):
- Two-phase LLM approach: derive taxonomy from sample, then apply to all
- Discovered 7 categories from 30-motion sample:
  veiligheid/justitie, corona/pandemie, economie/belasting, klimaat/milieu,
  defensie/buitenland, asiel/vreemdelingen, overig
- Applied to 50 motions with distribution shown in DB
- Adds category + category_explanation columns to right_wing_motions
2026-05-05 21:40:58 +02:00
sgeboers f94edc3d04 feat(right-wing): sentiment analysis pipeline for right-wing motions
Implements U5: sentiment_analysis.py uses LLM batch calls (fallback when no
local Dutch sentiment model is available) to score motion sentiment on [-1, 1]
scale.

Design:
- Prompt asks for sentiment from -1 (hostile/aggressive) to 1 (constructive)
- JSON schema enforces numeric score + Dutch explanation
- Batch size 10, max_workers 5 for parallel API calls
- Stores results in  table
- Updates  with avg_sentiment, sentiment_std,
  pct_strongly_negative per year

Sample validation (50 motions): good variance across [-0.9, 1.0] range.
2026-05-05 21:25:42 +02:00
sgeboers d2310edfc4 feat(right-wing): LLM-based policy extremity scoring
Implements U4: extremity_scorer.py uses ai_provider.chat_completion_json_parallel
with a JSON schema enforcing integer 1-5 + Dutch explanation.

Design:
- Batch size 10, max_workers 5 for parallel API calls
- Prompt asks for concrete policy + radicalism score in Dutch
- Stores results in  table (motion_id, score, explanation, error)
- Updates  with yearly averages
- Default sample=50 for validation; --sample -1 scores all motions

Sample validation (50 motions): scores distributed 1→2, 2→34, 3→7, 4→7,
yearly averages ~2.0-2.5 (mild-to-moderate radicalism).
2026-05-05 21:23:08 +02:00
sgeboers 1bc83c4384 feat(right-wing): temporal aggregation of right-wing motion trends
Implements U3: temporal_analysis.py computes yearly_summary from the
right_wing_motions table (U2 output).

Metrics per year:
- total_right_wing, pct_of_total, total_motions
- avg_right_support, avg_left_opposition, centrist_support
- avg_right_keyword_matches, extremity_index (U4 placeholder)
- yoy_right_wing_delta, yoy_pct_delta

Key finding: right-wing motions grew from ~4% (2018) to ~12% (2024-2025)
of all motions, with rising centrist support over time.
2026-05-05 21:20:12 +02:00
sgeboers d3dfb0ce2f feat(right-wing): hybrid motion classifier using keywords + votes
Implements U2: classify_motions.py loads keywords from U1 and classifies
motions as right-wing when:
- right_support >= 60% (CANONICAL_RIGHT parties voting 'voor')
- left_opposition >= 40% (CANONICAL_LEFT parties voting 'tegen')
- AND at least 1 right-wing keyword match in title/body_text

Outputs DuckDB table  with:
- motion_id, year, title, right_support, left_opposition, centrist_support
- right_keyword_matches, left_keyword_matches, classified flag

Classified 2986 of 28331 motions (10.5%) as right-wing.
2026-05-05 21:18:38 +02:00
sgeboers c6f8540671 feat(right-wing): derive right-wing keywords via differential TF-IDF
Implements U1: derive_keywords.py uses party voting patterns to classify
motions as right-wing vs left-wing, then computes differential TF-IDF on
cleaned motion titles to surface policy terms distinctive to right-wing
motions.

Key design choices:
- Vote threshold: 60% of parties in group must vote 'voor'
- Text cleaning strips motion prefixes aggressively (handles multi-word
  surnames, plural 'leden', t.v.v. parentheticals)
- Expanded Dutch stopword list filters procedural and generic noise
- Results written to analysis/right_wing/right_wing_keywords.json

Produces ~50 filtered terms including: asielzoekers, defensie, kernenergie,
boeren, vreemdelingenbeleid, stikstof, asielstop, strafrecht.
2026-05-05 21:14:11 +02:00
sgeboers 3a46485067 added ansible again 2026-05-04 22:05:06 +02:00
sgeboers 272d839a42 feat: agent-native refactor, SVD consistency fixes, UX cleanup, mobile support
- Refactor agent_tools to atomic primitives (24 tools, delete workflows)
- Fix SVD component score inconsistency between single-window and trajectory views
  (same PCA basis, same flip handling, same active-MP filter for current_parliament)
- Fix Dutch spelling: Huidig parliament -> Huidig parlement
- Remove all decorative emojis from UI (app.py, explorer.py, analysis tabs)
- Add dark theme matching sgeboers.nl (mint accent on dark background)
- Remove browser tab favicon and Streamlit chrome (deploy button, running status)
- Remove trajectories debug UI and EMA settings (hardcoded smooth_alpha=0.35)
- Switch layout to centered for mobile readability
- Add responsive CSS for mobile (touch targets, font sizing, overflow prevention)
- Update AGENTS.md and SYSTEM_PROMPT.md with active tool instructions
- Add compound docs for SVD consistency bug
- Update tests: 214 passed, 3 skipped
2026-05-04 21:56:40 +02:00
sgeboers efb3a8fbd2 fix: agent-native audit — parameterize thresholds, add CRUD tests, tool discovery
Audit fixes for agent-native architecture gaps:

- agent_tools/content.py: parameterize healthy_threshold in check_embedding_quality
- agent_tools/__init__.py: add __all__ exports and list_tools() runtime discovery
- agent_tools/database.py: add CRUD primitives (create_motion, update_motion, delete_report)
  plus query_embeddings, query_similar_motions, query_compass_positions
- tests/agent_tools/test_database_tools.py: add CRUD tool tests
- tests/agent_tools/test_content_tools.py: add parameterized threshold test
- tests/agent_tools/test_package.py: test list_tools() and package imports

Tests: 245 passed, 3 skipped
2026-05-04 20:05:59 +02:00
sgeboers 8af27bbf04 feat: implement agent-native architecture (U1-U6)
Implements the agent-native architecture plan (docs/plans/2026-05-01-002-agent-native-architecture-plan.md):

- U1: Database query primitives (agent_tools/database.py)
  - query_motions, query_votes, query_svd_vectors, query_party_positions, query_pipeline_status
- U2: Pipeline control primitives (agent_tools/pipeline.py)
  - pipeline_run_stage, pipeline_run_full, pipeline_check_health, pipeline_get_logs, pipeline_validate_output
- U3: Analysis & report generation (agent_tools/analysis.py, reports.py)
  - analyze_party_shift, analyze_axis_stability, validate_svd_labels, generate_report
- U4: Content validation primitives (agent_tools/content.py)
  - validate_motion_coverage, validate_layman_explanations, suggest_svd_label, check_embedding_quality
- U5: System prompt & context injection (SYSTEM_PROMPT.md, context.py, context.md)
- U6: Parity verification tests (tests/agent_tools/test_parity.py)

Tests: 238 passed, 2 skipped
AGENTS.md updated to surface agent_tools/
2026-05-04 19:38:01 +02:00
sgeboers 98358344a0 docs: add STRATEGY.md with product strategy
Captures target problem, approach, primary persona, key metrics,
tracks of work, and explicit non-goals for the Stemwijzer project.
2026-05-04 19:16:50 +02:00
sgeboers a634ceba2d cleanup: remove Docker, Ansible, and deployment infrastructure
Removes unused deployment and packaging infrastructure:
- Dockerfile and docker-compose.yml (Docker deployment not used)
- ansible/ directory (playbooks, inventory, config)
- packages/@ansible/example/ (npm package for Ansible example)
- docs/deployment/ansible-package-deploy.md
- docs/plans/2026-04-24-002-fix-docker-compose-scheduler-plan.md (obsolete)
- .github/workflows/publish-ansible-example.yml
- .github/workflows/ci-node-packages.yml (only tested packages/)

Updates:
- README.md: remove Deployment section
- docs/plans/2026-04-24-ROADMAP-stemwijzer-improvements.md: mark P1-002 as removed, update sprint 1
2026-05-04 19:11:37 +02:00
sgeboers 1f053f7d91 refactor: simplify Explorer to 3 focused tabs — compass, trajectories, SVD
Removes embedding-heavy search and generic browser tabs from the Explorer.
The project does not currently use embeddings meaningfully, so the similarity
search and browser features were dead weight.

Changes:
- explorer.py: Remove search and browser tabs, keep compass/trajectories/SVD
- explorer.py: Remove fused_embeddings and similarity_cache stats from sidebar
- Home.py: Update Explorer description to match new focused layout
- analysis/tabs/__init__.py: Remove search and browser exports
- tests: Update decomposition and import tests for new tab set

Result: Explorer now has 3 focused analytical tabs instead of 5.
2026-05-01 14:46:52 +02:00
sgeboers 2c60f41f29 cleanup: archive stale scripts and delete orphaned generate_extra_charts
Archives 8 one-off/backfill/research scripts to scripts/archive/:
- compare_svd_exclude_parties.py (diagnostic)
- compute_test_batch.py (test utility)
- fill_mp_votes_parties.py (backfill)
- generate_compass.py (generates to deleted outputs/)
- inspect_axis.py (diagnostic)
- qa_similarity.py (QA script, references deleted thoughts/ledgers/)
- recompute_svd.py (one-off recompute)
- semantic_gravity_examples.py (research)

Deletes:
- generate_extra_charts.py (0 references, generates to deleted outputs/)
- tests/test_qa_similarity.py (test for archived script)

Adds:
- scripts/archive/README.md explaining archive purpose
- docs/plans/2026-05-01-001-scripts-audit-cleanup-plan.md
2026-05-01 12:22:55 +02:00