Commit Graph
20 Commits
Author SHA1 Message Date
sgeboers 19e8d5b8ba feat(overton): add category domain decomposition with interactive charts and TDD tests
Populated the right_wing_motions.category column (previously 100% NULL across
3,030 motions) via parallel subagent classification — 80 agents derived a
10-category taxonomy and classified all motions in minutes.

Adds to the Overton QMD report:
- Plotly dropdown filter on Chart 1 to toggle between policy categories
- Chart 7: category delta bar chart (pre/post centrist support per domain)
- Chart 8: quarterly domain trajectories for the 5 largest categories
- Domain Decomposition narrative section

Also fixes a Streamlit tab crash (m.text -> m.body_text) and adds TDD tests.
2026-06-15 21:49:37 +02:00
sgeboers 5706e86777 feat(overton): coherent narrative architecture — Quarto article, Explorer Overton tab, report cleanup
- U1: Remove stale findings_report.md and blog_post.html, add cross-reference
  headers to all 13 appendix reports, switch HTML report to canonical 4-party
  centrist definition
- U2: Create Quarto narrative spine (overton_window.qmd) with 9 sections and
  6 interactive Plotly charts. Includes 'About Stemwijzer' platform section.
- U3: Add Overton tab to Explorer (centrist support trend, right-wing motion
  browser, explore-further links). Add Overton context expander to Kompas tab
  and 2024 breakpoint annotation to Trajectories tab.
- U4: Create build_all_reports.py master regeneration script (3-phase,
  dependency-ordered, --skip-llm support)
- U5: Update README with Research section, create reports/overton_window/README.md
  reading guide, update STATUS.md with broader platform framing

Plan: docs/plans/2026-06-06-001-overton-coherent-narrative-plan.md
282 tests pass.
2026-06-07 22:02:04 +02:00
sgeboers 0183bbc8a3 docs(solutions): compound extended Overton analysis methodology 2026-05-31 23:18:50 +02:00
sgeboers ff7665e86c docs(solutions): domain decomposition reveals hidden Overton variance
Aggregate centrist support surge masks two distinct mechanisms:
strategic moderation (85%) and migration acceptance expansion (15%).
Party-level: CDA+ChristenUnie drive the shift, D66 barely moves.
Compounding note for future Overton/domain analysis work.
2026-05-25 01:10:53 +02:00
sgeboers cea1468f15 docs: caveat SVD spatial positions reflect voting patterns, not policy content
SVD axes capture agreement structure — centrists 'moving left' means
voting patterns diverged from right-wing, not that parties changed
ideology. 'Acceptance without conversion' is a behavioral claim.
Documented as best-practice learning.
2026-05-25 00:37:34 +02:00
sgeboers 711a410df3 chore: simplify Overton scripts, update README, add stemwijzer.db to gitignore
- Extracted EXTREMITY_BUCKET_ORDER constant and _extremity_bucket() helper (4 duplications removed)
- Merged two-pass query loop in compute_yearly_baseline into single pass
- Removed unused import (mticker), dead code (year_titles_map), 12 obvious comments
- Extracted _fmt_axis() helper in SVD drift script
- Updated README analysis/ description to include right-wing motion analysis
2026-05-24 22:19:21 +02:00
sgeboers 7b5f97e177 docs(solutions): document Overton window shift analysis methodology
Captures the 7-step methodology developed through multiple analysis
iterations: strict centrist definition, Procrustes-aligned SVD with
anchor-party sign validation, centrist support fraction over pass rate,
acceptance-without-conversion test, opposition-only filter as coalition
control, extremity-stratified tolerance analysis, and LLM score auditing.
2026-05-24 22:11:56 +02:00
sgeboers 272d839a42 feat: agent-native refactor, SVD consistency fixes, UX cleanup, mobile support
- Refactor agent_tools to atomic primitives (24 tools, delete workflows)
- Fix SVD component score inconsistency between single-window and trajectory views
  (same PCA basis, same flip handling, same active-MP filter for current_parliament)
- Fix Dutch spelling: Huidig parliament -> Huidig parlement
- Remove all decorative emojis from UI (app.py, explorer.py, analysis tabs)
- Add dark theme matching sgeboers.nl (mint accent on dark background)
- Remove browser tab favicon and Streamlit chrome (deploy button, running status)
- Remove trajectories debug UI and EMA settings (hardcoded smooth_alpha=0.35)
- Switch layout to centered for mobile readability
- Add responsive CSS for mobile (touch targets, font sizing, overflow prevention)
- Update AGENTS.md and SYSTEM_PROMPT.md with active tool instructions
- Add compound docs for SVD consistency bug
- Update tests: 214 passed, 3 skipped
2026-05-04 21:56:40 +02:00
sgeboers 09bb99658f docs: compound code review findings
- Add verify-lint-rule-scope-before-relying-on-it: guidance on
  confirming lint rule coverage before trusting it for enforcement.
  Documents the P2-002 incident where ruff BLE only catches bare
   not .
- Update working-tree-hygiene: add dev-tool-in-venv check and
  ruff dependency example.
2026-05-01 01:23:49 +02:00
sgeboers c85a367a8e docs: add improvement roadmap, research notes, and solution docs
- Add 2026-04-24 ROADMAP with 5 phases / 17 items
- Add detailed implementation plans for P1-001 through P4-005
- Add research artifacts and solution docs from ledger merge
- Add test for SVD component 1 compass alignment
2026-04-30 23:55:24 +02:00
sgeboers 375955dbc4 cleanup: merge session ledgers into docs/solutions and delete artifacts
- Remove stale thoughts/ledgers/ and thoughts/shared/ artifacts
- Fix .gitignore duplicate .worktrees entry
- Move pyright to [dependency-groups] dev
- Replace hardcoded blog correlation with reproducible metric reference
- Add docs: verify-session-artifacts, fusion-vector-dimensions,
  working-tree-hygiene
- Update blog-numbers-from-pipeline-outputs with correlation example
2026-04-30 23:24:43 +02:00
sgeboers be4375b303 docs(solutions): document best practice for deriving blog numbers from pipeline outputs 2026-04-16 18:52:53 +02:00
sgeboers cf549dcc1c feat(svd): update 8 of 10 axis labels derived from motion content
Revise SVD_THEMES labels based on TF-IDF analysis of top 50 motions
per component (pool size: current_parliament). Manual review of motion
titles ensures labels reflect actual parliamentary content rather than
party position semantics.

Key corrections:
- Axis 1: fiscal/economic policy vs social welfare + international rights
- Axis 4: active international engagement vs restraint
- Axis 5: pragmatic financial support vs progressive individual rights
- Axis 6: fossil fuels/financial incentives vs climate/intl rights
- Axis 7: practical-administrative vs idealistico-procedural (kept)
- Axis 8: European defense cooperation vs domestic socioeconomic policy
- Axis 9: concrete-administrative vs systemic reform
- Axis 10: citizen protection vs government regulation

Subagent analysis caught that axes 5 and 6 are NOT the same
(Nationale soevereiniteit) — manual motion review confirms distinct
content for each. Axes 1, 5, 6 had completely wrong labels.

Refs: thoughts/explorer/svd_label_review.md
See also: docs/brainstorms/2026-04-13-topic-derived-svd-labels-requirements.md
2026-04-13 23:59:50 +02:00
sgeboers 67eda93cb1 docs: add overtone shift analysis and insights
- Add deep dive analysis on SVD axis overtone shift (docs/research/)
- Update brainstorm with results documenting key finding: stability and
  overtone shift are independent phenomena
- Add learning doc about axis stability vs semantic drift independence
- Update drift report with detailed findings
2026-04-05 19:40:23 +02:00
sgeboers bad9cd758d docs: add SVD theme divergence solution doc and validation hook 2026-04-05 09:58:53 +02:00
sgeboers 27486726b8 docs: add learning about agent push access to gitea 2026-04-05 00:53:07 +02:00
sgeboers 414c16ae9e refactor: extract data loading and trajectory logic from explorer.py
- Move trajectory analysis to analysis/trajectory.py (+136 lines)
- Move projection helpers to analysis/projections.py (+128 lines)
- Extract tab-specific data loaders to analysis/tabs/ (8 modules, +133 lines)
- Remove 702 lines from explorer.py (data loading extracted to
  analysis/explorer_data.py and new modules)
- Add axis label fallback tests (tests/test_axis_label_fallback.py)
- Add session docs: brainstorms, ideation, plans, and test-failures
2026-04-05 00:51:30 +02:00
sgeboers 154762a4c8 docs: refresh 2 stale references in solutions docs
- refactoring-streamlit-data-loading.md: update test count
  164/164 → 173/173 (7 new axis validation tests added)
- svd-component-labels-mismatch.md: SVD_THEMES moved from
  explorer.py:434-611 → analysis/config.py:67+ per the
  refactoring that extracted constants to analysis/config.py
2026-04-05 00:46:57 +02:00
sgeboers 805cc7e284 docs: add session ledger files and logic-errors docs 2026-04-04 18:47:05 +02:00
sgeboers 0308d20f12 docs: add AGENTS.md with docs/solutions reference and SVD label best practice
- Add AGENTS.md with documented solutions reference
- Include SVD label convention (right-wing parties on right side)
- Document SVD insight: labels reflect voting patterns, not semantics
- Fix SQL verification example to use Python approach
2026-04-04 18:36:05 +02:00