7 Commits (5ad83ef1beb445a9a500cb6af91deb07cec6df08)

 

All Branches   

Search

Author	SHA1	Message	Date
Sven Geboers	5ad83ef1be	fix(tests): update test_extract_mp_votes for party-level actor inclusion ... New extract_mp_votes behavior inserts all actors (party + individual MPs), not only comma-name MPs. Test now validates both types and their party column. Also adds generated HTML visualizations (political axis x5 windows + trajectories).	1 month ago
Sven Geboers	847b783877	fix(pipeline): fix API pagination, add skip_details fast path, bulk mp_votes insert ... - _get_voting_records returns (records, besluit_meta) tuple; paginate via Besluit?expand=Stemming (469/mo vs 8400) - get_motions(skip_details=True) bypasses per-motion detail chain (3 HTTP calls/motion) - extract_mp_votes rewritten: bulk DataFrame insert (80k rows in 1.9s), includes party-level actors - run_pipeline.py fixed: pass db_path not db, handle dict/int return types - download_past_year.py: skip_details=True default, limit-per-chunk default 50000	1 month ago
Sven Geboers	f2a831dfcf	feat(pipeline): add orchestrator CLI, analysis modules, and ActorFractie ingestion ... - pipeline/run_pipeline.py: CLI orchestrator for all 5 pipeline phases with --dry-run, --skip-*, --window-size, --svd-k, --start/end-date flags - analysis/{political_axis,trajectory,clustering,visualize}.py: PCA/anchor ideological axis, MP drift trajectories, UMAP + KMeans clustering, Plotly HTML output - api_client.py: capture ActorFractie per individual MP vote (comma in ActorNaam) into mp_vote_parties dict on each motion - database.insert_motion: auto-insert mp_votes rows with party affiliation for newly ingested motions when mp_vote_parties is present - Add scikit-learn to pyproject.toml for KMeans clustering - tests/test_run_pipeline.py: window generation, dry-run, skip-all paths - tests/test_analysis.py: PCA axis, anchor axis, trajectory drift, KMeans Ref: thoughts/shared/plans/2026-03-21-parliamentary-embedding-pipeline-plan.md	1 month ago
Sven Geboers	a36e6cba4e	feat(pipeline): implement parliamentary embedding pipeline MVP ... - Add 4 migration files: mp_votes, mp_metadata, svd_vectors, fused_embeddings - Extend database.py with 5 new helper methods and table init - Add pipeline/ package: extract_mp_votes, fetch_mp_metadata, text_pipeline, svd_pipeline (with Procrustes alignment), fusion - Add full test suite (17 tests) covering all pipeline modules and migrations - Fix Procrustes alignment bug: scipy scale is a norm value, not a multiplier - Fix DuckDB date type handling in test assertions (datetime.date vs string) - Remove duckdb.py shim; tests now run against real duckdb + scipy via uv Ref: thoughts/shared/plans/2026-03-21-parliamentary-embedding-pipeline-plan.md	1 month ago
Sven Geboers	c498c3467e	update plan: replace spike with confirmed FractieZetelPersoon fetch task	1 month ago
Sven Geboers	0bbda408fb	plan: parliamentary embedding pipeline MVP implementation plan	1 month ago
Sven Geboers	fd73da3752	design: parliamentary embedding pipeline (late fusion SVD + text)	1 month ago