feat(pipeline): implement parliamentary embedding pipeline MVP

- Add 4 migration files: mp_votes, mp_metadata, svd_vectors, fused_embeddings
- Extend database.py with 5 new helper methods and table init
- Add pipeline/ package: extract_mp_votes, fetch_mp_metadata, text_pipeline,
  svd_pipeline (with Procrustes alignment), fusion
- Add full test suite (17 tests) covering all pipeline modules and migrations
- Fix Procrustes alignment bug: scipy scale is a norm value, not a multiplier
- Fix DuckDB date type handling in test assertions (datetime.date vs string)
- Remove duckdb.py shim; tests now run against real duckdb + scipy via uv

Ref: thoughts/shared/plans/2026-03-21-parliamentary-embedding-pipeline-plan.md
This commit is contained in:
2026-03-21 22:31:22 +01:00
parent c498c3467e
commit a36e6cba4e
68 changed files with 6822 additions and 0 deletions
+1
View File
@@ -0,0 +1 @@
"""Fixtures package for tests."""
+40
View File
@@ -0,0 +1,40 @@
[
{
"motion_id": 1,
"date": "2024-01-15",
"voting_results": {
"VVD": "voor",
"PvdA": "tegen",
"CDA": "voor",
"D66": "voor",
"Wilders, G.": "voor",
"Yesilgöz-Zegerius, D.": "voor",
"Jetten, R.A.A.": "voor"
}
},
{
"motion_id": 2,
"date": "2024-02-10",
"voting_results": {
"VVD": "tegen",
"PvdA": "voor",
"CDA": "afwezig",
"D66": "voor",
"Wilders, G.": "tegen",
"Yesilgöz-Zegerius, D.": "tegen",
"Ploumen, L.J.": "voor"
}
},
{
"motion_id": 3,
"date": "2024-03-05",
"voting_results": {
"VVD": "voor",
"SP": "tegen",
"GroenLinks": "voor",
"PVV": "voor",
"Van der Plas, C.": "voor",
"Klever, N.C.": "voor"
}
}
]