feat(pipeline): implement parliamentary embedding pipeline MVP

- Add 4 migration files: mp_votes, mp_metadata, svd_vectors, fused_embeddings
- Extend database.py with 5 new helper methods and table init
- Add pipeline/ package: extract_mp_votes, fetch_mp_metadata, text_pipeline,
  svd_pipeline (with Procrustes alignment), fusion
- Add full test suite (17 tests) covering all pipeline modules and migrations
- Fix Procrustes alignment bug: scipy scale is a norm value, not a multiplier
- Fix DuckDB date type handling in test assertions (datetime.date vs string)
- Remove duckdb.py shim; tests now run against real duckdb + scipy via uv

Ref: thoughts/shared/plans/2026-03-21-parliamentary-embedding-pipeline-plan.md
This commit is contained in:
2026-03-21 22:31:22 +01:00
parent c498c3467e
commit a36e6cba4e
68 changed files with 6822 additions and 0 deletions
+22
View File
@@ -0,0 +1,22 @@
import json
from src.types.motion_types import SimilarityNeighbor, to_json, from_json
def test_similarity_neighbor_json_roundtrip():
neighbors = [
SimilarityNeighbor(motion_id="m1", score=0.9),
SimilarityNeighbor(motion_id="m2", score=0.75),
]
# Serialize to JSON string
json_str = to_json(neighbors)
assert isinstance(json_str, str)
# Ensure it's valid JSON
parsed = json.loads(json_str)
assert isinstance(parsed, list)
# Deserialize back to objects
recovered = from_json(json_str)
assert recovered == neighbors