feat(pipeline): implement parliamentary embedding pipeline MVP

- Add 4 migration files: mp_votes, mp_metadata, svd_vectors, fused_embeddings
- Extend database.py with 5 new helper methods and table init
- Add pipeline/ package: extract_mp_votes, fetch_mp_metadata, text_pipeline,
  svd_pipeline (with Procrustes alignment), fusion
- Add full test suite (17 tests) covering all pipeline modules and migrations
- Fix Procrustes alignment bug: scipy scale is a norm value, not a multiplier
- Fix DuckDB date type handling in test assertions (datetime.date vs string)
- Remove duckdb.py shim; tests now run against real duckdb + scipy via uv

Ref: thoughts/shared/plans/2026-03-21-parliamentary-embedding-pipeline-plan.md
This commit is contained in:
2026-03-21 22:31:22 +01:00
parent c498c3467e
commit a36e6cba4e
68 changed files with 6822 additions and 0 deletions
+50
View File
@@ -0,0 +1,50 @@
# Session: stemwijzer
Updated: 2026-03-20T00:23:33Z
## Goal
Preserve the minimal session state required to resume work on the stemwijzer project after context clears (success = ledger exists and is kept up-to-date).
## Constraints
- Keep the ledger CONCISE — only essential information
- Focus on WHAT and WHY, not HOW
- Mark uncertain information as UNCONFIRMED
- Include git branch and key file paths
## Progress
### Done
- [x] Create initial continuity ledger file
### In Progress
- [ ] Capture ongoing session context and update ledger after each meaningful change
### Blocked
- None currently
## Key Decisions
- **Session name = "stemwijzer"**: Chosen from repository context (UNCONFIRMED if a different canonical session name is preferred).
- **Do not auto-commit ledger changes**: Commits will only be made when the user explicitly requests it (follows Git Safety Protocol).
## Next Steps
1. Continue updating this ledger when tasks, files, or decisions change
2. Add entries for new branches or major feature work (mark as UNCONFIRMED when unsure)
3. Ask user before creating any git commits that include this ledger
## File Operations
### Read
- `README.md`
- `pyproject.toml`
- `thoughts/shared/plans/2026-03-19-stemwijzer-plan.md`
- `thoughts/shared/designs/2026-03-19-stemwijzer-design.md`
### Modified
- `thoughts/ledgers/CONTINUITY_stemwijzer.md` (new)
## Critical Context
- Repository branch observed: `main`
- Found project metadata in `pyproject.toml` indicating Python tooling preference
- Existing notes/plans located under `thoughts/shared/` (plans and designs from 2026-03-19)
- No existing continuity ledger was found prior to this creation
## Working Set
- Branch: `main`
- Key files: `README.md`, `pyproject.toml`, `thoughts/shared/plans/2026-03-19-stemwijzer-plan.md`, `thoughts/shared/designs/2026-03-19-stemwijzer-design.md`, `thoughts/ledgers/CONTINUITY_stemwijzer.md`