sgeboers/motief
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26 Commits 1 Branch 0 Tags
b09e580f655fb55da03570da5e1ed1fa8b41a925
Commit Graph
7 Commits
Author SHA1 Message Date
sgeboers b09e580f65 feat: motion content enrichment pipeline hardening 
- ai_provider_wrapper: retry/fallback with exponential backoff, None sentinel for failed items
- text_pipeline: use wrapper, return 5-tuple (stored, skipped_existing, skipped_no_text, errors, failed_ids)
- similarity/compute: filter trivial 1.0 matches on identical short titles (<12 chars)
- rerun_embeddings: --retry-missing mode, calls ensure_text_embeddings_for_ids on failed ids
- sync_motion_content: per-ext_id retries, HTTPAdapter pool, --max-body-workers CLI flag, audit on failure
- qa_similarity script: samples motions, writes JSON ledger to thoughts/ledgers/
- All tests green: 61 passed, 2 skipped
 2026-03-23 21:31:39 +01:00
sgeboers 07a89a207c docs: add test refactor design - replace monkeypatching with DI + in-memory DuckDB 2026-03-23 20:15:29 +01:00
sgeboers ce27dc6ac5 chore(ledgers): record fusion+similarity run summary and JSON details 2026-03-23 17:48:39 +01:00
sgeboers 2891e9ee70 feat: add StemAtlas Streamlit app, explorer, Docker deployment, blog charts 2026-03-22 22:38:17 +01:00
sgeboers a248807e03 Add design: embedding-based motion similarity cache 
Precomputed top-K similarity cache replacing the naive Python-scan
search_similar(). Also covers fixes for: embeddings table missing from
_init_database, fusion N+1 query, and ai_provider 429 retry.
 2026-03-22 02:39:20 +01:00
sgeboers a36e6cba4e feat(pipeline): implement parliamentary embedding pipeline MVP 
- Add 4 migration files: mp_votes, mp_metadata, svd_vectors, fused_embeddings
- Extend database.py with 5 new helper methods and table init
- Add pipeline/ package: extract_mp_votes, fetch_mp_metadata, text_pipeline,
  svd_pipeline (with Procrustes alignment), fusion
- Add full test suite (17 tests) covering all pipeline modules and migrations
- Fix Procrustes alignment bug: scipy scale is a norm value, not a multiplier
- Fix DuckDB date type handling in test assertions (datetime.date vs string)
- Remove duckdb.py shim; tests now run against real duckdb + scipy via uv

Ref: thoughts/shared/plans/2026-03-21-parliamentary-embedding-pipeline-plan.md
 2026-03-21 22:31:22 +01:00
sgeboers fd73da3752 design: parliamentary embedding pipeline (late fusion SVD + text) 2026-03-21 21:54:32 +01:00