refactor: delete stale files and consolidate .mindmodel structure

Deleted stale root-level Python files:
- main.py (unused 'Hello world' script)
- verify.py (unused table info script)
- scraper.py (unused MotionScraper class)
- scheduler.py (unused DataUpdateScheduler class)

Deleted duplicate .mindmodel root YAML files (subdirectory versions are more comprehensive):
- anti-patterns.yaml, architecture.yaml, conventions.yaml
- dependencies.yaml, domain.yaml, domain-glossary.yaml
- stack.yaml, tech-stack.yaml, workflows.yaml

Added comprehensive .mindmodel subdirectories:
- constraints/ (naming, db-schema, error-handling, types, etc.)
- patterns/ (api, architecture, database, python, streamlit, etc.)
- examples/ (code examples for each pattern)
- anti-patterns/, architecture/, conventions/, dependencies/, domain/, stack/

Updated ARCHITECTURE.md to reflect current codebase:
- Removed references to non-existent files
- Added missing files (explorer.py, explorer_helpers.py, pipeline/)
- Added directory structure documentation
- Updated tech stack to include scipy, sklearn, umap

Updated .gitignore:
- Added patterns for generated analysis files
- Added .worktrees/ pattern (was already in gitignore but dir was deleted)

Removed empty .worktrees/ directory
This commit is contained in:
2026-04-04 18:46:53 +02:00
parent 0308d20f12
commit f376300804
22 changed files with 3824 additions and 48 deletions
+55
View File
@@ -0,0 +1,55 @@
# Architecture
## Page Routing
- `Home.py` → thin wrapper, minimal logic
- `pages/1_🗳️_Stemwijzer.py` → thin wrapper delegating to quiz module
- `pages/2_🔍_Explorer.py` → thin wrapper delegating to `explorer.py`
- **Pattern**: thin Streamlit page files that import and call into core modules
## Core Modules
```
database.py → MotionDatabase singleton (shared across all pages)
explorer.py → Explorer page logic, tab routing
explorer_helpers.py → Pure functions, chart builders, coordinate computation
analysis/ → SVD, UMAP, clustering algorithms
pipeline/ → Data ingestion pipeline
config.py → Dataclass Config, PARTY_COLOURS dict
```
## Data Flow
```
DuckDB → MotionDatabase (singleton)
st.cache_data loaders
explorer_helpers (pure functions)
Plotly charts → Streamlit
```
## Key Patterns
1. **Singleton per module**: `database.py` exports one `db` instance; `config.py` exports config + PARTY_COLOURS
2. **Graceful degradation**: try/except around optional dependencies (UMAP, Plotly)
3. **Pipeline**: fetch → transform → store (see `pipeline/` directory)
4. **API client**: with retry/backoff for external data sources
5. **Dummy fallbacks**: if optional dep unavailable, use dummy stub
## Database Schema (key relationships)
```
motions (id, title, date, category)
mp_votes (mp_id, motion_id, vote: -1/0/1)
svd_vectors (entity_id, window, vector_2d) ← entity_id = mp_name OR party_name
party_centroids (party, window, centroid_2d)
mp_party_history (mp_id, party, start_date, end_date)
```
## SVD Computation Pipeline
1. Build MP × Motion vote matrix from `mp_votes`
2. Run SVD to get 2D embeddings per MP
3. Optionally aggregate to party centroids
4. Align across windows using Procrustes
5. Store in `svd_vectors` table