|
|
# Architecture
|
|
|
|
|
|
## Page Routing
|
|
|
- `Home.py` → thin wrapper, minimal logic
|
|
|
- `pages/1_🗳️_Stemwijzer.py` → thin wrapper delegating to quiz module
|
|
|
- `pages/2_🔍_Explorer.py` → thin wrapper delegating to `explorer.py`
|
|
|
- **Pattern**: thin Streamlit page files that import and call into core modules
|
|
|
|
|
|
## Core Modules
|
|
|
```
|
|
|
database.py → MotionDatabase singleton (shared across all pages)
|
|
|
explorer.py → Explorer page logic, tab routing
|
|
|
explorer_helpers.py → Pure functions, chart builders, coordinate computation
|
|
|
analysis/ → SVD, UMAP, clustering algorithms
|
|
|
pipeline/ → Data ingestion pipeline
|
|
|
config.py → Dataclass Config, PARTY_COLOURS dict
|
|
|
```
|
|
|
|
|
|
## Data Flow
|
|
|
```
|
|
|
DuckDB → MotionDatabase (singleton)
|
|
|
↓
|
|
|
st.cache_data loaders
|
|
|
↓
|
|
|
explorer_helpers (pure functions)
|
|
|
↓
|
|
|
Plotly charts → Streamlit
|
|
|
```
|
|
|
|
|
|
## Key Patterns
|
|
|
1. **Singleton per module**: `database.py` exports one `db` instance; `config.py` exports config + PARTY_COLOURS
|
|
|
2. **Graceful degradation**: try/except around optional dependencies (UMAP, Plotly)
|
|
|
3. **Pipeline**: fetch → transform → store (see `pipeline/` directory)
|
|
|
4. **API client**: with retry/backoff for external data sources
|
|
|
5. **Dummy fallbacks**: if optional dep unavailable, use dummy stub
|
|
|
|
|
|
## Database Schema (key relationships)
|
|
|
```
|
|
|
motions (id, title, date, category)
|
|
|
↓
|
|
|
mp_votes (mp_id, motion_id, vote: -1/0/1)
|
|
|
↓
|
|
|
svd_vectors (entity_id, window, vector_2d) ← entity_id = mp_name OR party_name
|
|
|
↓
|
|
|
party_centroids (party, window, centroid_2d)
|
|
|
↓
|
|
|
mp_party_history (mp_id, party, start_date, end_date)
|
|
|
```
|
|
|
|
|
|
## SVD Computation Pipeline
|
|
|
1. Build MP × Motion vote matrix from `mp_votes`
|
|
|
2. Run SVD to get 2D embeddings per MP
|
|
|
3. Optionally aggregate to party centroids
|
|
|
4. Align across windows using Procrustes
|
|
|
5. Store in `svd_vectors` table
|
|
|
|
