refactor: delete stale files and consolidate .mindmodel structure
Deleted stale root-level Python files: - main.py (unused 'Hello world' script) - verify.py (unused table info script) - scraper.py (unused MotionScraper class) - scheduler.py (unused DataUpdateScheduler class) Deleted duplicate .mindmodel root YAML files (subdirectory versions are more comprehensive): - anti-patterns.yaml, architecture.yaml, conventions.yaml - dependencies.yaml, domain.yaml, domain-glossary.yaml - stack.yaml, tech-stack.yaml, workflows.yaml Added comprehensive .mindmodel subdirectories: - constraints/ (naming, db-schema, error-handling, types, etc.) - patterns/ (api, architecture, database, python, streamlit, etc.) - examples/ (code examples for each pattern) - anti-patterns/, architecture/, conventions/, dependencies/, domain/, stack/ Updated ARCHITECTURE.md to reflect current codebase: - Removed references to non-existent files - Added missing files (explorer.py, explorer_helpers.py, pipeline/) - Added directory structure documentation - Updated tech stack to include scipy, sklearn, umap Updated .gitignore: - Added patterns for generated analysis files - Added .worktrees/ pattern (was already in gitignore but dir was deleted) Removed empty .worktrees/ directory
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
# Architecture
|
||||
|
||||
## Page Routing
|
||||
- `Home.py` → thin wrapper, minimal logic
|
||||
- `pages/1_🗳️_Stemwijzer.py` → thin wrapper delegating to quiz module
|
||||
- `pages/2_🔍_Explorer.py` → thin wrapper delegating to `explorer.py`
|
||||
- **Pattern**: thin Streamlit page files that import and call into core modules
|
||||
|
||||
## Core Modules
|
||||
```
|
||||
database.py → MotionDatabase singleton (shared across all pages)
|
||||
explorer.py → Explorer page logic, tab routing
|
||||
explorer_helpers.py → Pure functions, chart builders, coordinate computation
|
||||
analysis/ → SVD, UMAP, clustering algorithms
|
||||
pipeline/ → Data ingestion pipeline
|
||||
config.py → Dataclass Config, PARTY_COLOURS dict
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
```
|
||||
DuckDB → MotionDatabase (singleton)
|
||||
↓
|
||||
st.cache_data loaders
|
||||
↓
|
||||
explorer_helpers (pure functions)
|
||||
↓
|
||||
Plotly charts → Streamlit
|
||||
```
|
||||
|
||||
## Key Patterns
|
||||
1. **Singleton per module**: `database.py` exports one `db` instance; `config.py` exports config + PARTY_COLOURS
|
||||
2. **Graceful degradation**: try/except around optional dependencies (UMAP, Plotly)
|
||||
3. **Pipeline**: fetch → transform → store (see `pipeline/` directory)
|
||||
4. **API client**: with retry/backoff for external data sources
|
||||
5. **Dummy fallbacks**: if optional dep unavailable, use dummy stub
|
||||
|
||||
## Database Schema (key relationships)
|
||||
```
|
||||
motions (id, title, date, category)
|
||||
↓
|
||||
mp_votes (mp_id, motion_id, vote: -1/0/1)
|
||||
↓
|
||||
svd_vectors (entity_id, window, vector_2d) ← entity_id = mp_name OR party_name
|
||||
↓
|
||||
party_centroids (party, window, centroid_2d)
|
||||
↓
|
||||
mp_party_history (mp_id, party, start_date, end_date)
|
||||
```
|
||||
|
||||
## SVD Computation Pipeline
|
||||
1. Build MP × Motion vote matrix from `mp_votes`
|
||||
2. Run SVD to get 2D embeddings per MP
|
||||
3. Optionally aggregate to party centroids
|
||||
4. Align across windows using Procrustes
|
||||
5. Store in `svd_vectors` table
|
||||
Reference in New Issue
Block a user