refactor: delete stale files and consolidate .mindmodel structure
Deleted stale root-level Python files: - main.py (unused 'Hello world' script) - verify.py (unused table info script) - scraper.py (unused MotionScraper class) - scheduler.py (unused DataUpdateScheduler class) Deleted duplicate .mindmodel root YAML files (subdirectory versions are more comprehensive): - anti-patterns.yaml, architecture.yaml, conventions.yaml - dependencies.yaml, domain.yaml, domain-glossary.yaml - stack.yaml, tech-stack.yaml, workflows.yaml Added comprehensive .mindmodel subdirectories: - constraints/ (naming, db-schema, error-handling, types, etc.) - patterns/ (api, architecture, database, python, streamlit, etc.) - examples/ (code examples for each pattern) - anti-patterns/, architecture/, conventions/, dependencies/, domain/, stack/ Updated ARCHITECTURE.md to reflect current codebase: - Removed references to non-existent files - Added missing files (explorer.py, explorer_helpers.py, pipeline/) - Added directory structure documentation - Updated tech stack to include scipy, sklearn, umap Updated .gitignore: - Added patterns for generated analysis files - Added .worktrees/ pattern (was already in gitignore but dir was deleted) Removed empty .worktrees/ directory
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
# Domain Glossary - Dutch Political Terms
|
||||
|
||||
## Core Entities
|
||||
|
||||
### Motion / Motie
|
||||
- Parliamentary motion submitted by MPs
|
||||
- Fields: `id`, `title`, `date`, `category`
|
||||
- MPs vote: **For** (+1), **Against** (-1), **Abstain** (0), **Absent**
|
||||
|
||||
### MP / Kamerlid
|
||||
- Member of Parliament (Tweede Kamerlid)
|
||||
- Identified by full name (e.g., "Van Dijk, I.")
|
||||
- Has voting record, party affiliation, SVD position vector
|
||||
- Historical: `mp_party_history` tracks party changes over time
|
||||
|
||||
### Party / Fractie
|
||||
- Political party (e.g., "GroenLinks-PvdA", "PVV", "VVD")
|
||||
- Party centroids: average SVD position of all MPs in party
|
||||
- Aliases: multiple spelling variants exist (see anti-patterns.yaml)
|
||||
|
||||
### Vote / Stemming
|
||||
- Individual MP's vote on a motion: +1, 0, -1
|
||||
- Aggregated to compute SVD vectors
|
||||
|
||||
---
|
||||
|
||||
## Time & Analysis Concepts
|
||||
|
||||
### Window / Tijdsvenster
|
||||
- Time period for analysis (annual or quarterly)
|
||||
- Values: "2023", "2023-Q1", "2024", etc.
|
||||
- SVD vectors computed per window
|
||||
- Windows can be aligned across time using Procrustes
|
||||
|
||||
### Trajectory
|
||||
- MP's position change across multiple windows
|
||||
- Computed from `svd_vectors` + window ordering
|
||||
- Used for trend analysis in Evolution tab
|
||||
|
||||
---
|
||||
|
||||
## Mathematical / Algorithmic Terms
|
||||
|
||||
### SVD Vector
|
||||
- 2D vector from Singular Value Decomposition of MP × Motion vote matrix
|
||||
- Represents MP's position in political space
|
||||
- `entity_id` in `svd_vectors`: either MP name (when individual MPs) or party name (when party-level)
|
||||
|
||||
### Political Compass
|
||||
- 2D visualization: X-axis = Left↔Right, Y-axis = Progressive↔Conservative
|
||||
- SVD vectors mapped to compass quadrants
|
||||
- UMAP used for projection
|
||||
|
||||
### Procrustes Alignment
|
||||
- Algorithm to align SVD vectors across time windows
|
||||
- Ensures comparable positions across years/quarters
|
||||
- Implemented via `scipy.spatial.procrustes` or scikit-learn
|
||||
|
||||
### Centroid
|
||||
- Geometric center of a set of points
|
||||
- Party centroid = average SVD position of all MPs in that party
|
||||
- Computed from `svd_vectors` filtered by party
|
||||
|
||||
### UMAP
|
||||
- Uniform Manifold Approximation and Projection
|
||||
- Dimensionality reduction for visualization
|
||||
- Optional dependency — graceful fallback if unavailable
|
||||
|
||||
---
|
||||
|
||||
## Visualization
|
||||
|
||||
### PARTY_COLOURS
|
||||
- Dict mapping party names to hex color codes
|
||||
- Used in all Plotly charts for consistent party coloring
|
||||
- Source: `config.py` → `PARTY_COLOURS` constant
|
||||
- **Issue**: 3 separate alias dictionaries exist (no single source of truth)
|
||||
|
||||
---
|
||||
|
||||
## Application Pages
|
||||
|
||||
### Home
|
||||
- Landing page with app overview
|
||||
|
||||
### Stemwijzer (Quiz)
|
||||
- User answers questions → matched to parties
|
||||
- Thin wrapper around quiz module
|
||||
|
||||
### Explorer (4 tabs)
|
||||
- **Motion tab**: SVD positions colored by vote on selected motion
|
||||
- **MP tab**: Individual MP trajectories across windows
|
||||
- **Party tab**: Party centroids with members as scatter
|
||||
- **Evolution tab**: How positions change over time
|
||||
|
||||
---
|
||||
|
||||
## Database Table Reference
|
||||
| Table | Key Fields |
|
||||
|-------|-----------|
|
||||
| `motions` | id, title, date, category |
|
||||
| `mp_votes` | mp_id, motion_id, vote |
|
||||
| `svd_vectors` | entity_id, window, vector_2d (list[2]) |
|
||||
| `party_centroids` | party, window, centroid_2d |
|
||||
| `mp_party_history` | mp_id, party, start_date, end_date |
|
||||
| `windows` | window_id, start_date, end_date, period_type |
|
||||
| `mp_trajectories` | mp_id, window, trajectory_vector |
|
||||
Reference in New Issue
Block a user