parent
805cc7e284
commit
eb71328967
@ -1,15 +1,74 @@ |
|||||||
# DO NOT EDIT - read-only until validated |
name: stemwijzer |
||||||
# Sanitized manifest: contains non-sensitive sample excerpts only |
version: 2 |
||||||
files: |
description: Dutch political voting compass (Stemwijzer) - Mind Model constraints |
||||||
- path: src/lib/schema.ts |
|
||||||
evidence_excerpt: "Defines schema for user input validation" |
categories: |
||||||
flags: |
# Core documentation |
||||||
needs_review: true |
- path: system.md |
||||||
- path: src/api/handler.ts |
description: System overview and architecture summary |
||||||
evidence_excerpt: "Handles API requests and routing" |
group: docs |
||||||
flags: |
- path: tech-stack.yaml |
||||||
needs_review: false |
description: Technology stack with versions and purposes |
||||||
- path: README.md |
group: docs |
||||||
evidence_excerpt: "Project overview and setup instructions" |
- path: conventions.yaml |
||||||
flags: |
description: Coding conventions and style guide |
||||||
needs_review: true |
group: docs |
||||||
|
- path: domain.yaml |
||||||
|
description: Domain entities, terms, and relationships |
||||||
|
group: docs |
||||||
|
|
||||||
|
# Design patterns |
||||||
|
- path: patterns/architecture.yaml |
||||||
|
description: Repository, Facade, Pipeline architectural patterns |
||||||
|
group: patterns |
||||||
|
- path: patterns/python.yaml |
||||||
|
description: Python-specific patterns (Singleton, dataclass, context manager) |
||||||
|
group: patterns |
||||||
|
- path: patterns/database.yaml |
||||||
|
description: DuckDB connection patterns and ORM usage |
||||||
|
group: patterns |
||||||
|
- path: patterns/api.yaml |
||||||
|
description: API client patterns with retry logic and pagination |
||||||
|
group: patterns |
||||||
|
- path: patterns/streamlit.yaml |
||||||
|
description: Streamlit session state and page patterns |
||||||
|
group: patterns |
||||||
|
|
||||||
|
# Coding constraints |
||||||
|
- path: constraints/error-handling.yaml |
||||||
|
description: Error handling patterns with safe fallbacks |
||||||
|
group: constraints |
||||||
|
- path: constraints/logging.yaml |
||||||
|
description: Logging conventions and best practices |
||||||
|
group: constraints |
||||||
|
- path: constraints/naming.yaml |
||||||
|
description: File, class, function naming rules |
||||||
|
group: constraints |
||||||
|
- path: constraints/imports.yaml |
||||||
|
description: Import organization and module structure |
||||||
|
group: constraints |
||||||
|
- path: constraints/types.yaml |
||||||
|
description: Type hint conventions |
||||||
|
group: constraints |
||||||
|
|
||||||
|
# Code examples |
||||||
|
- path: examples/database-example.py |
||||||
|
description: MotionDatabase usage example |
||||||
|
group: examples |
||||||
|
- path: examples/api-client-example.py |
||||||
|
description: TweedeKamerAPI usage |
||||||
|
group: examples |
||||||
|
- path: examples/pipeline-example.py |
||||||
|
description: Pipeline phase example |
||||||
|
group: examples |
||||||
|
- path: examples/streamlit-page-example.py |
||||||
|
description: Streamlit page pattern |
||||||
|
group: examples |
||||||
|
|
||||||
|
# Anti-patterns and workflows |
||||||
|
- path: anti-patterns.yaml |
||||||
|
description: Known anti-patterns to avoid |
||||||
|
group: meta |
||||||
|
- path: workflows.yaml |
||||||
|
description: Key workflows (VotingSession, DataIngestion, EmbeddingGeneration) |
||||||
|
group: meta |
||||||
|
|||||||
@ -1,14 +1,69 @@ |
|||||||
# System Overview: Stemwijzer |
# System Overview |
||||||
|
|
||||||
This mindmodel documents constraints, conventions and patterns for the Stemwijzer |
## Project: Stemwijzer (Dutch Political Voting Compass) |
||||||
project (Python Streamlit app with DuckDB-backed pipeline for parliamentary |
|
||||||
motions embedding analysis). |
|
||||||
|
|
||||||
Key points: |
**Purpose**: A web application that maps the Dutch Tweede Kamer (House of Representatives) based on real parliamentary votes, helping citizens discover which political party aligns best with their views. |
||||||
- Language: Python >=3.13 |
|
||||||
- UI: Streamlit multi-page app (Home.py, pages/) |
|
||||||
- Storage: DuckDB with JSON fallback for tests/dev (database.py) |
|
||||||
- Pipeline: ETL and SVD/text fusion pipeline (pipeline/run_pipeline.py) |
|
||||||
- AI: ai_provider adapter uses HTTP-based OpenRouter/OpenAI-compatible API with retry/backoff and local fallback. QWEN via OpenRouter is the recommended path; prefer OPENROUTER_API_KEY with OPENAI_API_KEY as a fallback where applicable. |
|
||||||
|
|
||||||
Use the .mindmodel/ constraints files to guide code changes, CI, and onboarding. |
## Architecture Summary |
||||||
|
|
||||||
|
### Data Flow |
||||||
|
``` |
||||||
|
TweedeKamer OData API |
||||||
|
↓ |
||||||
|
API Client (api_client.py) |
||||||
|
↓ |
||||||
|
DuckDB Database (database.py) |
||||||
|
↓ |
||||||
|
Pipeline Processing (pipeline/) |
||||||
|
├── fetch_mp_metadata # MP party + tenure |
||||||
|
├── extract_mp_votes # voting_results → mp_votes |
||||||
|
├── svd_pipeline # SVD on vote matrix + Procrustes |
||||||
|
├── text_pipeline # AI embeddings via OpenRouter |
||||||
|
└── fusion # Combine SVD + text vectors |
||||||
|
↓ |
||||||
|
Streamlit Web App (app.py, pages/) |
||||||
|
├── Home.py # Landing page |
||||||
|
├── 1_Stemwijzer.py # Voting quiz |
||||||
|
└── 2_Explorer.py # Political compass explorer |
||||||
|
``` |
||||||
|
|
||||||
|
### Key Components |
||||||
|
|
||||||
|
| Component | Purpose | File(s) | |
||||||
|
|-----------|---------|---------| |
||||||
|
| **Database** | Motion storage, MP votes, embeddings | `database.py` | |
||||||
|
| **API Client** | TweedeKamer OData API integration | `api_client.py` | |
||||||
|
| **AI Provider** | OpenRouter API for embeddings/summaries | `ai_provider.py` | |
||||||
|
| **Pipeline** | Orchestrated data processing | `pipeline/run_pipeline.py` | |
||||||
|
| **Analysis** | SVD, clustering, trajectory computation | `analysis/*.py` | |
||||||
|
| **Similarity** | Motion similarity search | `similarity/*.py` | |
||||||
|
| **Web App** | Streamlit UI | `app.py`, `pages/*.py` | |
||||||
|
|
||||||
|
### Data Models |
||||||
|
|
||||||
|
**Core Entities**: |
||||||
|
- `Motion`: Parliamentary motion with voting results |
||||||
|
- `MP` / `MPMetadata`: Member of Parliament with party/tenure |
||||||
|
- `MPVote`: Individual vote record (Voor/Tegen/Onthouden/Geen stem/Afwezig) |
||||||
|
- `Party`: Political party |
||||||
|
- `UserSession` / `UserVote`: Voting session tracking |
||||||
|
- `SVDVector`: Dimensionality-reduced vote vectors |
||||||
|
- `FusedEmbedding`: Combined SVD + text embedding |
||||||
|
- `SimilarityCache`: Pre-computed motion similarities |
||||||
|
|
||||||
|
### Technical Decisions |
||||||
|
|
||||||
|
1. **DuckDB over SQLite**: Chosen for OLAP performance with complex analytical queries |
||||||
|
2. **ibis ORM**: Database-agnostic query building (currently using DuckDB backend) |
||||||
|
3. **SVD + Procrustes**: Aligns voting vectors across time windows |
||||||
|
4. **UMAP for visualization**: Non-linear dimensionality reduction for compass display |
||||||
|
5. **OpenRouter API**: Abstraction layer for AI embeddings (currently using Qwen) |
||||||
|
6. **Module-level singletons**: `db = MotionDatabase()` pattern for shared state |
||||||
|
|
||||||
|
### Key Conventions |
||||||
|
|
||||||
|
- **DuckDB connections**: Short-lived per method, always close |
||||||
|
- **Error handling**: Catch `Exception`, return safe fallbacks (False/[]/None) |
||||||
|
- **Logging**: Use `logging.getLogger(__name__)` - avoid print() |
||||||
|
- **Type hints**: Required on public functions with typing module imports |
||||||
|
- **Config**: Dataclass `Config` in `config.py`, accessed as `from config import config` |
||||||
|
|||||||
File diff suppressed because one or more lines are too long
Loading…
Reference in new issue