parent
805cc7e284
commit
eb71328967
@ -1,15 +1,74 @@ |
||||
# DO NOT EDIT - read-only until validated |
||||
# Sanitized manifest: contains non-sensitive sample excerpts only |
||||
files: |
||||
- path: src/lib/schema.ts |
||||
evidence_excerpt: "Defines schema for user input validation" |
||||
flags: |
||||
needs_review: true |
||||
- path: src/api/handler.ts |
||||
evidence_excerpt: "Handles API requests and routing" |
||||
flags: |
||||
needs_review: false |
||||
- path: README.md |
||||
evidence_excerpt: "Project overview and setup instructions" |
||||
flags: |
||||
needs_review: true |
||||
name: stemwijzer |
||||
version: 2 |
||||
description: Dutch political voting compass (Stemwijzer) - Mind Model constraints |
||||
|
||||
categories: |
||||
# Core documentation |
||||
- path: system.md |
||||
description: System overview and architecture summary |
||||
group: docs |
||||
- path: tech-stack.yaml |
||||
description: Technology stack with versions and purposes |
||||
group: docs |
||||
- path: conventions.yaml |
||||
description: Coding conventions and style guide |
||||
group: docs |
||||
- path: domain.yaml |
||||
description: Domain entities, terms, and relationships |
||||
group: docs |
||||
|
||||
# Design patterns |
||||
- path: patterns/architecture.yaml |
||||
description: Repository, Facade, Pipeline architectural patterns |
||||
group: patterns |
||||
- path: patterns/python.yaml |
||||
description: Python-specific patterns (Singleton, dataclass, context manager) |
||||
group: patterns |
||||
- path: patterns/database.yaml |
||||
description: DuckDB connection patterns and ORM usage |
||||
group: patterns |
||||
- path: patterns/api.yaml |
||||
description: API client patterns with retry logic and pagination |
||||
group: patterns |
||||
- path: patterns/streamlit.yaml |
||||
description: Streamlit session state and page patterns |
||||
group: patterns |
||||
|
||||
# Coding constraints |
||||
- path: constraints/error-handling.yaml |
||||
description: Error handling patterns with safe fallbacks |
||||
group: constraints |
||||
- path: constraints/logging.yaml |
||||
description: Logging conventions and best practices |
||||
group: constraints |
||||
- path: constraints/naming.yaml |
||||
description: File, class, function naming rules |
||||
group: constraints |
||||
- path: constraints/imports.yaml |
||||
description: Import organization and module structure |
||||
group: constraints |
||||
- path: constraints/types.yaml |
||||
description: Type hint conventions |
||||
group: constraints |
||||
|
||||
# Code examples |
||||
- path: examples/database-example.py |
||||
description: MotionDatabase usage example |
||||
group: examples |
||||
- path: examples/api-client-example.py |
||||
description: TweedeKamerAPI usage |
||||
group: examples |
||||
- path: examples/pipeline-example.py |
||||
description: Pipeline phase example |
||||
group: examples |
||||
- path: examples/streamlit-page-example.py |
||||
description: Streamlit page pattern |
||||
group: examples |
||||
|
||||
# Anti-patterns and workflows |
||||
- path: anti-patterns.yaml |
||||
description: Known anti-patterns to avoid |
||||
group: meta |
||||
- path: workflows.yaml |
||||
description: Key workflows (VotingSession, DataIngestion, EmbeddingGeneration) |
||||
group: meta |
||||
|
||||
@ -1,14 +1,69 @@ |
||||
# System Overview: Stemwijzer |
||||
# System Overview |
||||
|
||||
This mindmodel documents constraints, conventions and patterns for the Stemwijzer |
||||
project (Python Streamlit app with DuckDB-backed pipeline for parliamentary |
||||
motions embedding analysis). |
||||
## Project: Stemwijzer (Dutch Political Voting Compass) |
||||
|
||||
Key points: |
||||
- Language: Python >=3.13 |
||||
- UI: Streamlit multi-page app (Home.py, pages/) |
||||
- Storage: DuckDB with JSON fallback for tests/dev (database.py) |
||||
- Pipeline: ETL and SVD/text fusion pipeline (pipeline/run_pipeline.py) |
||||
- AI: ai_provider adapter uses HTTP-based OpenRouter/OpenAI-compatible API with retry/backoff and local fallback. QWEN via OpenRouter is the recommended path; prefer OPENROUTER_API_KEY with OPENAI_API_KEY as a fallback where applicable. |
||||
**Purpose**: A web application that maps the Dutch Tweede Kamer (House of Representatives) based on real parliamentary votes, helping citizens discover which political party aligns best with their views. |
||||
|
||||
Use the .mindmodel/ constraints files to guide code changes, CI, and onboarding. |
||||
## Architecture Summary |
||||
|
||||
### Data Flow |
||||
``` |
||||
TweedeKamer OData API |
||||
↓ |
||||
API Client (api_client.py) |
||||
↓ |
||||
DuckDB Database (database.py) |
||||
↓ |
||||
Pipeline Processing (pipeline/) |
||||
├── fetch_mp_metadata # MP party + tenure |
||||
├── extract_mp_votes # voting_results → mp_votes |
||||
├── svd_pipeline # SVD on vote matrix + Procrustes |
||||
├── text_pipeline # AI embeddings via OpenRouter |
||||
└── fusion # Combine SVD + text vectors |
||||
↓ |
||||
Streamlit Web App (app.py, pages/) |
||||
├── Home.py # Landing page |
||||
├── 1_Stemwijzer.py # Voting quiz |
||||
└── 2_Explorer.py # Political compass explorer |
||||
``` |
||||
|
||||
### Key Components |
||||
|
||||
| Component | Purpose | File(s) | |
||||
|-----------|---------|---------| |
||||
| **Database** | Motion storage, MP votes, embeddings | `database.py` | |
||||
| **API Client** | TweedeKamer OData API integration | `api_client.py` | |
||||
| **AI Provider** | OpenRouter API for embeddings/summaries | `ai_provider.py` | |
||||
| **Pipeline** | Orchestrated data processing | `pipeline/run_pipeline.py` | |
||||
| **Analysis** | SVD, clustering, trajectory computation | `analysis/*.py` | |
||||
| **Similarity** | Motion similarity search | `similarity/*.py` | |
||||
| **Web App** | Streamlit UI | `app.py`, `pages/*.py` | |
||||
|
||||
### Data Models |
||||
|
||||
**Core Entities**: |
||||
- `Motion`: Parliamentary motion with voting results |
||||
- `MP` / `MPMetadata`: Member of Parliament with party/tenure |
||||
- `MPVote`: Individual vote record (Voor/Tegen/Onthouden/Geen stem/Afwezig) |
||||
- `Party`: Political party |
||||
- `UserSession` / `UserVote`: Voting session tracking |
||||
- `SVDVector`: Dimensionality-reduced vote vectors |
||||
- `FusedEmbedding`: Combined SVD + text embedding |
||||
- `SimilarityCache`: Pre-computed motion similarities |
||||
|
||||
### Technical Decisions |
||||
|
||||
1. **DuckDB over SQLite**: Chosen for OLAP performance with complex analytical queries |
||||
2. **ibis ORM**: Database-agnostic query building (currently using DuckDB backend) |
||||
3. **SVD + Procrustes**: Aligns voting vectors across time windows |
||||
4. **UMAP for visualization**: Non-linear dimensionality reduction for compass display |
||||
5. **OpenRouter API**: Abstraction layer for AI embeddings (currently using Qwen) |
||||
6. **Module-level singletons**: `db = MotionDatabase()` pattern for shared state |
||||
|
||||
### Key Conventions |
||||
|
||||
- **DuckDB connections**: Short-lived per method, always close |
||||
- **Error handling**: Catch `Exception`, return safe fallbacks (False/[]/None) |
||||
- **Logging**: Use `logging.getLogger(__name__)` - avoid print() |
||||
- **Type hints**: Required on public functions with typing module imports |
||||
- **Config**: Dataclass `Config` in `config.py`, accessed as `from config import config` |
||||
|
||||
File diff suppressed because one or more lines are too long
Loading…
Reference in new issue