You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
67 lines
2.1 KiB
67 lines
2.1 KiB
---
|
|
title: Tech Stack
|
|
category: stack
|
|
---
|
|
|
|
# Tech Stack
|
|
|
|
## Runtime & Language
|
|
- **Python >=3.13**
|
|
|
|
## Web Framework
|
|
- **Streamlit** - Multi-page app with Home, Stemwijzer, Explorer pages
|
|
|
|
## Data Layer
|
|
- **DuckDB** - Embedded OLAP database
|
|
- Tables: motions, mp_votes, svd_vectors, fused_embeddings, embeddings, user_sessions, party_results, mp_metadata
|
|
- **ibis** - ORM (referenced but DuckDB-native implementation used)
|
|
|
|
## AI / LLM
|
|
- **OpenRouter** - API abstraction for AI providers
|
|
- **QWEN** - Primary model
|
|
- Embeddings: `qwen/qwen3-embedding-4b`
|
|
- Chat: `qwen/qwen-2.5-72b-instruct`
|
|
- **requests** - HTTP client (not raw openai)
|
|
|
|
## ML / Analytics
|
|
- **scikit-learn** - KMeans clustering, cosine_similarity, StandardScaler
|
|
- **scipy** - SVD (scipy.linalg.svd), spatial.procrustes
|
|
- **umap-learn** - Dimensionality reduction (optional, graceful fallback to SVD)
|
|
- **numpy** - Numerical computing
|
|
|
|
## Visualization
|
|
- **Plotly** - Interactive charts (go.Figure, _DummyTrace fallback)
|
|
- **matplotlib** - Static plotting (optional)
|
|
|
|
## HTTP & Parsing
|
|
- **requests** - Session pooling, retry with backoff
|
|
- **beautifulsoup4** - HTML parsing
|
|
- **lxml** - XML/HTML processing
|
|
|
|
## Key Source Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `database.py` | MotionDatabase singleton, DuckDB connection, 9-table schema |
|
|
| `explorer.py` | Explorer page with 4 tabs (Motion, MP, Party, Evolution) |
|
|
| `explorer_helpers.py` | Pure helper functions, Plotly chart builders |
|
|
| `analysis/` | SVD pipeline, UMAP projection, clustering |
|
|
| `pipeline/` | Data fetch, transform, store pipeline |
|
|
| `pages/1_Stemwijzer.py` | Quiz page |
|
|
| `pages/2_Explorer.py` | Explorer page |
|
|
| `config.py` | Dataclass Config pattern |
|
|
| `ai_provider.py` | OpenRouter API wrapper with retry |
|
|
| `api_client.py` | TweedeKamer OData API client |
|
|
|
|
## Singleton Instances
|
|
|
|
| Module | Instance | Type |
|
|
|--------|----------|------|
|
|
| `database.py` | `db` | `MotionDatabase` |
|
|
| `config.py` | `config` | `Config` (dataclass) |
|
|
| `config.py` | `PARTY_COLOURS` | `dict[str, str]` |
|
|
|
|
## Environment
|
|
- Python >=3.13
|
|
- Environment variables via `.env` (DB path, API keys)
|
|
- No `.env` values in constraint files (security)
|
|
|