motief/docs/plans/2026-04-24-ROADMAP-stemwijzer-improvements.md

# Stemwijzer Improvement Roadmap

**Created:** 2026-04-24
**Status:** Active

This roadmap captures 17 improvement opportunities identified during a codebase review, organized into 5 phases by dependency and risk. Each item links to a detailed implementation plan (created separately) when available.

---

## Phase 1: Foundation (CI/CD, Config, Documentation)

*Prerequisite for everything else. Low risk, high developer-experience impact.*

| # | Improvement | Priority | Effort | Plan | TDD |
|---|------------|----------|--------|------|-----|
| 1 | Fix broken CI test workflow (mindmodel-schedule.yml references missing requirements.txt) | High | Small | P1-001 | Yes |
| 2 | ~~Fix docker-compose.yml (missing scheduler.py)~~ — *Removed: Docker deployment not used* | — | — | — | — |
| 3 | Consolidate duplicate config sources (config.py vs analysis/config.py) | Medium | Small | P1-003 | Yes |
| 4 | Rewrite README.md (22 lines → proper quickstart) | High | Small | P1-004 | No (docs) |
| 5 | Add pyright type-checking to CI | Medium | Small | P1-005 | Yes |
| 6 | Activate pre-commit hooks (black, ruff, isort) | Medium | Small | P1-006 | Yes |

**Phase goal:** Reliable CI, clean config, and onboarding docs that don't require discovering ARCHITECTURE.md.

---

## Phase 2: Code Quality (Logging, Error Handling, Import Safety)

*Builds on Phase 1 CI. Makes the codebase maintainable and production-ready.*

| # | Improvement | Priority | Effort | Plan | TDD |
|---|------------|----------|--------|------|-----|
| 7 | Replace ~225 print() calls with structured logging | Medium | Medium | P2-001 | Yes |
| 8 | Fix broad `except Exception:` blocks in database.py and api_client.py | Medium | Medium | P2-002 | Yes |
| 9 | Complete import-safe module guards (extend existing work) | Medium | Medium | — | Yes |

**Phase goal:** Observable, debuggable production behavior with clear error propagation.

---

## Phase 3: Architecture (Decompose explorer.py)

*Already partially completed. Remaining work is decoupling Streamlit from tab logic.*

| # | Improvement | Priority | Effort | Plan | TDD |
|---|------------|----------|--------|------|-----|
| 10 | Complete explorer.py decomposition (extract tab logic from Streamlit) | Medium | Large | P3-001 | Yes |

**Status:** Constants extracted to analysis/config.py, placeholder tab modules created. Remaining: move build_*_tab functions out of explorer.py while preserving @st.cache_data decorators.

**Phase goal:** explorer.py under 1500 lines, tab modules independently testable.

---

## Phase 4: New Features

*User-facing value. Depends on Phase 2 for observability and Phase 3 for clean architecture.*

| # | Improvement | Priority | Effort | Plan | TDD |
|---|------------|----------|--------|------|-----|
| 11 | REST API layer (read-only, FastAPI) | Low | Large | P4-001 | Yes |
| 12 | Automated pipeline scheduling (real scheduler.py) | Medium | Medium | P4-002 | Yes |
| 13 | Motion recommendation engine | Low | Medium | P4-003 | Yes |
| 14 | Export user voting profile (JSON/CSV/shareable image) | Low | Small | P4-004 | Yes |
| 15 | Data quality dashboard (Streamlit page or API) | Medium | Medium | P4-005 | Yes |

**Phase goal:** External API consumers, automated data freshness, and user engagement features.

---

## Phase 5: Observability & Robustness

*Production confidence. Can run in parallel with Phase 4.*

| # | Improvement | Priority | Effort | Plan | TDD |
|---|------------|----------|--------|------|-----|
| 16 | Add Sentry or error tracking | Low | Small | P5-001 | No (config) |
| 17 | Pipeline health checks / alerting script | Medium | Medium | P5-002 | Yes |
| 18 | Benchmark suite (pytest-benchmark for SVD/fusion) | Low | Small | P5-003 | Yes |

**Phase goal:** Know when things break before users do; detect performance regressions.

---

## Dependency Graph

```
Phase 1 (Foundation)
  ├─→ Phase 2 (Code Quality) ─┬─→ Phase 3 (Architecture)
  │                           │        └─→ Phase 4 (Features)
  │                           └──────────────→ Phase 4 (Features)
  └─→ Phase 5 (Observability) ───────────────→
```

Phase 1 must come first. Phase 2 makes Phase 3/4 safer. Phase 3 unlocks some Phase 4 items. Phase 5 is largely independent.

---

## Recommended Execution Order

**Sprint 1:** Items 1, 4, 6 (CI + docs + pre-commit)
**Sprint 2:** Items 5, 7, 8 (type checking + logging + errors)
**Sprint 3:** Item 10 (explorer decomposition)
**Sprint 4:** Items 12, 15, 17 (pipeline automation + health checks)
**Sprint 5+:** Items 11, 13, 14, 16, 18 (API + features + observability)

---

## Plan Document Inventory

| Plan ID | File | Status |
|---------|------|--------|
| P1-001 | docs/plans/2026-04-24-001-fix-ci-test-workflow-plan.md | Planned |
| P1-002 | ~~docs/plans/2026-04-24-002-fix-docker-compose-scheduler-plan.md~~ | Removed |
| P1-003 | docs/plans/2026-04-24-003-consolidate-config-sources-plan.md | Planned |
| P1-004 | docs/plans/2026-04-24-004-rewrite-readme-plan.md | Planned |
| P1-005 | docs/plans/2026-04-24-005-add-pyright-ci-plan.md | Planned |
| P1-006 | docs/plans/2026-04-24-006-activate-pre-commit-hooks-plan.md | Planned |
| P2-001 | docs/plans/2026-04-24-007-replace-print-with-logging-plan.md | Planned |
| P2-002 | docs/plans/2026-04-24-008-fix-broad-exception-handling-plan.md | Planned |
| P3-001 | docs/plans/2026-04-04-003-refactor-complete-explorer-decomposition-plan.md | In progress |
| P4-005 | docs/plans/2026-04-24-009-pipeline-health-checks-plan.md | Planned |
| P5-002 | docs/plans/2026-04-24-010-pipeline-health-checks-plan.md | Planned |

---

## Notes

- All implementation plans use TDD (test-first) for code-bearing units.
- Config-only units (README) skip TDD but include verification checklists.
- Existing plans (e.g., explorer decomposition) are referenced rather than duplicated.