A small, developer-focused guide for the mindmodel validator used by reviewers and contributors.
What this validator does
-----------------------
- Validates the repository's mindmodel manifest and evidence against project policies.
- Flags common issues for reviewers (secrets, missing evidence, excessively truncated evidence, policy violations).
Where the manifest lives
------------------------
The canonical manifest is stored at:
.mindmodel/manifest.yaml
Reviewer checklist
------------------
When reviewing mindmodel submissions, make a quick pass over the following items:
1. Secrets: Ensure there are no secrets (API keys, tokens, private credentials) included in the manifest or evidence. If you spot secrets, escalate and remove them immediately.
2. Evidence truncation: Verify that evidence files or snippets are not truncated in a way that removes important context. If evidence is truncated for size, confirm the truncated portion is non-essential and that a pointer to full evidence is provided.
3. Read-only policy: Confirm that the mindmodel only documents read-only artifacts. The validator and reviewers must ensure no actions, credentials, or writable endpoints are exposed.
4. Completeness: Check that required fields from the manifest schema are present and that evidence links to real files or reports in the repository.
Running the validator locally
---------------------------
You can run the validator locally with the provided Python script. Example:
Goal: Implement a lightweight, safe Constraint Validator for the generated .mindmodel/ snapshot plus small CI / config artifacts to validate and integrate the manifest incrementally and safely.
Goal: Integrate a generated .mindmodel/ snapshot safely via an audit-first, incremental approach. Add a report-only validator, CI validation job, and a small set of conservative remediation changes (dev-deps, formatter configs) in separate low-risk PRs.
Purpose: Provide a conservative, report-only validator API that consumes a .mindmodel/ manifest and emits a structured report (missing files, truncated evidence, potential secrets). Implementation will be a safe skeleton (no auto-fixes). The module will expose a function validate_manifest(manifest_path: str, report_only: bool = True) -> dict.
Purpose: Add the generated snapshot (or a sanitized copy) under .mindmodel/ in the repo as read-only content. The manifest should be explicitly marked in-file as "DO NOT EDIT - read-only until validated" and include a small sample of constraints (3-5) for validator development. Do NOT include secrets or lockfiles.
Purpose: Explain how the validator works, where the manifest lives, and reviewer checklist (check for secrets, truncated evidence). This is developer documentation to speed review.
Verify: manual review of the file in the PR.
Commit message suggestion: `docs(mindmodel): add README and reviewer checklist`
These tasks depend on Batch 1 (validator types + sample manifest present).
### Task 2.1: CLI wrapper to run validator
**File:** `scripts/validate_mindmodel.py`
**Test:** `tests/scripts/test_validate_cli.py`
**Depends:** 1.1, 1.2, 1.3
**Effort:** S
Purpose: Provide a tiny CLI that calls the validator and writes a structured JSON report to stdout and to `reports/mindmodel-report-YYYYMMDD.json`. Defaults: report-only = True. This lets local and CI runs use a single entrypoint.
Purpose: Add unit tests that exercise key failure modes: missing files referenced by constraints, truncated evidence excerpts, evidence pointers that look like secrets (simple heuristics), and constraint marked needs-review. These tests will assert the validator reports issues (report-only) but do not raise exceptions.
These are the small, non-invasive repo edits recommended in the design. They depend on Batch 1 tests/tools being present to validate effects.
### Task 3.1: Move test runner to dev dependency (pyproject change)
**File:** `pyproject.toml` (UPDATE)
**Test:** `tests/config/test_pyproject_deps.py`
**Depends:** 1.2
**Effort:** M
Purpose: Remove testing tools (pytest) from top-level production dependencies and document them as dev-dependencies. If the project uses Poetry or PEP 621 style, follow project's existing pattern; if unclear, add a `[tool.dev-deps]` section or a `requirements-dev.txt` and reference it. This change must be small and isolated.
This plan breaks work into four batches: Foundation, Core, Components, Integration/Configs. Each micro-task is small and independently testable. Tests accompany core modules. The validator intentionally avoids reading repository secret files and only scans manifest text and evidence snippets.
Risk mitigation: Keep the change to a single commit; include CI job that still installs test deps for CI runs.
## Batch 1: Foundation (parallel)
Commit message suggestion: `chore(deps): move pytest to dev-dependencies`
- Task 1.1: Manifest loader
- Path: scripts/mindmodel/loader.py
- Test: tests/scripts/mindmodel/test_loader.py
- Behavior: load YAML or JSON manifest, normalize to dict, raise ManifestLoadError on failure
Purpose: Add pre-commit and formatter config stubs (black, ruff, isort) to make future automation deterministic. This does not change code behavior and can be staged in a separate PR.
Purpose: Add a CI workflow that runs the CLI against `.mindmodel/manifest.yaml` and uploads `reports/mindmodel-report-*.json` as an artifact. Important: the job should be non-blocking for merges initially (report-only). Job steps:
- Task 4.1: CI workflow to run validator on PRs and scheduled checks
- Path: .github/workflows/mindmodel-validate.yml
- Behavior: run tests, then run validator against .mindmodel/manifest.yaml if present
Purpose: Add a cron-scheduled workflow to run the validator daily/weekly and produce artifacts, helping detect drift over time. Keep the schedule job report-only at first.
1) PR A (Batch 1 - Validator skeleton + types + tests, no .mindmodel/ content) — Adds validator API and types. (Small, S)
2) PR B (Batch 1 - Add sanitized read-only `.mindmodel/manifest.yaml` + docs) — Separate PR so reviewers can inspect the raw manifest without behavioral changes. (S)
3) PR C (Batch 2 - Add CLI wrapper + validator edge-case tests) — Enables local/CI execution. (S)
4) PR D (Batch 4 - Add CI workflow as report-only) — Hook CI to run the validator and upload reports; do not fail CI yet. (M)
5) PR E (Batch 3 - Move pytest to dev-deps) — Small config change in pyproject; CI continues to install test deps. (M)
6) PR F (Batch 3 - Add pre-commit/formatters) — Non-invasive tooling. (S)
7) PR G (Batch 4 - Add scheduled validation job) — Optional, report-only. (S)
Rationale: each PR is kept small and focused. PR A/B/C/D are prioritized so we have validator + CI reporting quickly without touching production behavior. Remediation changes (E/F) are separate, so reviewers can focus on policy vs. code changes.
---
## Risk mitigation and decisions made
- Validator is report-only by default. Decision: safer to surface issues and build trust before enforcing failures.
- .mindmodel/ files will be added read-only and explicitly labeled in-file and in PR description.
- Move pytest to dev-deps rather than removing from pyproject entirely if project conventions are unclear. Decision: add a `[tool.dev-deps]` or `requirements-dev.txt` depending on project tools; the implementer will choose the minimally invasive approach.
- No automated fixes in validator; only reporting. If trivial YAML path reformatting is desired later, add an opt-in flag after human review.
---
## CI policy / timeline suggestion
- Week 0: Merge PRs A-C (validator, manifest, CLI). CI runs report-only jobs and uploads reports.
- Week 1: Merge PR D (CI workflow) so reports appear in PR runs. Collect feedback and sample manual reviews on 3-5 constraints.
- Week 2: Merge remediation PRs (E/F) as separate changes. Keep CI non-blocking.
- Week 3-4: After confidence is built, update CI job to fail on a small set of clear, high-confidence checks (missing files, secrets) behind a feature flag or branch protection rule.
---
## Files to be added/modified (summary)
- src/validators/mindmodel_validator.py — validator API (S)
- src/validators/types.py — manifest dataclasses/types (S)
- .mindmodel/manifest.yaml — sanitized manifest (S) (read-only)
- thoughts/shared/mindmodel/README.md — developer docs (S)
- scripts/validate_mindmodel.py — CLI wrapper (S)
- .github/workflows/mindmodel-validation.yml — CI workflow (M)
- pyproject.toml — small update to move pytest to dev-deps (M)
- .pre-commit-config.yaml — formatter config (S)
- tests/... corresponding tests for each file (S/M as noted)
---
## Short summary for each microtask (one-line)
- 1.1: validator skeleton exposing validate_manifest(...). (S)
- 1.2: typed manifest models (dataclasses / pydantic). (S)
- 1.3: add sanitized .mindmodel/manifest.yaml read-only snapshot. (S)
- 1.4: developer README with reviewer checklist. (S)
- 2.1: CLI wrapper script to run validator and emit JSON reports. (S)
1. Create the files above in small commits (one micro-task per commit).
2. Run unit tests for each new module as added.
3. Open a small PR with the validator + CI + docs; request reviewers to run the validator locally.
If you'd like, I can now split these microtasks into individual ticket-sized action items (one file + test per task) with ready-to-apply patch templates for each; tell me how many parallel implementers you expect and I will group them into batches accordingly.
