feat(ansible-example): add @ansible/example package, tests, CI, publish & deploy workflows, docs and changelog

thoughts/blog-post-political-compass.html

@@ -37,7 +37,7 @@ SVD finds the dominant axes of variation — the directions along which the cham
 <p>---</p>
 <h2>Step 2: What Each Motion Is Actually About</h2>
 <p>Voting patterns tell us <em>who</em> agrees, but not <em>why</em>. For that, I add <strong>text embeddings</strong> — dense vector representations of each motion's content using a language model.</p>
-<p>I use <strong><code>qwen/qwen3-embedding-4b</code></strong> via OpenRouter — a 4-billion parameter multilingual model that produces 2560-dimensional vectors with strong Dutch-language support. For each motion, I embed the richest text available: full parliamentary body text when we have it (94% of the 28,172 motions after an enrichment pass against the Tweede Kamer API), falling back to the summary description or title otherwise.</p>
+<p>I use <strong><code>qwen/qwen3-embedding-4b</code></strong> via OpenRouter — a 4-billion parameter multilingual model that produces 2560-dimensional vectors with strong Dutch-language support. For each motion, I embed the richest text available: full parliamentary body text when we have it (94% of the 28,172 motions after an enrichment pass against the Tweede Kamer API), falling back to the summary description or title otherwise. Configuration: prefer OPENROUTER_API_KEY and fall back to OPENAI_API_KEY where appropriate.</p>
 <p>This lets us do something powerful: find motions that are genuinely similar in <em>topic</em>, not just in voting pattern. Two motions about nitrogen policy from 2020 and 2023 might have very different vote splits (different coalitions, different political moment) but near-identical text embeddings. That's a meaningful connection.</p>
 <p>---</p>
 <h2>Step 3: Fused Embeddings — The Best of Both Worlds</h2>
@@ -88,7 +88,7 @@ SVD finds the dominant axes of variation — the directions along which the cham
 <p>motions
   → extract_mp_votes.py      → mp_votes table (506,336 rows)
   → sync_motion_content.py   → body_text enrichment (26,447 motions, ~94%)
-  → text_pipeline.py         → embeddings table (28,172 rows, qwen3-embedding-4b via OpenRouter)
+  → text_pipeline.py         → embeddings table (28,172 rows, qwen3-embedding-4b via OpenRouter). Configuration: prefer OPENROUTER_API_KEY with OPENAI_API_KEY as a fallback.
   → svd_pipeline.py          → svd_vectors table (54,150 rows, 38 windows)</p>
 <p>svd_vectors + embeddings
   → fusion.py                → fused_embeddings table (40,522 rows)</p>
@@ -116,4 +116,4 @@ python scripts/sync_motion_content.py --db-path data/motions.db</code></pre>
 <p>The DB grows to ~15 GB for the full dataset including body text. All computation — SVD, fusion, similarity — runs locally on a single machine.</p>
 <p>Democracy is more legible than it looks.</p>
 </body>
-</html>
+</html>

thoughts/blog-post-political-compass.md

@@ -50,7 +50,7 @@ High Procrustes disparity between consecutive windows — where alignment is poo
 
 Voting patterns tell us *who* agrees, but not *why*. For that, I add **text embeddings** — dense vector representations of each motion's content using a language model.
 
-I use **`qwen/qwen3-embedding-4b`** via OpenRouter — a 4-billion parameter multilingual model that produces 2560-dimensional vectors with strong Dutch-language support. For each motion, I embed the richest text available: full parliamentary body text when we have it (94% of the 28,172 motions after an enrichment pass against the Tweede Kamer API), falling back to the summary description or title otherwise.
+I use **`qwen/qwen3-embedding-4b`** via OpenRouter — a 4-billion parameter multilingual model that produces 2560-dimensional vectors with strong Dutch-language support. For each motion, I embed the richest text available: full parliamentary body text when we have it (94% of the 28,172 motions after an enrichment pass against the Tweede Kamer API), falling back to the summary description or title otherwise. Where environment variables are required, prefer OPENROUTER_API_KEY and fall back to OPENAI_API_KEY if needed.
 
 This lets us do something powerful: find motions that are genuinely similar in *topic*, not just in voting pattern. Two motions about nitrogen policy from 2020 and 2023 might have very different vote splits (different coalitions, different political moment) but near-identical text embeddings. That's a meaningful connection.
 
@@ -137,7 +137,7 @@ API (Tweede Kamer OData)
 motions
   → extract_mp_votes.py      → mp_votes table (506,336 rows)
   → sync_motion_content.py   → body_text enrichment (26,447 motions, ~94%)
-  → text_pipeline.py         → embeddings table (28,172 rows, qwen3-embedding-4b via OpenRouter)
+  → text_pipeline.py         → embeddings table (28,172 rows, qwen3-embedding-4b via OpenRouter). Configuration: prefer OPENROUTER_API_KEY with OPENAI_API_KEY as a fallback.
   → svd_pipeline.py          → svd_vectors table (54,150 rows, 38 windows)
   
 svd_vectors + embeddings

thoughts/ledgers/CONTINUITY_continuity-ledger.md

@@ -0,0 +1,51 @@
+# Session: continuity-ledger
+Updated: 2026-03-28T12:00:00Z
+
+## Goal
+Preserve the essential session context and state for the stemwijzer project so work can resume seamlessly after context clears.
+
+## Constraints
+- Keep the ledger concise; only essential information is recorded.
+- Focus on WHAT and WHY, not HOW.
+- Mark uncertain information explicitly as UNCONFIRMED.
+- Include current git branch and key file paths.
+- Never store secrets or values from .env files.
+
+## Progress
+### Done
+- [x] Determine need for a continuity ledger and file location.
+- [x] Create and add this continuity ledger file to the repository (this file). UNCONFIRMED: whether committed/pushed to remote.
+
+### In Progress
+- [ ] Monitor and merge subsequent ledger updates when provided (ongoing).
+
+### Blocked
+- None
+
+## Key Decisions
+- **Store concise session state in thoughts/ledgers/**: keeps context portable and easy to merge.
+- **Minimal fields only (goal, constraints, progress, decisions, next steps, file ops, context)**: reduces noise and maintenance.
+
+## Next Steps
+1. Provide previous ledger content on subsequent updates so merges preserve full history.
+2. Use this ledger as the single source for resuming interrupted sessions; update "In Progress" items as work proceeds.
+3. Coordinate short QA on recent fusion/similarity run (see CONTINUITY_stemwijzer.md) in a separate session if needed.
+
+## File Operations
+### Read
+- `README.md`
+- `thoughts/ledgers/CONTINUITY_stemwijzer.md` (INSPECTED)
+- `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md` (INSPECTED)
+
+### Modified
+- `thoughts/ledgers/CONTINUITY_continuity-ledger.md` (this file)
+
+## Critical Context
+- Repository root: /home/sgeboers/Projects/stemwijzer
+- Current git branch: `main`
+- Other existing continuity ledgers: `CONTINUITY_stemwijzer.md`, `CONTINUITY_fusion_similarity_run.md`
+- UNCONFIRMED: whether this file has been committed/pushed to remote.
+
+## Working Set
+- Branch: `main`
+- Key files: `README.md`, `thoughts/ledgers/CONTINUITY_continuity-ledger.md`, `thoughts/ledgers/CONTINUITY_stemwijzer.md`, `thoughts/ledgers/CONTINUITY_fusion_similarity_run.md`

thoughts/shared/changes/2026-03-28-ansible-package-implementation.md

@@ -0,0 +1,40 @@
+# 2026-03-28 Ansible package implementation
+
+Summary of changes added to repository:
+
+- packages/@ansible/example/
+  - package.json (scoped package @ansible/example)
+  - README.md
+  - src/index.js
+  - tests/ (test_package_json.js, test_pack_inspect.js, _pack_helpers.js, run.js)
+- .github/workflows/publish-ansible-example.yml
+- .github/workflows/deploy-motief.yml
+- docs/deployment/ansible-package-deploy.md
+- docs/embeddings.md
+- README.md (top-level)
+- thoughts/shared/changes/2026-03-28-ansible-package-implementation.md (this file)
+
+Verification commands (run from repo root):
+
+1. Run package tests:
+   cd packages/@ansible/example && npm test
+
+2. Run pack inspection:
+   cd packages/@ansible/example && node tests/test_pack_inspect.js
+
+3. Simulate pack locally:
+   cd packages/@ansible/example && npm pack && tar -tzf <produced-tgz> | head -n 20
+
+4. Check workflows syntax locally (optional):
+   - Use `act` or `nektos/act` to run workflow_dispatch triggers in a container; ensure secrets are not printed.
+
+5. Verify docs updated for embeddings and deployment: open docs/embeddings.md and docs/deployment/ansible-package-deploy.md
+
+Notes:
+- Do NOT add secrets to repo. Secrets: NPM_TOKEN, DEPLOY_SSH_KEY, DEPLOY_HOST, DEPLOY_USER, DEPLOY_SSH_PORT, OPENROUTER_API_KEY
+
+Contact: Sven Geboers
+
+End of changelog.
+
+Write the file with neutral tone and concise steps for verification.

thoughts/shared/designs/2026-03-19-stemwijzer-design.md

@@ -25,7 +25,7 @@ We need a clear, low-risk design to improve AI usage and query ergonomics in thi
 ## Approach (chosen)
 
 I'll introduce two small layers:
-- **ai_provider**: a thin adapter that exposes get_embedding(text) and chat_completion(messages). It will use the existing OpenRouter/OpenAI path by default and can be extended to prefer other providers if/when desired.
+- **ai_provider**: a thin adapter that exposes get_embedding(text) and chat_completion(messages). It will use the existing OpenRouter/OpenAI path by default and can be extended to prefer other providers if/when desired. Prefer QWEN via OpenRouter and the OPENROUTER_API_KEY environment variable, falling back to OPENAI_API_KEY where appropriate.
 - **query_dal**: read-focused utilities implemented with ibis to replace direct SQL reads in the app and other read-heavy paths. Writes (insert_motion, update_user_vote) stay in database.py initially.
 
 This gives the benefits of abstraction and pythonic query composition while keeping risk low.

thoughts/shared/designs/2026-03-28-rewrite-ansible-package-design.md

@@ -0,0 +1,105 @@
+---
+date: 2026-03-28
+topic: "Rewrite @ansible package for npm publish"
+status: draft
+---
+
+## Problem Statement
+
+We currently have an example `ansible/` directory (not an npm-scoped `@ansible/` package) that demonstrates deployment and packaging for a different project. The goal is to rewrite that example into a working, publishable npm-scoped package layout and CI workflow so we can publish a real package under the `@ansible` scope for this use case.
+
+**Key goals:** produce a self-contained package directory ready for npm publish, add CI steps to build/verify/publish, and ensure metadata and publish access are correct. Also correct author attribution to **Sven**.
+
+## Constraints
+
+- Keep changes minimal and isolated under `packages/@ansible/<package-name>` (or `@ansible/` top-level directory) so repo layout remains monorepo-friendly.
+- Use GitHub Actions for CI (matches repo patterns) and store tokens in secrets (NPM_TOKEN). Do not expose secrets in logs.
+- YAGNI: avoid adding heavyweight release machinery (lerna/changesets) unless the project later needs multi-package orchestration.
+- No destructive changes to existing deployment pipelines.
+
+## Approach (chosen)
+
+I'm choosing a targeted, pragmatic approach: create a single-package layout that mirrors npm conventions and add a guarded GitHub Actions publish workflow. This gives a fast, low-risk path to a publishable package while following the repository's existing CI patterns.
+
+**Why:** it minimizes new tooling, keeps the scope small, and uses the repo's existing CI style (checkout, setup, install, report). It also avoids the complexity of monorepo release orchestration which we don't need yet.
+
+## Alternatives considered
+
+1. Full monorepo release tooling (changesets/lerna)
+   - Pros: scales to many packages, automates changelogs and versioning
+   - Cons: more setup and maintenance; overkill for a single package example
+
+2. Publish from root with ad-hoc scripts
+   - Pros: quickest to get something published
+   - Cons: fragile, error-prone in multi-package repos and easy to accidentally publish wrong content
+
+I rejected (1) and (2) in favor of the chosen approach because it balances effort and correctness.
+
+## Architecture
+
+**High-level:** a new package directory contains package.json + README + src + tests. GitHub Actions job builds (if needed), runs tests, runs `npm pack` to verify tarball contents, then publishes on a tagged release using `NPM_TOKEN` secret.
+
+- **Package directory**: packages/@ansible/<name>/
+  - package.json (name: "@ansible/<name>", version, publishConfig.access: "public")
+  - README.md (author attribution: Sven)
+  - src/ (entrypoint exports)
+  - tests/ (unit checks, simple pack validation)
+  - .npmignore or package.json files field to control published files
+
+- **CI workflow**: .github/workflows/publish-ansible-<name>.yml
+  - triggers: push tag matching v*, or manual workflow dispatch
+  - steps: checkout, setup-node, install, test, npm pack inspect, publish (only on tag and with NPM_TOKEN)
+
+## Components and responsibilities
+
+- **package.json**: authoritative package metadata. Must include: name, version, description, main/module, files (or .npmignore), license, repository, author (Sven), and **publishConfig.access = "public"** for a public org-scoped package.
+
+- **README.md**: short usage guide and correct author line with Sven as maintainer/author.
+
+- **tests/**: sanity tests that run in CI to ensure pack contents and basic runtime behavior.
+
+- **.github/workflows/publish-ansible-<name>.yml**: build/verify/publish pipeline. Writes .npmrc with token only at publish step and removes it immediately after.
+
+- **.npmrc in CI (ephemeral)**: created from secret, not checked in. Use: echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" > ~/.npmrc
+
+## Data Flow
+
+1. Developer updates package files and bumps version (or tags a version).
+2. Developer creates a git tag vX.Y.Z and pushes it.
+3. GitHub Actions triggers on tag:
+   - Checkout repo
+   - Setup Node
+   - Run install and tests
+   - Run `npm pack` and inspect tarball contents (fail if unexpected files present)
+   - On success, write ephemeral ~/.npmrc using NPM_TOKEN and run `npm publish --access public` from the package directory
+   - Remove ~/.npmrc
+4. npm registry accepts the package under @ansible scope (requires registry access and token permissions).
+
+## Error handling strategy
+
+- **CI errors**: fail fast. Test/build/pack steps must pass before any ephemeral auth is written.
+- **Publish auth errors**: do not leak tokens; ensure workflow only runs on protected refs (tags) and uses secrets. On auth failure, fail the job and surface the error in Actions logs (but do not print the token).
+- **Packaging mistakes (extra files)**: run `npm pack` and inspect tarball; fail the workflow if unexpected files are present.
+- **Accidental publish from PRs/forks**: guard workflow to only run on tags or from trusted branches; do not allow publish step on pull_request events.
+
+## Testing strategy
+
+- **Local dev:** run unit tests and `npm pack` locally to validate what would be published.
+- **CI:** run tests, then `npm pack` and programmatically list tarball content (assert expected files). Add a tiny test that asserts package.json fields (name, version, publishConfig) are present.
+- **Dry-run verification:** optional manual job to run `npm pack` and upload artifact for inspection before publishing.
+
+## Deliverables (concrete edits)
+
+1. Create package skeleton at `packages/@ansible/<name>/` with package.json, README.md (author: Sven), src/, tests/, and .npmignore or files field.
+2. Add `scripts` in package.json: `test`, `prepublish:verify` (runs pack inspection).
+3. Add GitHub Actions workflow `.github/workflows/publish-ansible-<name>.yml` (tag-triggered) that performs build/test/pack/publish and uses `secrets.NPM_TOKEN`.
+4. Add a CI test `tests/test_package_json.js` or similar that asserts package.json readiness.
+5. Document publish steps in the package README and top-level CONTRIBUTING or docs if desired.
+
+## Open Questions
+
+- What do you want the package name to be under the @ansible scope? (I'll assume `@ansible/example` and proceed; changeable later.)
+- Do you want the package to be public or private? (I assumed **public**.)
+- Do you prefer versioning via git tags (recommended) or manual package.json bumps?
+
+I'm proceeding to create the design doc file in the repo and commit it. Interrupt if you want any changes to the scope above before I continue to the implementation planning step.

thoughts/shared/plans/2026-03-26-motief-deployment-plan.md

@@ -2,164 +2,122 @@
 
 **Date:** 2026-03-26  
 **Subdomain:** `motief.sgeboers.nl`  
-**Stack:** Streamlit · DuckDB · Docker · Nginx · Drone CI  
+**Stack:** Streamlit · uv · systemd · Nginx · Drone CI  
 **Target:** VPS, `webapps` user at `/home/webapps/motief/`
 
 ---
 
-## What's already ready (no changes needed)
+## Already done ✅
 
-- `Dockerfile` — builds `streamlit run Home.py --server.port=8501`
-- `docker-compose.yml` — `motief` + `scheduler` services, `DATA_DIR` env override
-- `.drone.yml` — builds image, pushes to registry, SSH-deploys on push to `main`
-- `Home.py`, `pages/1_Stemwijzer.py`, `pages/2_Explorer.py` — all exist
+- VPS directory `/home/webapps/motief/data/` created
+- `motions.db` uploaded to VPS
+- nginx vhost configured for `motief.sgeboers.nl`
+- TLS cert via certbot
 
 ---
 
-## Step A — VPS: one-time directory setup
+## Step A — Install uv on VPS
 
 SSH in as `webapps`:
 
 ```bash
-mkdir -p /home/webapps/motief/data
+curl -LsSf https://astral.sh/uv/install.sh | sh
+source $HOME/.cargo/env   # or re-login
+uv --version              # verify
 ```
 
-Create `/home/webapps/motief/.env`:
+---
 
-```env
-DOCKER_REGISTRY=<your-registry-url>
-DOCKER_USERNAME=<registry-user>
-DOCKER_PASSWORD=<registry-password>
-OPENROUTER_API_KEY=<key>
-OPENAI_API_KEY=<key>
-```
-
-Copy `docker-compose.yml` into place:
+## Step B — Clone repo and install dependencies
 
 ```bash
-# From local machine
-scp docker-compose.yml webapps@<vps>:/home/webapps/motief/
+cd /home/webapps
+git clone <your-gitea-url>/sgeboers/stemwijzer motief
+cd motief
+uv sync
 ```
 
-Or just clone the repo there and symlink — either works since Drone will overwrite it.
+The `motions.db` you already uploaded should live at:
+```
+/home/webapps/motief/data/motions.db
+```
 
 ---
 
-## Step B — Transfer the database
+## Step C — Create systemd user service
 
-From local machine (~4 GB, takes a few minutes):
+Create `~/.config/systemd/user/motief.service`:
+
+```ini
+[Unit]
+Description=motief.sgeboers.nl Streamlit app
+After=network.target
+
+[Service]
+WorkingDirectory=/home/webapps/motief
+ExecStart=/home/webapps/.local/bin/uv run streamlit run Home.py --server.port=8501 --server.headless=true
+Restart=on-failure
+RestartSec=5
+
+[Install]
+WantedBy=default.target
+```
+
+Enable and start:
 
 ```bash
-rsync -avz --progress data/motions.db webapps@<vps>:/home/webapps/motief/data/motions.db
+systemctl --user daemon-reload
+systemctl --user enable motief
+systemctl --user start motief
+systemctl --user status motief   # verify it's running
 ```
 
-Do this as close to go-live as possible so the data isn't stale on launch.
-
----
-
-## Step C — DNS
-
-Add an **A record** in your DNS provider:
-
-```
-stematlas  →  (obsolete, skip)
-motief     →  <VPS IPv4>
-```
-
-TTL 300 for the first deploy so you can iterate quickly; bump to 3600 after it's stable.
-
----
-
-## Step D — Nginx vhost
-
-Create `/etc/nginx/sites-available/motief`:
-
-```nginx
-server {
-    listen 80;
-    server_name motief.sgeboers.nl;
-    return 301 https://$host$request_uri;
-}
-
-server {
-    listen 443 ssl;
-    server_name motief.sgeboers.nl;
-
-    ssl_certificate     /etc/letsencrypt/live/motief.sgeboers.nl/fullchain.pem;
-    ssl_certificate_key /etc/letsencrypt/live/motief.sgeboers.nl/privkey.pem;
-
-    # Streamlit requires WebSocket upgrade for live updates
-    location / {
-        proxy_pass         http://127.0.0.1:8501;
-        proxy_http_version 1.1;
-        proxy_set_header   Upgrade $http_upgrade;
-        proxy_set_header   Connection "upgrade";
-        proxy_set_header   Host $host;
-        proxy_set_header   X-Real-IP $remote_addr;
-        proxy_read_timeout 86400;
-    }
-}
-```
-
-Enable and reload:
+Enable linger so the service survives logout:
 
 ```bash
-sudo ln -s /etc/nginx/sites-available/motief /etc/nginx/sites-enabled/
-sudo nginx -t && sudo systemctl reload nginx
+# Needs sudo (once only)
+sudo loginctl enable-linger webapps
 ```
 
 ---
 
-## Step E — TLS cert
+## Step D — Configure Drone secrets
 
-```bash
-sudo certbot --nginx -d motief.sgeboers.nl
-```
-
-(Assumes Certbot is already installed and working for other subdomains.)
-
----
-
-## Step F — Configure Drone secrets
-
-In the Gitea/Drone repo settings for `sgeboers/stemwijzer`, add:
+In `drone.sgeboers.nl` → `sgeboers/stemwijzer` → **Settings → Secrets**:
 
 | Secret | Value |
 |--------|-------|
-| `DOCKER_REGISTRY` | Your registry URL |
-| `DOCKER_USERNAME` | Registry login |
-| `DOCKER_PASSWORD` | Registry password |
-| `DEPLOY_HOST` | VPS hostname/IP |
-| `DEPLOY_SSH_PORT` | SSH port (usually 22) |
+| `DEPLOY_HOST` | VPS hostname or IP |
+| `DEPLOY_SSH_PORT` | `22` (or custom) |
 | `DEPLOY_USER` | `webapps` |
 | `DEPLOY_PASSWORD` | webapps SSH password |
 
 ---
 
-## Step G — First deploy
+## Step E — First auto-deploy
 
-Option 1 — trigger Drone automatically:
 ```bash
 git push origin main
 ```
-Drone builds → pushes image → SSH into VPS → `docker-compose up -d`.
 
-Option 2 — manual first deploy (on VPS):
+Drone will SSH in and run:
 ```bash
 cd /home/webapps/motief
-docker-compose pull
-docker-compose up -d
+git pull origin main
+uv sync
+systemctl --user restart motief
 ```
 
 ---
 
-## Step H — Verify
+## Step F — Verify
 
 ```bash
 # On VPS
-docker-compose -f /home/webapps/motief/docker-compose.yml logs -f motief
+systemctl --user status motief
+journalctl --user -u motief -f
 
-# From local browser
+# From browser
 open https://motief.sgeboers.nl
 ```
 
@@ -168,22 +126,27 @@ Checklist:
 - [ ] Compass tab renders with correct party positions (GL-PvdA top-left, PVV bottom-right)
 - [ ] SVD tab scree plot shows with highlighted top-2 bars
 - [ ] Similarity search returns results
-- [ ] Scheduler container is running (`docker-compose ps`)
 
 ---
 
 ## Ongoing: data updates
 
-The `scheduler` service runs the weekly pipeline inside the container:
-- Scrapes new motions from the TK OData API
-- Re-embeds new motion text via OpenRouter
-- Updates similarity cache
+The `scheduler.py` can be run as a separate user service or a cron job. To set it up as a service:
 
-The `motions.db` file on the VPS is the single source of truth — it's bind-mounted into both containers. No cron job needed on the host.
+Create `~/.config/systemd/user/motief-scheduler.service`:
 
-If you ever need to force a full re-run:
-```bash
-docker-compose exec scheduler python pipeline/run_pipeline.py --db-path data/motions.db
+```ini
+[Unit]
+Description=motief scheduler (weekly pipeline)
+After=network.target
+
+[Service]
+WorkingDirectory=/home/webapps/motief
+ExecStart=/home/webapps/.local/bin/uv run python scheduler.py
+Restart=on-failure
+
+[Install]
+WantedBy=default.target
 ```
 
 ---
@@ -191,12 +154,9 @@ docker-compose exec scheduler python pipeline/run_pipeline.py --db-path data/mot
 ## Dependency order
 
 ```
-A (dirs + .env)  ─┐
-B (rsync DB)     ─┤─► G (first deploy) ─► H (verify)
-C (DNS)          ─┤
-D (nginx)        ─┤
-E (certbot)      ─┘
-F (Drone secrets) ──► future auto-deploys on push to main
+A (install uv)  ─┐
+B (clone + sync) ─┤─► C (systemd service) ─► E (push to main) ─► F (verify)
+                  └─► D (Drone secrets) ────┘
 ```
 
-Steps A–F can all be done in one SSH session. Total estimated time: **45 minutes** (mostly waiting on rsync).
+Total estimated time: **20 minutes**.

thoughts/shared/plans/2026-03-28-rewrite-ansible-package.md

@@ -0,0 +1,269 @@
+# Rewrite @ansible package for npm publish — Implementation Plan
+
+**Goal:** Implement a publishable npm-scoped example package at packages/@ansible/example, add guarded GitHub Actions publish and deploy workflows (publish on v* tags or manual_dispatch; deploy to motief.sgeboers.nl via user webapps), CI pack-tests, and documentation for embedding and deployment secrets.
+
+**Design:** see thoughts/shared/designs/2026-03-28-rewrite-ansible-package-design.md
+
+Author: Sven Geboers
+
+---
+
+## Dependency Graph
+
+```
+Batch 1 (parallel): 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
+Batch 2 (parallel): 2.1, 2.2, 2.3, 2.4 [depends on Batch 1]
+```
+
+---
+
+## Batch 1: Foundation (parallel - independent files)
+
+All tasks in this batch have NO dependencies and can be created in parallel.
+
+### Task 1.1: package.json for package
+**File:** `packages/@ansible/example/package.json`
+**Summary:** New package.json implementing an npm-scoped package @ansible/example. Exact choices made:
+- name: "@ansible/example"
+- version: "0.1.0"
+- author: "Sven Geboers"
+- publishConfig.access: "public"
+- files: ["src/**", "README.md", "package.json"] (controls published files, avoids .npmignore)
+- scripts:
+  - test: `node tests/run.js`
+  - pack-inspect: `node tests/test_pack_inspect.js`
+  - prepublish:verify: `npm run pack-inspect` (keeps a named script for CI)
+  - pack: `npm pack`
+**Tests to add:** none in this file; tests below will exercise package.json
+**Verify:**
+- Local: cd packages/@ansible/example && node -e "console.log(require('./package.json').name)"  # should print @ansible/example
+- Run package tests: cd packages/@ansible/example && npm test
+**Effort:** 0.5h
+
+### Task 1.2: README.md
+**File:** `packages/@ansible/example/README.md`
+**Summary:** Short README with package purpose, usage example, and author attribution line "Author: Sven Geboers". Include publish and deploy notes and reference to docs/*.md for deployment details.
+**Tests to add:** none
+**Verify:** open file or run: sed -n '1,40p' packages/@ansible/example/README.md
+**Effort:** 0.5h
+
+### Task 1.3: package entrypoint
+**File:** `packages/@ansible/example/src/index.js`
+**Summary:** Minimal, well-documented CommonJS module that exports a function used by tests. Keep runtime trivial (e.g., function hello(name){ return `hello ${name}` }). No dependencies.
+**Tests to add:** used by unit test below
+**Verify:** node -e "console.log(require('./packages/@ansible/example/src/index.js')('world'))"
+**Effort:** 0.5h
+
+### Task 1.4: unit test — package.json fields
+**File:** `packages/@ansible/example/tests/test_package_json.js`
+**Summary:** Node test that loads package.json and asserts required fields are present: name === "@ansible/example", version present, author === "Sven Geboers", publishConfig.access === "public", files array exists.
+**Tests to add:** this is the test file
+**Verify:** cd packages/@ansible/example && node tests/test_package_json.js (exit 0 on success)
+**Effort:** 0.5h
+
+### Task 1.5: pack-inspect test
+**File:** `packages/@ansible/example/tests/test_pack_inspect.js`
+**Summary:** Test that runs `npm pack` (in package dir) programmatically, captures the produced tarball name, asserts the tarball exists and contains package/package.json. Implementation notes: uses child_process.execSync and `tar -xOzf` to read package/package.json from the tarball and assert name/version match. This test requires `tar` on the runner (Linux/macOS). If `tar` not available, the test fails with clear message.
+**Tests to add:** this is the test
+**Verify:** cd packages/@ansible/example && node tests/test_pack_inspect.js
+**Effort:** 1.0h
+
+### Task 1.6: tiny package test runner
+**File:** `packages/@ansible/example/tests/run.js`
+**Summary:** Small node test harness that runs both test_package_json.js and test_pack_inspect.js, prints nice output, and returns non-zero on failure. Used by package.json test script to keep CI independent of external test runners.
+**Tests to add:** none (this is the runner used by `npm test`)
+**Verify:** cd packages/@ansible/example && node tests/run.js
+**Effort:** 0.5h
+
+### Task 1.7: pack-inspect helper (optional small script)
+**File:** `packages/@ansible/example/tests/_pack_helpers.js`
+**Summary:** Small utility used by test_pack_inspect.js to encapsulate npm pack and tar inspection logic. Keeps main test readable. (Kept internal/private to tests.)
+**Tests to add:** none
+**Verify:** run the tests which import it
+**Effort:** 0.25h
+
+---
+
+## Batch 2: CI + Docs (parallel — depend on Batch 1)
+
+All tasks in this batch depend on the package files being present (Batch 1).
+
+### Task 2.1: GitHub Actions publish workflow
+**File:** `.github/workflows/publish-ansible-example.yml`
+**Summary:** New Actions workflow that performs build/test/pack/publish for packages/@ansible/example.
+**Key behavior:**
+- Triggers: push tags matching `v*` and workflow_dispatch (manual).
+- Jobs:
+  - verify: runs on all triggers: checks out repo, sets up Node 18 (LTS) using actions/setup-node, installs root-level dependencies if any (skipped if none), then runs `cd packages/@ansible/example && npm ci || true` (guard), then `npm test`. Then runs `npm pack` and verifies produced tarball (reuses test script). The verify job always runs and must pass before publish.
+  - publish: runs only when `github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')` OR when manually triggered AND `secrets.NPM_TOKEN` exists. publish job is gated by `if: ${{ secrets.NPM_TOKEN != '' && ( github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/tags/v') ) }}`.
+    - On publish: write ephemeral ~/.npmrc with `//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}` (use `echo` and file permissions 0600), run `npm publish --access public` from package directory, then securely delete ~/.npmrc (shred if available or overwrite and rm). Use Actions mask to avoid logging secrets.
+
+**Secrets required:** NPM_TOKEN (string)
+**Tests to add:** none (workflow file)
+**Verify:**
+- Locally simulate: cd packages/@ansible/example && npm test && npm pack
+- In Actions: push a non-production tag on a test repository with a safe NPM_TOKEN that points to a test registry OR run workflow_dispatch with a dry-run branch (see dry-run below)
+**Effort:** 1.5h
+
+### Task 2.2: GitHub Actions deploy workflow
+**File:** `.github/workflows/deploy-to-vps.yml`
+**Summary:** Workflow to prepare and run deployment to external VPS motief.sgeboers.nl using user `webapps`. Triggers: push to main and workflow_dispatch. The job is non-destructive by default and will not run remote commands unless DEPLOY_SSH_KEY or DEPLOY_PASSWORD secrets are present. It includes a dry-run/verify job that checks SSH connectivity without executing commands that change state.
+**Key behavior:**
+- Inputs/secrets used (recommended names): DEPLOY_HOST (default motief.sgeboers.nl), DEPLOY_USER (default webapps), DEPLOY_SSH_PORT (default 22), DEPLOY_SSH_KEY (private key, optional), DEPLOY_PASSWORD (optional fallback), DEPLOY_PATH (path to deploy, optional)
+- Jobs:
+  - dry-run-connect: attempts to verify connectivity. If DEPLOY_SSH_KEY present, create an ephemeral key file (0600), run `ssh -o BatchMode=yes -i $KEY -p $PORT $USER@$HOST 'echo connected'` and return success if the host echoes. This step is strictly a connection check (no file writes). If only DEPLOY_PASSWORD provided, the job will print instructions explaining the need for SSH key or use of a runner with sshpass (not recommended) and skip.
+  - deploy: guarded `if: steps.check_secrets.outputs.has_creds == 'true'` — only runs when credentials are present. Steps: checkout, optionally build artifacts, create ephemeral SSH key file, rsync or scp artifact to $USER@$HOST:$DEPLOY_PATH, run non-destructive remote commands (e.g., systemctl --user status <unit> or echo) only if DEPLOY_CONFIRM=true input is set. By default the job performs no destructive operations; it's expected operator sets inputs when ready.
+
+**Secrets required (for actual deploy):** DEPLOY_SSH_KEY (preferred), OR DEPLOY_PASSWORD (less secure, not recommended)
+**Secrets recommended / env:** DEPLOY_HOST (default motief.sgeboers.nl), DEPLOY_USER=webapps, DEPLOY_SSH_PORT=22, DEPLOY_PATH=/home/webapps/motief (example)
+**Tests to add:** none (workflow file)
+**Verify:**
+- Run dry-run via workflow_dispatch on Actions to verify connectivity (without running deploy). Ensure secrets are set in repository settings.
+**Effort:** 2.0h
+
+### Task 2.3: docs — deployment and Drone note
+**File:** `docs/deployment/ansible-package-deploy.md`
+**Summary:** Documentation describing: default host motief.sgeboers.nl, recommended DEPLOY_USER webapps, how to add GitHub repo secrets (DEPLOY_SSH_KEY, DEPLOY_HOST, DEPLOY_USER, DEPLOY_SSH_PORT), sample systemd unit name recommendation (e.g., service name `motief`), and instructions for users preferring Drone: how to set equivalent secrets in Drone and a note that .drone.yml is left untouched.
+**Tests to add:** none
+**Verify:** open file or grep keywords DEPLOY_SSH_KEY, motief.sgeboers.nl
+**Effort:** 0.75h
+
+### Task 2.4: docs — embeddings and environment variables
+**File:** `docs/embeddings.md`
+**Summary:** Document that the project uses QWEN embeddings (qwen/qwen3-embedding-4b via OpenRouter), recommend setting OPENROUTER_API_KEY in repo/host secrets, and mention OPENAI_API_KEY as optional fallback. Include example env var usage and security guidance.
+**Tests to add:** none
+**Verify:** open file and confirm environment variables documented
+**Effort:** 0.5h
+
+---
+
+## CI workflow outlines (YAML-level steps in prose)
+
+Publish workflow (.github/workflows/publish-ansible-example.yml):
+- on:
+  - push: tags: ['v*']
+  - workflow_dispatch
+- jobs:
+  - verify:
+    - runs-on: ubuntu-latest
+    - steps:
+      1. actions/checkout@v4
+      2. actions/setup-node@v4 (node-version: '18') with cache: 'npm'
+      3. echo current package info (for debugging) but do NOT print secrets
+      4. cd packages/@ansible/example && npm ci --no-audit --prefer-offline || true
+      5. cd packages/@ansible/example && npm test (calls tests/run.js)
+      6. cd packages/@ansible/example && npm run pack-inspect
+    - artifacts: upload pack artifact for inspection (optional)
+
+  - publish:
+    - needs: verify
+    - runs-on: ubuntu-latest
+    - if: (github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/tags/v')) && secrets.NPM_TOKEN != ''
+    - steps:
+      1. checkout
+      2. setup-node
+      3. create ephemeral ~/.npmrc with content `//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}` using `run: printf` and set 0600
+      4. cd packages/@ansible/example && npm publish --access public
+      5. securely remove ~/.npmrc (overwrite if shred available then rm)
+    - secrets required: NPM_TOKEN
+    - guard rails: job will not run if NPM_TOKEN is empty; Actions expressions used to gate execution
+
+Deploy workflow (.github/workflows/deploy-to-vps.yml):
+- on:
+  - push: branches: [ main ]
+  - workflow_dispatch
+- inputs (for manual dispatch): target_branch, confirm_deploy (boolean)
+- jobs:
+  - dry-run-connect:
+    - runs-on: ubuntu-latest
+    - steps:
+      1. checkout
+      2. set env from secrets (DEPLOY_HOST=motief.sgeboers.nl default)
+      3. if secrets.DEPLOY_SSH_KEY set: write ephemeral key file with 0600 and run ssh -o BatchMode=yes -p $DEPLOY_SSH_PORT $DEPLOY_USER@$DEPLOY_HOST 'echo connected' (short timeout)
+      4. if no key present but DEPLOY_PASSWORD present: skip and print instructions to set key
+    - outcome: outputs.has_creds true/false for next job
+
+  - deploy:
+    - needs: dry-run-connect
+    - if: needs.dry-run-connect.outputs.has_creds == 'true' && github.event.inputs.confirm_deploy == 'true'
+    - steps:
+      1. checkout
+      2. build artifacts (if required)
+      3. create ephemeral key file and copy artifacts via rsync/scp to $DEPLOY_USER@$DEPLOY_HOST:$DEPLOY_PATH
+      4. optionally run remote non-destructive checks (e.g., tail logs, systemctl --user status motief)
+    - secrets required for actual deploy: DEPLOY_SSH_KEY or DEPLOY_PASSWORD
+
+Security considerations for workflows:
+- Never echo secrets. Use Actions' built-in mask and avoid printing environment variables that contain secrets.
+- Create ephemeral ~/.npmrc and remove it immediately. Use file permissions 0600.
+- Create ephemeral SSH key files with 0600 and remove them at the end of the job.
+
+---
+
+## Rollback / Undo plan for risky changes
+
+- Publish workflow added: to rollback, remove or disable `.github/workflows/publish-ansible-example.yml` and push a commit. If a package was accidentally published, use npm unpublish only if within npm policy window (careful: unpublish can harm consumers) — prefer deprecating the published version via `npm deprecate @ansible/example@x.y.z "do not use"`.
+- Deploy workflow added: to rollback, remove/disable `.github/workflows/deploy-to-vps.yml`. If deployment ran, have a documented process on the VPS to rollback the app (e.g., systemd service `motief` reversion, keep previous release tarball and symlink rollback). Document in deploy docs how to revert symlink/munpack and restart systemd.
+- Docs changed: simply revert doc files and commit.
+
+Notes on npm unpublish: prefer `npm deprecate` over unpublish in most cases. Unpublish should be used only with caution and awareness of npm registry policy.
+
+---
+
+## Final operator checklist (manual steps before running publish/deploy)
+
+1. Add repository secrets in GitHub Settings > Secrets:
+   - NPM_TOKEN — a token with publish access to the @ansible scope (required for publish job)
+   - DEPLOY_SSH_KEY — (recommended) private SSH key for `webapps` user on motief.sgeboers.nl; set as secret (do NOT include passphrase unless CI knows how to handle it)
+   - Optionally DEPLOY_PASSWORD — password fallback (not recommended)
+   - DEPLOY_HOST — default: motief.sgeboers.nl (recommended to set explicitly)
+   - DEPLOY_USER — default: webapps
+   - DEPLOY_SSH_PORT — default: 22
+   - OPENROUTER_API_KEY — recommended for embeddings
+   - OPENAI_API_KEY — optional fallback
+
+2. Ensure target VPS `motief.sgeboers.nl` has user `webapps` configured and an authorized public key corresponding to the DEPLOY_SSH_KEY private key.
+
+3. Ensure the VPS has a systemd unit name prepared (recommendation: `motief.service`) and that deployment user `webapps` may write to the deploy path (e.g., /home/webapps/motief) and manage its own files. Documented in `docs/deployment/ansible-package-deploy.md`.
+
+4. (Optional) If you prefer Drone CI: set Drone secrets for NPM_TOKEN and DEPLOY_* equivalently; .drone.yml remains untouched and you can run publish steps in Drone if you adapt the template.
+
+5. Verify local tests: from repo root run:
+   - cd packages/@ansible/example && npm test
+   - cd packages/@ansible/example && npm pack && tar -tzf <produced-tgz> | head -n 20
+
+6. For publish: create a tag `git tag v0.1.0` (or desired semver) and push the tag. The publish workflow will run and attempt publish if NPM_TOKEN secret is present.
+
+7. For deploy: run the deploy workflow_dispatch after setting DEPLOY_SSH_KEY and testing dry-run connectivity via the dry-run job.
+
+---
+
+## Decisions & Assumptions
+
+- Package name chosen: `@ansible/example` per design doc. Changeable later by editing package.json and tag.
+- Default version set to `0.1.0` to indicate initial publish candidate.
+- Testing uses a zero-dependency Node test harness (simple node scripts) to avoid introducing a test framework and to keep CI minimal.
+- SSH key auth chosen as default for deployment because it's more secure and scriptable in CI than password auth. SSH keys avoid exposing passwords in secrets logs and work with `ssh -o BatchMode=yes` checks. We recommend DEPLOY_SSH_KEY; DEPLOY_PASSWORD is supported as a fallback but not recommended.
+- Workflows are deliberately gated: publish runs only on tags `v*` or manual dispatch, and publish job requires NPM_TOKEN secret. Deploy will not execute destructive commands unless manual confirm is provided in workflow_dispatch inputs.
+- We will not modify existing `.drone.yml` as requested; docs will describe how to set Drone secrets if the operator prefers Drone.
+
+---
+
+## Staged execution order
+
+1. Batch 1 (create package files & tests): tasks 1.1 — 1.7 (all parallel if multiple implementers available)
+2. Batch 2 (CI & docs): tasks 2.1 — 2.4 (parallel after Batch 1 complete)
+
+---
+
+## Estimates summary
+
+- Batch 1 total: ~3.75h
+- Batch 2 total: ~4.75h
+- Grand total: ~8.5h (approx)
+
+---
+
+Write this plan to: `thoughts/shared/plans/2026-03-28-rewrite-ansible-package.md`
+
+Done.