feat(ansible-example): add @ansible/example package, tests, CI, publish & deploy workflows, docs and changelog
This commit is contained in:
@@ -2,164 +2,122 @@
|
||||
|
||||
**Date:** 2026-03-26
|
||||
**Subdomain:** `motief.sgeboers.nl`
|
||||
**Stack:** Streamlit · DuckDB · Docker · Nginx · Drone CI
|
||||
**Stack:** Streamlit · uv · systemd · Nginx · Drone CI
|
||||
**Target:** VPS, `webapps` user at `/home/webapps/motief/`
|
||||
|
||||
---
|
||||
|
||||
## What's already ready (no changes needed)
|
||||
## Already done ✅
|
||||
|
||||
- `Dockerfile` — builds `streamlit run Home.py --server.port=8501`
|
||||
- `docker-compose.yml` — `motief` + `scheduler` services, `DATA_DIR` env override
|
||||
- `.drone.yml` — builds image, pushes to registry, SSH-deploys on push to `main`
|
||||
- `Home.py`, `pages/1_Stemwijzer.py`, `pages/2_Explorer.py` — all exist
|
||||
- VPS directory `/home/webapps/motief/data/` created
|
||||
- `motions.db` uploaded to VPS
|
||||
- nginx vhost configured for `motief.sgeboers.nl`
|
||||
- TLS cert via certbot
|
||||
|
||||
---
|
||||
|
||||
## Step A — VPS: one-time directory setup
|
||||
## Step A — Install uv on VPS
|
||||
|
||||
SSH in as `webapps`:
|
||||
|
||||
```bash
|
||||
mkdir -p /home/webapps/motief/data
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
source $HOME/.cargo/env # or re-login
|
||||
uv --version # verify
|
||||
```
|
||||
|
||||
Create `/home/webapps/motief/.env`:
|
||||
---
|
||||
|
||||
```env
|
||||
DOCKER_REGISTRY=<your-registry-url>
|
||||
DOCKER_USERNAME=<registry-user>
|
||||
DOCKER_PASSWORD=<registry-password>
|
||||
OPENROUTER_API_KEY=<key>
|
||||
OPENAI_API_KEY=<key>
|
||||
```
|
||||
|
||||
Copy `docker-compose.yml` into place:
|
||||
## Step B — Clone repo and install dependencies
|
||||
|
||||
```bash
|
||||
# From local machine
|
||||
scp docker-compose.yml webapps@<vps>:/home/webapps/motief/
|
||||
cd /home/webapps
|
||||
git clone <your-gitea-url>/sgeboers/stemwijzer motief
|
||||
cd motief
|
||||
uv sync
|
||||
```
|
||||
|
||||
Or just clone the repo there and symlink — either works since Drone will overwrite it.
|
||||
The `motions.db` you already uploaded should live at:
|
||||
```
|
||||
/home/webapps/motief/data/motions.db
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step B — Transfer the database
|
||||
## Step C — Create systemd user service
|
||||
|
||||
From local machine (~4 GB, takes a few minutes):
|
||||
Create `~/.config/systemd/user/motief.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=motief.sgeboers.nl Streamlit app
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
WorkingDirectory=/home/webapps/motief
|
||||
ExecStart=/home/webapps/.local/bin/uv run streamlit run Home.py --server.port=8501 --server.headless=true
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
```
|
||||
|
||||
Enable and start:
|
||||
|
||||
```bash
|
||||
rsync -avz --progress data/motions.db webapps@<vps>:/home/webapps/motief/data/motions.db
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable motief
|
||||
systemctl --user start motief
|
||||
systemctl --user status motief # verify it's running
|
||||
```
|
||||
|
||||
Do this as close to go-live as possible so the data isn't stale on launch.
|
||||
|
||||
---
|
||||
|
||||
## Step C — DNS
|
||||
|
||||
Add an **A record** in your DNS provider:
|
||||
|
||||
```
|
||||
stematlas → (obsolete, skip)
|
||||
motief → <VPS IPv4>
|
||||
```
|
||||
|
||||
TTL 300 for the first deploy so you can iterate quickly; bump to 3600 after it's stable.
|
||||
|
||||
---
|
||||
|
||||
## Step D — Nginx vhost
|
||||
|
||||
Create `/etc/nginx/sites-available/motief`:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name motief.sgeboers.nl;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name motief.sgeboers.nl;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/motief.sgeboers.nl/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/motief.sgeboers.nl/privkey.pem;
|
||||
|
||||
# Streamlit requires WebSocket upgrade for live updates
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8501;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_read_timeout 86400;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Enable and reload:
|
||||
Enable linger so the service survives logout:
|
||||
|
||||
```bash
|
||||
sudo ln -s /etc/nginx/sites-available/motief /etc/nginx/sites-enabled/
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
# Needs sudo (once only)
|
||||
sudo loginctl enable-linger webapps
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step E — TLS cert
|
||||
## Step D — Configure Drone secrets
|
||||
|
||||
```bash
|
||||
sudo certbot --nginx -d motief.sgeboers.nl
|
||||
```
|
||||
|
||||
(Assumes Certbot is already installed and working for other subdomains.)
|
||||
|
||||
---
|
||||
|
||||
## Step F — Configure Drone secrets
|
||||
|
||||
In the Gitea/Drone repo settings for `sgeboers/stemwijzer`, add:
|
||||
In `drone.sgeboers.nl` → `sgeboers/stemwijzer` → **Settings → Secrets**:
|
||||
|
||||
| Secret | Value |
|
||||
|--------|-------|
|
||||
| `DOCKER_REGISTRY` | Your registry URL |
|
||||
| `DOCKER_USERNAME` | Registry login |
|
||||
| `DOCKER_PASSWORD` | Registry password |
|
||||
| `DEPLOY_HOST` | VPS hostname/IP |
|
||||
| `DEPLOY_SSH_PORT` | SSH port (usually 22) |
|
||||
| `DEPLOY_HOST` | VPS hostname or IP |
|
||||
| `DEPLOY_SSH_PORT` | `22` (or custom) |
|
||||
| `DEPLOY_USER` | `webapps` |
|
||||
| `DEPLOY_PASSWORD` | webapps SSH password |
|
||||
|
||||
---
|
||||
|
||||
## Step G — First deploy
|
||||
## Step E — First auto-deploy
|
||||
|
||||
Option 1 — trigger Drone automatically:
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
Drone builds → pushes image → SSH into VPS → `docker-compose up -d`.
|
||||
|
||||
Option 2 — manual first deploy (on VPS):
|
||||
Drone will SSH in and run:
|
||||
```bash
|
||||
cd /home/webapps/motief
|
||||
docker-compose pull
|
||||
docker-compose up -d
|
||||
git pull origin main
|
||||
uv sync
|
||||
systemctl --user restart motief
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step H — Verify
|
||||
## Step F — Verify
|
||||
|
||||
```bash
|
||||
# On VPS
|
||||
docker-compose -f /home/webapps/motief/docker-compose.yml logs -f motief
|
||||
systemctl --user status motief
|
||||
journalctl --user -u motief -f
|
||||
|
||||
# From local browser
|
||||
# From browser
|
||||
open https://motief.sgeboers.nl
|
||||
```
|
||||
|
||||
@@ -168,22 +126,27 @@ Checklist:
|
||||
- [ ] Compass tab renders with correct party positions (GL-PvdA top-left, PVV bottom-right)
|
||||
- [ ] SVD tab scree plot shows with highlighted top-2 bars
|
||||
- [ ] Similarity search returns results
|
||||
- [ ] Scheduler container is running (`docker-compose ps`)
|
||||
|
||||
---
|
||||
|
||||
## Ongoing: data updates
|
||||
|
||||
The `scheduler` service runs the weekly pipeline inside the container:
|
||||
- Scrapes new motions from the TK OData API
|
||||
- Re-embeds new motion text via OpenRouter
|
||||
- Updates similarity cache
|
||||
The `scheduler.py` can be run as a separate user service or a cron job. To set it up as a service:
|
||||
|
||||
The `motions.db` file on the VPS is the single source of truth — it's bind-mounted into both containers. No cron job needed on the host.
|
||||
Create `~/.config/systemd/user/motief-scheduler.service`:
|
||||
|
||||
If you ever need to force a full re-run:
|
||||
```bash
|
||||
docker-compose exec scheduler python pipeline/run_pipeline.py --db-path data/motions.db
|
||||
```ini
|
||||
[Unit]
|
||||
Description=motief scheduler (weekly pipeline)
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
WorkingDirectory=/home/webapps/motief
|
||||
ExecStart=/home/webapps/.local/bin/uv run python scheduler.py
|
||||
Restart=on-failure
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
```
|
||||
|
||||
---
|
||||
@@ -191,12 +154,9 @@ docker-compose exec scheduler python pipeline/run_pipeline.py --db-path data/mot
|
||||
## Dependency order
|
||||
|
||||
```
|
||||
A (dirs + .env) ─┐
|
||||
B (rsync DB) ─┤─► G (first deploy) ─► H (verify)
|
||||
C (DNS) ─┤
|
||||
D (nginx) ─┤
|
||||
E (certbot) ─┘
|
||||
F (Drone secrets) ──► future auto-deploys on push to main
|
||||
A (install uv) ─┐
|
||||
B (clone + sync) ─┤─► C (systemd service) ─► E (push to main) ─► F (verify)
|
||||
└─► D (Drone secrets) ────┘
|
||||
```
|
||||
|
||||
Steps A–F can all be done in one SSH session. Total estimated time: **45 minutes** (mostly waiting on rsync).
|
||||
Total estimated time: **20 minutes**.
|
||||
|
||||
@@ -0,0 +1,269 @@
|
||||
# Rewrite @ansible package for npm publish — Implementation Plan
|
||||
|
||||
**Goal:** Implement a publishable npm-scoped example package at packages/@ansible/example, add guarded GitHub Actions publish and deploy workflows (publish on v* tags or manual_dispatch; deploy to motief.sgeboers.nl via user webapps), CI pack-tests, and documentation for embedding and deployment secrets.
|
||||
|
||||
**Design:** see thoughts/shared/designs/2026-03-28-rewrite-ansible-package-design.md
|
||||
|
||||
Author: Sven Geboers
|
||||
|
||||
---
|
||||
|
||||
## Dependency Graph
|
||||
|
||||
```
|
||||
Batch 1 (parallel): 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
|
||||
Batch 2 (parallel): 2.1, 2.2, 2.3, 2.4 [depends on Batch 1]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Batch 1: Foundation (parallel - independent files)
|
||||
|
||||
All tasks in this batch have NO dependencies and can be created in parallel.
|
||||
|
||||
### Task 1.1: package.json for package
|
||||
**File:** `packages/@ansible/example/package.json`
|
||||
**Summary:** New package.json implementing an npm-scoped package @ansible/example. Exact choices made:
|
||||
- name: "@ansible/example"
|
||||
- version: "0.1.0"
|
||||
- author: "Sven Geboers"
|
||||
- publishConfig.access: "public"
|
||||
- files: ["src/**", "README.md", "package.json"] (controls published files, avoids .npmignore)
|
||||
- scripts:
|
||||
- test: `node tests/run.js`
|
||||
- pack-inspect: `node tests/test_pack_inspect.js`
|
||||
- prepublish:verify: `npm run pack-inspect` (keeps a named script for CI)
|
||||
- pack: `npm pack`
|
||||
**Tests to add:** none in this file; tests below will exercise package.json
|
||||
**Verify:**
|
||||
- Local: cd packages/@ansible/example && node -e "console.log(require('./package.json').name)" # should print @ansible/example
|
||||
- Run package tests: cd packages/@ansible/example && npm test
|
||||
**Effort:** 0.5h
|
||||
|
||||
### Task 1.2: README.md
|
||||
**File:** `packages/@ansible/example/README.md`
|
||||
**Summary:** Short README with package purpose, usage example, and author attribution line "Author: Sven Geboers". Include publish and deploy notes and reference to docs/*.md for deployment details.
|
||||
**Tests to add:** none
|
||||
**Verify:** open file or run: sed -n '1,40p' packages/@ansible/example/README.md
|
||||
**Effort:** 0.5h
|
||||
|
||||
### Task 1.3: package entrypoint
|
||||
**File:** `packages/@ansible/example/src/index.js`
|
||||
**Summary:** Minimal, well-documented CommonJS module that exports a function used by tests. Keep runtime trivial (e.g., function hello(name){ return `hello ${name}` }). No dependencies.
|
||||
**Tests to add:** used by unit test below
|
||||
**Verify:** node -e "console.log(require('./packages/@ansible/example/src/index.js')('world'))"
|
||||
**Effort:** 0.5h
|
||||
|
||||
### Task 1.4: unit test — package.json fields
|
||||
**File:** `packages/@ansible/example/tests/test_package_json.js`
|
||||
**Summary:** Node test that loads package.json and asserts required fields are present: name === "@ansible/example", version present, author === "Sven Geboers", publishConfig.access === "public", files array exists.
|
||||
**Tests to add:** this is the test file
|
||||
**Verify:** cd packages/@ansible/example && node tests/test_package_json.js (exit 0 on success)
|
||||
**Effort:** 0.5h
|
||||
|
||||
### Task 1.5: pack-inspect test
|
||||
**File:** `packages/@ansible/example/tests/test_pack_inspect.js`
|
||||
**Summary:** Test that runs `npm pack` (in package dir) programmatically, captures the produced tarball name, asserts the tarball exists and contains package/package.json. Implementation notes: uses child_process.execSync and `tar -xOzf` to read package/package.json from the tarball and assert name/version match. This test requires `tar` on the runner (Linux/macOS). If `tar` not available, the test fails with clear message.
|
||||
**Tests to add:** this is the test
|
||||
**Verify:** cd packages/@ansible/example && node tests/test_pack_inspect.js
|
||||
**Effort:** 1.0h
|
||||
|
||||
### Task 1.6: tiny package test runner
|
||||
**File:** `packages/@ansible/example/tests/run.js`
|
||||
**Summary:** Small node test harness that runs both test_package_json.js and test_pack_inspect.js, prints nice output, and returns non-zero on failure. Used by package.json test script to keep CI independent of external test runners.
|
||||
**Tests to add:** none (this is the runner used by `npm test`)
|
||||
**Verify:** cd packages/@ansible/example && node tests/run.js
|
||||
**Effort:** 0.5h
|
||||
|
||||
### Task 1.7: pack-inspect helper (optional small script)
|
||||
**File:** `packages/@ansible/example/tests/_pack_helpers.js`
|
||||
**Summary:** Small utility used by test_pack_inspect.js to encapsulate npm pack and tar inspection logic. Keeps main test readable. (Kept internal/private to tests.)
|
||||
**Tests to add:** none
|
||||
**Verify:** run the tests which import it
|
||||
**Effort:** 0.25h
|
||||
|
||||
---
|
||||
|
||||
## Batch 2: CI + Docs (parallel — depend on Batch 1)
|
||||
|
||||
All tasks in this batch depend on the package files being present (Batch 1).
|
||||
|
||||
### Task 2.1: GitHub Actions publish workflow
|
||||
**File:** `.github/workflows/publish-ansible-example.yml`
|
||||
**Summary:** New Actions workflow that performs build/test/pack/publish for packages/@ansible/example.
|
||||
**Key behavior:**
|
||||
- Triggers: push tags matching `v*` and workflow_dispatch (manual).
|
||||
- Jobs:
|
||||
- verify: runs on all triggers: checks out repo, sets up Node 18 (LTS) using actions/setup-node, installs root-level dependencies if any (skipped if none), then runs `cd packages/@ansible/example && npm ci || true` (guard), then `npm test`. Then runs `npm pack` and verifies produced tarball (reuses test script). The verify job always runs and must pass before publish.
|
||||
- publish: runs only when `github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')` OR when manually triggered AND `secrets.NPM_TOKEN` exists. publish job is gated by `if: ${{ secrets.NPM_TOKEN != '' && ( github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/tags/v') ) }}`.
|
||||
- On publish: write ephemeral ~/.npmrc with `//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}` (use `echo` and file permissions 0600), run `npm publish --access public` from package directory, then securely delete ~/.npmrc (shred if available or overwrite and rm). Use Actions mask to avoid logging secrets.
|
||||
|
||||
**Secrets required:** NPM_TOKEN (string)
|
||||
**Tests to add:** none (workflow file)
|
||||
**Verify:**
|
||||
- Locally simulate: cd packages/@ansible/example && npm test && npm pack
|
||||
- In Actions: push a non-production tag on a test repository with a safe NPM_TOKEN that points to a test registry OR run workflow_dispatch with a dry-run branch (see dry-run below)
|
||||
**Effort:** 1.5h
|
||||
|
||||
### Task 2.2: GitHub Actions deploy workflow
|
||||
**File:** `.github/workflows/deploy-to-vps.yml`
|
||||
**Summary:** Workflow to prepare and run deployment to external VPS motief.sgeboers.nl using user `webapps`. Triggers: push to main and workflow_dispatch. The job is non-destructive by default and will not run remote commands unless DEPLOY_SSH_KEY or DEPLOY_PASSWORD secrets are present. It includes a dry-run/verify job that checks SSH connectivity without executing commands that change state.
|
||||
**Key behavior:**
|
||||
- Inputs/secrets used (recommended names): DEPLOY_HOST (default motief.sgeboers.nl), DEPLOY_USER (default webapps), DEPLOY_SSH_PORT (default 22), DEPLOY_SSH_KEY (private key, optional), DEPLOY_PASSWORD (optional fallback), DEPLOY_PATH (path to deploy, optional)
|
||||
- Jobs:
|
||||
- dry-run-connect: attempts to verify connectivity. If DEPLOY_SSH_KEY present, create an ephemeral key file (0600), run `ssh -o BatchMode=yes -i $KEY -p $PORT $USER@$HOST 'echo connected'` and return success if the host echoes. This step is strictly a connection check (no file writes). If only DEPLOY_PASSWORD provided, the job will print instructions explaining the need for SSH key or use of a runner with sshpass (not recommended) and skip.
|
||||
- deploy: guarded `if: steps.check_secrets.outputs.has_creds == 'true'` — only runs when credentials are present. Steps: checkout, optionally build artifacts, create ephemeral SSH key file, rsync or scp artifact to $USER@$HOST:$DEPLOY_PATH, run non-destructive remote commands (e.g., systemctl --user status <unit> or echo) only if DEPLOY_CONFIRM=true input is set. By default the job performs no destructive operations; it's expected operator sets inputs when ready.
|
||||
|
||||
**Secrets required (for actual deploy):** DEPLOY_SSH_KEY (preferred), OR DEPLOY_PASSWORD (less secure, not recommended)
|
||||
**Secrets recommended / env:** DEPLOY_HOST (default motief.sgeboers.nl), DEPLOY_USER=webapps, DEPLOY_SSH_PORT=22, DEPLOY_PATH=/home/webapps/motief (example)
|
||||
**Tests to add:** none (workflow file)
|
||||
**Verify:**
|
||||
- Run dry-run via workflow_dispatch on Actions to verify connectivity (without running deploy). Ensure secrets are set in repository settings.
|
||||
**Effort:** 2.0h
|
||||
|
||||
### Task 2.3: docs — deployment and Drone note
|
||||
**File:** `docs/deployment/ansible-package-deploy.md`
|
||||
**Summary:** Documentation describing: default host motief.sgeboers.nl, recommended DEPLOY_USER webapps, how to add GitHub repo secrets (DEPLOY_SSH_KEY, DEPLOY_HOST, DEPLOY_USER, DEPLOY_SSH_PORT), sample systemd unit name recommendation (e.g., service name `motief`), and instructions for users preferring Drone: how to set equivalent secrets in Drone and a note that .drone.yml is left untouched.
|
||||
**Tests to add:** none
|
||||
**Verify:** open file or grep keywords DEPLOY_SSH_KEY, motief.sgeboers.nl
|
||||
**Effort:** 0.75h
|
||||
|
||||
### Task 2.4: docs — embeddings and environment variables
|
||||
**File:** `docs/embeddings.md`
|
||||
**Summary:** Document that the project uses QWEN embeddings (qwen/qwen3-embedding-4b via OpenRouter), recommend setting OPENROUTER_API_KEY in repo/host secrets, and mention OPENAI_API_KEY as optional fallback. Include example env var usage and security guidance.
|
||||
**Tests to add:** none
|
||||
**Verify:** open file and confirm environment variables documented
|
||||
**Effort:** 0.5h
|
||||
|
||||
---
|
||||
|
||||
## CI workflow outlines (YAML-level steps in prose)
|
||||
|
||||
Publish workflow (.github/workflows/publish-ansible-example.yml):
|
||||
- on:
|
||||
- push: tags: ['v*']
|
||||
- workflow_dispatch
|
||||
- jobs:
|
||||
- verify:
|
||||
- runs-on: ubuntu-latest
|
||||
- steps:
|
||||
1. actions/checkout@v4
|
||||
2. actions/setup-node@v4 (node-version: '18') with cache: 'npm'
|
||||
3. echo current package info (for debugging) but do NOT print secrets
|
||||
4. cd packages/@ansible/example && npm ci --no-audit --prefer-offline || true
|
||||
5. cd packages/@ansible/example && npm test (calls tests/run.js)
|
||||
6. cd packages/@ansible/example && npm run pack-inspect
|
||||
- artifacts: upload pack artifact for inspection (optional)
|
||||
|
||||
- publish:
|
||||
- needs: verify
|
||||
- runs-on: ubuntu-latest
|
||||
- if: (github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/tags/v')) && secrets.NPM_TOKEN != ''
|
||||
- steps:
|
||||
1. checkout
|
||||
2. setup-node
|
||||
3. create ephemeral ~/.npmrc with content `//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}` using `run: printf` and set 0600
|
||||
4. cd packages/@ansible/example && npm publish --access public
|
||||
5. securely remove ~/.npmrc (overwrite if shred available then rm)
|
||||
- secrets required: NPM_TOKEN
|
||||
- guard rails: job will not run if NPM_TOKEN is empty; Actions expressions used to gate execution
|
||||
|
||||
Deploy workflow (.github/workflows/deploy-to-vps.yml):
|
||||
- on:
|
||||
- push: branches: [ main ]
|
||||
- workflow_dispatch
|
||||
- inputs (for manual dispatch): target_branch, confirm_deploy (boolean)
|
||||
- jobs:
|
||||
- dry-run-connect:
|
||||
- runs-on: ubuntu-latest
|
||||
- steps:
|
||||
1. checkout
|
||||
2. set env from secrets (DEPLOY_HOST=motief.sgeboers.nl default)
|
||||
3. if secrets.DEPLOY_SSH_KEY set: write ephemeral key file with 0600 and run ssh -o BatchMode=yes -p $DEPLOY_SSH_PORT $DEPLOY_USER@$DEPLOY_HOST 'echo connected' (short timeout)
|
||||
4. if no key present but DEPLOY_PASSWORD present: skip and print instructions to set key
|
||||
- outcome: outputs.has_creds true/false for next job
|
||||
|
||||
- deploy:
|
||||
- needs: dry-run-connect
|
||||
- if: needs.dry-run-connect.outputs.has_creds == 'true' && github.event.inputs.confirm_deploy == 'true'
|
||||
- steps:
|
||||
1. checkout
|
||||
2. build artifacts (if required)
|
||||
3. create ephemeral key file and copy artifacts via rsync/scp to $DEPLOY_USER@$DEPLOY_HOST:$DEPLOY_PATH
|
||||
4. optionally run remote non-destructive checks (e.g., tail logs, systemctl --user status motief)
|
||||
- secrets required for actual deploy: DEPLOY_SSH_KEY or DEPLOY_PASSWORD
|
||||
|
||||
Security considerations for workflows:
|
||||
- Never echo secrets. Use Actions' built-in mask and avoid printing environment variables that contain secrets.
|
||||
- Create ephemeral ~/.npmrc and remove it immediately. Use file permissions 0600.
|
||||
- Create ephemeral SSH key files with 0600 and remove them at the end of the job.
|
||||
|
||||
---
|
||||
|
||||
## Rollback / Undo plan for risky changes
|
||||
|
||||
- Publish workflow added: to rollback, remove or disable `.github/workflows/publish-ansible-example.yml` and push a commit. If a package was accidentally published, use npm unpublish only if within npm policy window (careful: unpublish can harm consumers) — prefer deprecating the published version via `npm deprecate @ansible/example@x.y.z "do not use"`.
|
||||
- Deploy workflow added: to rollback, remove/disable `.github/workflows/deploy-to-vps.yml`. If deployment ran, have a documented process on the VPS to rollback the app (e.g., systemd service `motief` reversion, keep previous release tarball and symlink rollback). Document in deploy docs how to revert symlink/munpack and restart systemd.
|
||||
- Docs changed: simply revert doc files and commit.
|
||||
|
||||
Notes on npm unpublish: prefer `npm deprecate` over unpublish in most cases. Unpublish should be used only with caution and awareness of npm registry policy.
|
||||
|
||||
---
|
||||
|
||||
## Final operator checklist (manual steps before running publish/deploy)
|
||||
|
||||
1. Add repository secrets in GitHub Settings > Secrets:
|
||||
- NPM_TOKEN — a token with publish access to the @ansible scope (required for publish job)
|
||||
- DEPLOY_SSH_KEY — (recommended) private SSH key for `webapps` user on motief.sgeboers.nl; set as secret (do NOT include passphrase unless CI knows how to handle it)
|
||||
- Optionally DEPLOY_PASSWORD — password fallback (not recommended)
|
||||
- DEPLOY_HOST — default: motief.sgeboers.nl (recommended to set explicitly)
|
||||
- DEPLOY_USER — default: webapps
|
||||
- DEPLOY_SSH_PORT — default: 22
|
||||
- OPENROUTER_API_KEY — recommended for embeddings
|
||||
- OPENAI_API_KEY — optional fallback
|
||||
|
||||
2. Ensure target VPS `motief.sgeboers.nl` has user `webapps` configured and an authorized public key corresponding to the DEPLOY_SSH_KEY private key.
|
||||
|
||||
3. Ensure the VPS has a systemd unit name prepared (recommendation: `motief.service`) and that deployment user `webapps` may write to the deploy path (e.g., /home/webapps/motief) and manage its own files. Documented in `docs/deployment/ansible-package-deploy.md`.
|
||||
|
||||
4. (Optional) If you prefer Drone CI: set Drone secrets for NPM_TOKEN and DEPLOY_* equivalently; .drone.yml remains untouched and you can run publish steps in Drone if you adapt the template.
|
||||
|
||||
5. Verify local tests: from repo root run:
|
||||
- cd packages/@ansible/example && npm test
|
||||
- cd packages/@ansible/example && npm pack && tar -tzf <produced-tgz> | head -n 20
|
||||
|
||||
6. For publish: create a tag `git tag v0.1.0` (or desired semver) and push the tag. The publish workflow will run and attempt publish if NPM_TOKEN secret is present.
|
||||
|
||||
7. For deploy: run the deploy workflow_dispatch after setting DEPLOY_SSH_KEY and testing dry-run connectivity via the dry-run job.
|
||||
|
||||
---
|
||||
|
||||
## Decisions & Assumptions
|
||||
|
||||
- Package name chosen: `@ansible/example` per design doc. Changeable later by editing package.json and tag.
|
||||
- Default version set to `0.1.0` to indicate initial publish candidate.
|
||||
- Testing uses a zero-dependency Node test harness (simple node scripts) to avoid introducing a test framework and to keep CI minimal.
|
||||
- SSH key auth chosen as default for deployment because it's more secure and scriptable in CI than password auth. SSH keys avoid exposing passwords in secrets logs and work with `ssh -o BatchMode=yes` checks. We recommend DEPLOY_SSH_KEY; DEPLOY_PASSWORD is supported as a fallback but not recommended.
|
||||
- Workflows are deliberately gated: publish runs only on tags `v*` or manual dispatch, and publish job requires NPM_TOKEN secret. Deploy will not execute destructive commands unless manual confirm is provided in workflow_dispatch inputs.
|
||||
- We will not modify existing `.drone.yml` as requested; docs will describe how to set Drone secrets if the operator prefers Drone.
|
||||
|
||||
---
|
||||
|
||||
## Staged execution order
|
||||
|
||||
1. Batch 1 (create package files & tests): tasks 1.1 — 1.7 (all parallel if multiple implementers available)
|
||||
2. Batch 2 (CI & docs): tasks 2.1 — 2.4 (parallel after Batch 1 complete)
|
||||
|
||||
---
|
||||
|
||||
## Estimates summary
|
||||
|
||||
- Batch 1 total: ~3.75h
|
||||
- Batch 2 total: ~4.75h
|
||||
- Grand total: ~8.5h (approx)
|
||||
|
||||
---
|
||||
|
||||
Write this plan to: `thoughts/shared/plans/2026-03-28-rewrite-ansible-package.md`
|
||||
|
||||
Done.
|
||||
Reference in New Issue
Block a user