diff --git a/SESSION_HANDOFF_2026-04-21_CREDENTIAL_ROTATION.md b/SESSION_HANDOFF_2026-04-21_CREDENTIAL_ROTATION.md new file mode 100644 index 00000000..9fba2852 --- /dev/null +++ b/SESSION_HANDOFF_2026-04-21_CREDENTIAL_ROTATION.md @@ -0,0 +1,118 @@ +# Session Handoff — 2026-04-21 — Credential Rotation + Forgejo SSH Port-Mapping Fix + +**Status:** COMPLETE. +**Previous session:** `SESSION_HANDOFF_2026-04-20_EUPL12_OUT_OF_SCOPE_SWEEP.md` (closed 2026-04-20 late evening after 7 commits + GitHub->Codeberg sweep). This handoff covers the overnight pause + next-morning completion of credential rotation and one server-side fix discovered during verification. +**Session model:** Opus 4.7 (1M context) — `claude-opus-4-7[1m]`. +**Mode:** NON-TECHNICAL OPERATOR MODE throughout — browser walkthroughs, Bitwarden-first secret generation, step-by-step with per-step confirmation checkpoints. + +--- + +## Why rotation + +Yesterday's EUPL-1.2 + GitHub-purge work exposed two plaintext credentials to the conversation context — both previously embedded in `.git/config` remote URLs: + +1. Codeberg personal access token `mysovereignty` +2. Forgejo password for user `john` + +Both treated as burned and rotated. No action needed on the literal values now — both are dead. + +--- + +## Rotation work — summary + +### Local machine (`the-flow`, `~/projects/tractatus`) +- Generated ed25519 SSH keypair at `~/.ssh/tractatus_git` / `~/.ssh/tractatus_git.pub`, comment `theflow-tractatus-2026-04-20`. Passphrase chosen (not blank); saved to Bitwarden. +- Updated `~/.ssh/config` with two new `Host` blocks (`codeberg.org`, `git.mysovereignty.digital`), each with `IdentityFile ~/.ssh/tractatus_git` and `IdentitiesOnly yes`. `chmod 600` applied. +- `git remote set-url` on both remotes: + - `codeberg` -> `git@codeberg.org:mysovereignty/tractatus-framework.git` + - `origin` -> `ssh://git@git.mysovereignty.digital:2222/john/tractatus.git` +- `.git/config` now has no plaintext credentials anywhere. + +### Codeberg (mysovereignty) +- SSH key attached (fingerprint `SHA256:+tzsQfhhf7oei+MqG9TjRqXjS4ybwJ5pzU68IDvwZSs`). +- Aegis TOTP 2FA enrolled. Single-use recovery key saved to Bitwarden. +- All personal access tokens revoked (1 token was live; it was the burned one). + +### Forgejo (`git.mysovereignty.digital`, user `john`) +- SSH key attached (same fingerprint — it's one key). +- Account secret replaced: new value generated inside Bitwarden Generator (20+ char, all charset classes). Previous value immediately dead. +- Aegis TOTP 2FA enrolled. Single-use recovery key saved to Bitwarden. +- No existing access tokens to revoke. + +### Bitwarden items created this session (all in root; Phase 2 Collections reorg will sweep them later) +1. `SSH tractatus_git passphrase` (Secure Note) +2. `Codeberg mysovereignty — 2FA recovery key` (Secure Note, single-use) +3. `Forgejo john — 2FA recovery key` (Secure Note, single-use) +4. `git.mysovereignty.digital — john` (Login item; new password, URI `https://git.mysovereignty.digital`) + +### Verification (final scorecard) + +| Item | Status | +|---|---| +| Codeberg old token revoked | done | +| Forgejo old password dead | done | +| `.git/config` plaintext creds removed | done | +| SSH key attached on Codeberg + Forgejo | done | +| Aegis TOTP 2FA on both hosts | done | +| Recovery keys saved to Bitwarden | done | +| `git fetch codeberg` (SSH) | done | +| `git fetch origin` (SSH) | done (after Forgejo server-side port-mapping fix, below) | + +--- + +## Server-side Forgejo fix (the final blocker) + +After all the rotation work, `git fetch origin` via the new SSH URL failed with `Permission denied (publickey)` on the standard SSH port, and `Connection refused` on the advertised Forgejo SSH port. The SSH key was correctly attached on the Forgejo account (fingerprints matched exactly). Yet authentication failed. + +Diagnosis path: ruled out OVH upstream filter (tcpdump captured SYN arriving at `ens3`), ruled out UFW (rules allow the advertised port), ruled out iptables DOCKER chain (DNAT and ACCEPT rules present), and finally isolated the TCP RST origin to inside the Forgejo container itself. Root cause was a port-mapping error in `/home/ubuntu/forgejo/docker-compose.yml`: + +- Before: `- "2222:2222"` — host's advertised SSH port mapped to container's advertised port. +- Problem: Forgejo's internal sshd listens on container `port 22` (the conventional SSH port inside the image). The container-side `2222` had no listener. So DNATed traffic hit a dead port and the container kernel replied with TCP RST. +- After: `- "2222:22"` — host's advertised SSH port now maps to container's internal sshd port 22. DNAT arrives where a listener exists. + +This mapping has been mis-set since the Forgejo instance was deployed ~3 weeks ago. HTTPS worked throughout, which masked the issue. SSH git operations never functioned on this instance until today's fix. + +Files changed on the Forgejo VPS (`vps-7f023e40.vps.ovh.net`, reverse lookup `meet.myfamilyhistory.digital` — this single box also hosts Jitsi, Matrix, Collabora, and other services): +- `/home/ubuntu/forgejo/docker-compose.yml` — the one-line port mapping fix +- Backup kept at `/home/ubuntu/forgejo/docker-compose.yml.bak-20260421-pre-port-fix` +- `docker compose down && docker compose up -d` applied. Container restarted clean. + +The compose file is not under version control — lives on-server only. Version-controlling it is a sensible follow-up but not urgent. + +--- + +## Things in this document NOT reproduced verbatim + +For hygiene: the literal revoked Codeberg token value, the old Forgejo password, the Bitwarden-generated new Forgejo password, the 2FA recovery keys, and the SSH passphrase are all referenced by their Bitwarden item name only — not by value. All sensitive-at-time-of-writing values live in Bitwarden. + +--- + +## Follow-ups (optional, none urgent) + +1. Add `Port 2222` directive to the `Host git.mysovereignty.digital` block in `~/.ssh/config` for future cloners who use `git@...` URL syntax instead of `ssh://...:PORT/...` inline form. The current inline-port URL works without the directive so this is cosmetic. +2. Version-control the Forgejo `docker-compose.yml` (currently on-server only at `/home/ubuntu/forgejo/`). Put it in a small infra repo, alongside the compose files for Jitsi / Matrix / Collabora on the same box — they all share this pattern. +3. Remove `/home/ubuntu/forgejo/docker-compose.yml.bak-20260421-pre-port-fix` once confident the fix is stable. +4. (Security hygiene, not blocking) Rotate any other self-hosted-service admin credentials that were in this conversation's scroll-back earlier today: Matrix admin account, Collabora admin, Jitsi JWT secrets — if applicable. Scope unknown to me; flagging the class. + +--- + +## Change log + +| Time (NZST) | Event | +|---|---| +| 2026-04-20 ~22:00 | SSH keypair generated, passphrase saved to Bitwarden. | +| ~22:05 | Codeberg: SSH key added, Aegis 2FA enrolled, recovery key to Bitwarden, old token revoked. | +| ~22:15 | Forgejo: SSH key added, password changed via Bitwarden Generator, Aegis 2FA enrolled, recovery key to Bitwarden. | +| ~22:25 | `~/.ssh/config` updated (ended up using separate `echo` commands after a heredoc copy-paste indentation trap caused a runaway heredoc). `chmod 600` applied. | +| ~22:30 | `.git/config` remote URLs sanitised (plaintext HTTPS credentials removed). | +| ~22:35 | `git fetch codeberg` clean. `git fetch origin` failed `Permission denied (publickey)` on `port 22`, `Connection refused` on `port 2222`. User paused overnight. | +| 2026-04-21 07:15 | Resume. `ssh-keygen -lf ~/.ssh/tractatus_git.pub` confirmed local fingerprint matches the key attached to Forgejo. | +| ~07:25 | Forgejo clone URL confirmed as `ssh://git@git.mysovereignty.digital:2222/john/tractatus.git`. | +| ~07:30 — ~07:42 | Extensive diagnostics: port-reachability scan across 20 ports, bi-directional `tcpdump` on `ens3` and `br-08a69dd7f908`, test-port nc listeners, loopback comparison, Docker container internal `netstat`, Forgejo `app.ini` inspection. Ruled out OVH network filter, UFW, fail2ban, rp_filter. TCP RST origin isolated to inside the Forgejo container. | +| ~07:43 | Root cause identified: compose port mapping mismatch. Fix applied via Python replace (not sed). `docker compose down && up -d`. New mapping verified: `22/tcp -> 0.0.0.0:2222`. Container sshd listening on `0.0.0.0:22` internally. | +| ~07:44 | SSH handshake from local: `Hi there, john! You've successfully authenticated with the key named theflow-tractatus-2026-04-20`. `git fetch origin` clean. | +| ~07:50 | This handoff written. | + +--- + +*Rotation closed. Both tractatus git remotes (`codeberg` + `origin`) now authenticate over SSH using the new key; old credentials are dead; 2FA is active on both accounts; all sensitive values live in Bitwarden.*