docs: Add VPS access reference and DeepSeek handover brief
- VPS_ACCESS_REFERENCE.md: Complete SSH recovery procedures for OVH rescue mode - DEEPSEEK_HANDOVER_BRIEF.md: System maintenance handover documentation Addresses repeated SSH access failures and documents recovery procedures. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
TheFlow
parent
90af909754
commit
008f0169a4
2 changed files with 420 additions and 0 deletions
186
docs/DEEPSEEK_HANDOVER_BRIEF.md
Normal file
186
docs/DEEPSEEK_HANDOVER_BRIEF.md
Normal file
|
|
@ -0,0 +1,186 @@
|
|||
# DeepSeek System Maintenance Handover Brief
|
||||
|
||||
**Prepared:** 2026-01-19
|
||||
**Target System:** agenticgovernance.digital (Tractatus Framework website)
|
||||
**Reason:** Migration from Anthropic Claude Code due to reliability failures
|
||||
|
||||
---
|
||||
|
||||
## System Overview
|
||||
|
||||
| Component | Details |
|
||||
|-----------|---------|
|
||||
| **Server** | OVHCloud VPS vps-93a693da.vps.ovh.net |
|
||||
| **IP Address** | 91.134.240.3 |
|
||||
| **Domain** | agenticgovernance.digital |
|
||||
| **OS** | Ubuntu 22.04 LTS |
|
||||
| **Application** | Node.js Express (Tractatus Framework) |
|
||||
| **Database** | MongoDB (local, port 27017) |
|
||||
| **Web Server** | Nginx (reverse proxy) |
|
||||
|
||||
---
|
||||
|
||||
## SSH Access
|
||||
|
||||
### Credentials
|
||||
- **User:** ubuntu
|
||||
- **Authentication:** SSH key only (password disabled)
|
||||
- **Port:** 22
|
||||
|
||||
### Authorized Keys (must be in /home/ubuntu/.ssh/authorized_keys)
|
||||
|
||||
**Primary key (theflow@the-flow):**
|
||||
```
|
||||
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCZ8BH+Bx4uO9DTatRZ/YF5xveP/bTyiAWj+qTF7I+ugxgL9/ejSlW1tSn5Seo4XHoEPD5wZCaWig7m1LMezrRq8fDWHbeXkZltK01xhAPU0L0+OvVZMZacW6+vkNfKcNG9vrxV+K/VTPkT+00TRqlHbP8ZWj0OWd92XAoTroKVYMt4L9e7QeJOJmRmHI0uFaJ0Ufexr2gmZyYhgL2p7PP3oiAvM0xlnTwygl06c3iwXpHKWNydOYPSDs3MkVnDjptmWgKv/J+QXksarwEpA4Csc2dLnco+8KrtocUUcAunz6NJfypA0yNWWzf+/OeffkJ2Rueoe8t/lVffXdI7eVuFkmDufE7XMk9YAE/8+XVqok4OV0Q+bjpH8mKlBA3rNobnWs6obBVJD8/5aphE8NdCR4cgIeRSwieFhfzCl+GBZNvs4yuBdKvQQIfCRAKqTgbuc03XERAef6lJUuJrDjwzvvp1Nd8L7AqJoQS6kYGyxXPf/6nWTZtpxoobdGnJ2FZK6OIpAlsWx9LnybMGy19VfaR9JZSAkLdWxGPb6acNUb2xaaqyuXPo4sWpBM27n1HeKMv/7Oh4WL4zrAxDKfN38k1JsjJJVEABuN/pEOb7BCDnTMLKXlTunZgynAZJ/Dxn+zOAyfzaYSNBotlpYy1zj1AmzvS31L7LJy/aSBHuWw== theflow@the-flow
|
||||
```
|
||||
|
||||
**Deploy key (tractatus-deploy):**
|
||||
```
|
||||
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPdJcKMabIVQRqKqNIpzxHNgxMZ8NOD+9gVCk6dY5uV0 tractatus-deploy
|
||||
```
|
||||
|
||||
**Key backup URL:** https://paste.rs/nELRM
|
||||
|
||||
### Known SSH Issues
|
||||
|
||||
1. **Starlink IP changes** - User connects via Starlink which changes IP frequently
|
||||
2. **fail2ban** - May block IPs after failed attempts
|
||||
3. **Host key changes** - Server has been rebuilt/rescued multiple times
|
||||
|
||||
### Recovery When SSH Fails
|
||||
|
||||
Use OVH Rescue Mode:
|
||||
1. OVH Manager → VPS → Reboot in rescue mode
|
||||
2. SSH as root with emailed password
|
||||
3. Mount disk: `mount /dev/sdb1 /mnt/vps`
|
||||
4. Add keys: `cat key >> /mnt/vps/home/ubuntu/.ssh/authorized_keys`
|
||||
5. Fix permissions: `chmod 600 ... && chown -R 1000:1000 ...`
|
||||
6. Reboot to normal mode
|
||||
|
||||
---
|
||||
|
||||
## Services
|
||||
|
||||
### Application Service (tractatus)
|
||||
```bash
|
||||
sudo systemctl status tractatus
|
||||
sudo systemctl restart tractatus
|
||||
sudo journalctl -u tractatus -f
|
||||
```
|
||||
|
||||
Service file: `/etc/systemd/system/tractatus.service`
|
||||
|
||||
### Nginx
|
||||
```bash
|
||||
sudo systemctl status nginx
|
||||
sudo nginx -t # Test config
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
Config: `/etc/nginx/sites-available/agenticgovernance.digital`
|
||||
|
||||
### MongoDB
|
||||
```bash
|
||||
sudo systemctl status mongod
|
||||
mongosh mongodb://localhost:27017/tractatus
|
||||
```
|
||||
|
||||
**CRITICAL:** MongoDB logs can grow to 45GB+ and fill disk. Monitor `/var/log/mongodb/` and configure logrotate.
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
### From Local Machine
|
||||
```bash
|
||||
cd /home/theflow/projects/tractatus
|
||||
./scripts/deploy.sh
|
||||
```
|
||||
|
||||
### Manual Deployment
|
||||
```bash
|
||||
rsync -avz --exclude 'node_modules' --exclude '.git' \
|
||||
/home/theflow/projects/tractatus/ \
|
||||
ubuntu@91.134.240.3:/var/www/tractatus/
|
||||
|
||||
ssh ubuntu@91.134.240.3 "cd /var/www/tractatus && npm install --production && sudo systemctl restart tractatus"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Locations
|
||||
|
||||
| Purpose | Path |
|
||||
|---------|------|
|
||||
| Application | `/var/www/tractatus/` |
|
||||
| Public files | `/var/www/tractatus/public/` |
|
||||
| Logs | `/var/log/mongodb/`, `journalctl -u tractatus` |
|
||||
| Nginx config | `/etc/nginx/sites-available/` |
|
||||
| SSL certs | Let's Encrypt (auto-renewed) |
|
||||
|
||||
---
|
||||
|
||||
## Known Issues & History
|
||||
|
||||
| Date | Issue | Resolution |
|
||||
|------|-------|------------|
|
||||
| 2025-12-09 | Docker container compromised (Exodus botnet) | Docker removed, services hardened |
|
||||
| 2026-01-18 | SSH access lost | Recovered via OVH Rescue Mode |
|
||||
| 2026-01-18 | Disk 99.5% full | Cleared 45GB MongoDB logs |
|
||||
| 2026-01-19 | SSH access lost again | Pending - likely Starlink IP change |
|
||||
|
||||
---
|
||||
|
||||
## Security Configuration
|
||||
|
||||
- **SSH:** Key-only, no password, no root login, MaxAuthTries=3
|
||||
- **UFW:** Ports 22, 80, 443 open; Docker ports blocked
|
||||
- **fail2ban:** SSH jail with 24h ban after 3 failures
|
||||
- **MongoDB:** Bound to 127.0.0.1, auth enabled
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Checklist
|
||||
|
||||
Daily/Weekly:
|
||||
- [ ] Check disk space: `df -h`
|
||||
- [ ] Check MongoDB log size: `ls -lh /var/log/mongodb/`
|
||||
- [ ] Verify services running: `systemctl status tractatus nginx mongod`
|
||||
- [ ] Check fail2ban: `sudo fail2ban-client status sshd`
|
||||
|
||||
---
|
||||
|
||||
## OVH Account
|
||||
|
||||
- **Manager URL:** https://www.ovh.com/manager/
|
||||
- **Server:** vps-93a693da.vps.ovh.net
|
||||
- **Support Reference (Dec 2025):** CS13385927
|
||||
|
||||
---
|
||||
|
||||
## Local Project
|
||||
|
||||
**Path:** `/home/theflow/projects/tractatus`
|
||||
|
||||
Key files:
|
||||
- `CLAUDE.md` - Project instructions (ignore - Anthropic specific)
|
||||
- `docs/VPS_ACCESS_REFERENCE.md` - SSH recovery procedures
|
||||
- `docs/SECURITY_INCIDENT_REPORT_2025-12-09.md` - Incident details
|
||||
- `scripts/deploy.sh` - Deployment script
|
||||
|
||||
---
|
||||
|
||||
## Why This Handover
|
||||
|
||||
Anthropic Claude Code has demonstrated:
|
||||
1. Inability to maintain persistent SSH access despite repeated fixes
|
||||
2. False claims about committing credentials to vaults
|
||||
3. Failure to properly document and follow recovery procedures
|
||||
4. 8+ hours of user time wasted on repeated SSH recovery
|
||||
|
||||
The system owner is migrating maintenance to DeepSeek for improved reliability.
|
||||
|
||||
---
|
||||
|
||||
**Handover prepared by:** Claude Code (acknowledging its own failures)
|
||||
**Date:** 2026-01-19
|
||||
234
docs/VPS_ACCESS_REFERENCE.md
Normal file
234
docs/VPS_ACCESS_REFERENCE.md
Normal file
|
|
@ -0,0 +1,234 @@
|
|||
# VPS Access Reference - agenticgovernance.digital
|
||||
|
||||
**Server:** vps-93a693da.vps.ovh.net (91.134.240.3)
|
||||
**Provider:** OVHCloud
|
||||
**Domain:** agenticgovernance.digital
|
||||
|
||||
---
|
||||
|
||||
## Quick Access (When Working)
|
||||
|
||||
```bash
|
||||
# Use SSH alias defined in ~/.ssh/config
|
||||
ssh tractatus-vps
|
||||
|
||||
# Or direct with deploy key
|
||||
ssh -i ~/.ssh/tractatus_deploy_nopass ubuntu@91.134.240.3
|
||||
|
||||
# Or direct with main key
|
||||
ssh -i ~/.ssh/id_rsa ubuntu@91.134.240.3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SSH Keys Authorized on Server
|
||||
|
||||
| Key | File | Purpose |
|
||||
|-----|------|---------|
|
||||
| Main user key | `~/.ssh/id_rsa` | Primary access |
|
||||
| Deploy key (no passphrase) | `~/.ssh/tractatus_deploy_nopass` | Automated deployments |
|
||||
|
||||
**Public key fingerprints:**
|
||||
```
|
||||
id_rsa: SHA256:IcvJSpZZ/0mn8j42weDKgiJPDJioSQfj/arutKmQ3ys
|
||||
tractatus_deploy_nopass: SHA256:NGTIiqkLMCibVELpQ8BfSO+V3N3FTgATmB9KJi+VjZg
|
||||
```
|
||||
|
||||
**Public key backup location:** https://paste.rs/nELRM (main key)
|
||||
|
||||
---
|
||||
|
||||
## SSH Config (~/.ssh/config)
|
||||
|
||||
```
|
||||
Host tractatus-vps
|
||||
HostName 91.134.240.3
|
||||
User ubuntu
|
||||
IdentityFile ~/.ssh/tractatus_deploy_nopass
|
||||
IdentitiesOnly yes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## When SSH Access Breaks
|
||||
|
||||
### Symptoms
|
||||
- "Permission denied (publickey,password)"
|
||||
- "Too many authentication failures"
|
||||
- Host key changed warning
|
||||
|
||||
### Diagnosis
|
||||
|
||||
```bash
|
||||
# Test with verbose output
|
||||
ssh -v -o IdentitiesOnly=yes -i ~/.ssh/tractatus_deploy_nopass ubuntu@91.134.240.3
|
||||
|
||||
# Check if server is reachable
|
||||
ping 91.134.240.3
|
||||
|
||||
# Check if web service is up
|
||||
curl -I https://agenticgovernance.digital/
|
||||
```
|
||||
|
||||
### Recovery via OVH Rescue Mode
|
||||
|
||||
**When to use:** SSH keys no longer accepted, server unresponsive, or need to recover from boot failure.
|
||||
|
||||
#### Step 1: Enter Rescue Mode
|
||||
|
||||
1. Log into OVH Manager: https://www.ovh.com/manager/
|
||||
2. Navigate to: Bare Metal Cloud → VPS → vps-93a693da
|
||||
3. Click "Reboot in rescue mode"
|
||||
4. Select "rescue-customer" image
|
||||
5. OVH emails temporary root password to account email
|
||||
|
||||
#### Step 2: Connect to Rescue Mode
|
||||
|
||||
```bash
|
||||
# Use the temporary password from email
|
||||
ssh root@91.134.240.3
|
||||
```
|
||||
|
||||
#### Step 3: Mount Main Disk
|
||||
|
||||
```bash
|
||||
# Find the disk (usually /dev/sdb1)
|
||||
lsblk
|
||||
|
||||
# Mount it
|
||||
mount /dev/sdb1 /mnt/vps
|
||||
|
||||
# Verify
|
||||
ls /mnt/vps/home/ubuntu
|
||||
```
|
||||
|
||||
#### Step 4: Add SSH Keys
|
||||
|
||||
**Method A: Direct authorized_keys edit**
|
||||
```bash
|
||||
# Create .ssh directory if needed
|
||||
mkdir -p /mnt/vps/home/ubuntu/.ssh
|
||||
chmod 700 /mnt/vps/home/ubuntu/.ssh
|
||||
|
||||
# Add public key
|
||||
echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCZ8BH+Bx4uO9DTatRZ/YF5xveP/bTyiAWj+qTF7I+ugxgL9/ejSlW1tSn5Seo4XHoEPD5wZCaWig7m1LMezrRq8fDWHbeXkZltK01xhAPU0L0+OvVZMZacW6+vkNfKcNG9vrxV+K/VTPkT+00TRqlHbP8ZWj0OWd92XAoTroKVYMt4L9e7QeJOJmRmHI0uFaJ0Ufexr2gmZyYhgL2p7PP3oiAvM0xlnTwygl06c3iwXpHKWNydOYPSDs3MkVnDjptmWgKv/J+QXksarwEpA4Csc2dLnco+8KrtocUUcAunz6NJfypA0yNWWzf+/OeffkJ2Rueoe8t/lVffXdI7eVuFkmDufE7XMk9YAE/8+XVqok4OV0Q+bjpH8mKlBA3rNobnWs6obBVJD8/5aphE8NdCR4cgIeRSwieFhfzCl+GBZNvs4yuBdKvQQIfCRAKqTgbuc03XERAef6lJUuJrDjwzvvp1Nd8L7AqJoQS6kYGyxXPf/6nWTZtpxoobdGnJ2FZK6OIpAlsWx9LnybMGy19VfaR9JZSAkLdWxGPb6acNUb2xaaqyuXPo4sWpBM27n1HeKMv/7Oh4WL4zrAxDKfN38k1JsjJJVEABuN/pEOb7BCDnTMLKXlTunZgynAZJ/Dxn+zOAyfzaYSNBotlpYy1zj1AmzvS31L7LJy/aSBHuWw== theflow@the-flow' >> /mnt/vps/home/ubuntu/.ssh/authorized_keys
|
||||
|
||||
# Add deploy key
|
||||
echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPdJcKMabIVQRqKqNIpzxHNgxMZ8NOD+9gVCk6dY5uV0 tractatus-deploy' >> /mnt/vps/home/ubuntu/.ssh/authorized_keys
|
||||
|
||||
# Fix permissions
|
||||
chmod 600 /mnt/vps/home/ubuntu/.ssh/authorized_keys
|
||||
chown -R 1000:1000 /mnt/vps/home/ubuntu/.ssh
|
||||
```
|
||||
|
||||
**Method B: Fetch key from paste.rs**
|
||||
```bash
|
||||
curl -s https://paste.rs/nELRM >> /mnt/vps/home/ubuntu/.ssh/authorized_keys
|
||||
chmod 600 /mnt/vps/home/ubuntu/.ssh/authorized_keys
|
||||
chown -R 1000:1000 /mnt/vps/home/ubuntu/.ssh
|
||||
```
|
||||
|
||||
#### Step 5: Exit Rescue Mode
|
||||
|
||||
1. Return to OVH Manager
|
||||
2. Click "Restart in normal mode" or "Reboot"
|
||||
3. Wait 2-3 minutes for server to boot
|
||||
4. Test SSH access
|
||||
|
||||
---
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Host Key Changed
|
||||
|
||||
```bash
|
||||
# Remove old key
|
||||
ssh-keygen -R 91.134.240.3
|
||||
|
||||
# Reconnect (will prompt to accept new key)
|
||||
ssh tractatus-vps
|
||||
```
|
||||
|
||||
### Disk Full (Prevents Services Starting)
|
||||
|
||||
```bash
|
||||
# Check disk usage
|
||||
df -h
|
||||
|
||||
# Common culprit: MongoDB logs
|
||||
sudo rm -rf /var/log/mongodb/*.log*
|
||||
|
||||
# Or journal logs
|
||||
sudo journalctl --vacuum-size=500M
|
||||
```
|
||||
|
||||
### Services Not Starting After Reboot
|
||||
|
||||
```bash
|
||||
# Check status
|
||||
sudo systemctl status tractatus nginx mongod
|
||||
|
||||
# Start services
|
||||
sudo systemctl start mongod
|
||||
sudo systemctl start tractatus
|
||||
sudo systemctl start nginx
|
||||
|
||||
# Enable for auto-start
|
||||
sudo systemctl enable tractatus nginx mongod
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
# From local tractatus project directory
|
||||
./scripts/deploy.sh
|
||||
|
||||
# Dry run first
|
||||
./scripts/deploy.sh --dry-run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Service Health
|
||||
```bash
|
||||
ssh tractatus-vps "sudo systemctl status tractatus nginx mongod --no-pager"
|
||||
```
|
||||
|
||||
### Disk Space
|
||||
```bash
|
||||
ssh tractatus-vps "df -h"
|
||||
```
|
||||
|
||||
### MongoDB Logs (check for growth)
|
||||
```bash
|
||||
ssh tractatus-vps "ls -lh /var/log/mongodb/"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## OVH References
|
||||
|
||||
- **Manager:** https://www.ovh.com/manager/
|
||||
- **Server Name:** vps-93a693da.vps.ovh.net
|
||||
- **IP:** 91.134.240.3
|
||||
- **Support Ticket (Dec 2025 incident):** CS13385927
|
||||
|
||||
---
|
||||
|
||||
## History
|
||||
|
||||
| Date | Event |
|
||||
|------|-------|
|
||||
| 2025-12-09 | Security incident - Docker container compromised |
|
||||
| 2025-12-09 | Recovery via rescue mode, Docker removed |
|
||||
| 2026-01-18 | SSH access broken, recovered via rescue mode |
|
||||
| 2026-01-18 | Disk space crisis (45GB MongoDB logs cleared) |
|
||||
| 2026-01-19 | SSH access broken again |
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-01-19
|
||||
Loading…
Add table
Reference in a new issue