tractatus/docs/SECURITY_INCIDENT_REPORT_2026-02-11.md

Security Incident Report: Internal Documentation on Production Server

Incident ID: SEC-2026-02-11-001 Severity: HIGH Date Discovered: 2026-02-11 Date Resolved: 2026-02-11 Reported By: User (during routine session) Status: RESOLVED

---

Summary

356 internal documentation files (19MB) were present on the production server filesystem at /var/www/tractatus/docs/ for approximately 4 months. Files included credential rotation procedures, VPS access references, Stripe financial configuration details, security audit reports, previous incident reports, session handoffs, and deployment architecture documentation.

Exposure Assessment

HTTP accessibility: NO. The files were NOT served by the web application. Express serves only public/ as static files (app.use(express.static('public'))). Nginx proxies all requests to Express with no direct static file serving from the docs/ directory. All tested HTTP requests to docs/*.md returned 404.

Note: public/docs/ (a separate directory containing SVGs and PDFs referenced by the site) IS intentionally web-served and was not part of this incident.

Filesystem accessibility: YES. Files were world-readable (permissions 664, owner ubuntu:ubuntu) on the production server for the full exposure window.

Active probes detected in nginx logs:

• 158.220.97.237 probed /docs/.env on 2026-01-30 (4 attempts, all returned 400/404)
• 164.68.124.190 probed /docs/.env on 2026-02-11 (returned 404)
• 91.134.240.3 probed /docs/CREDENTIAL_ROTATION_PROCEDURES.md on 2026-02-11 (returned 404)

All probes received 404 responses. No evidence of successful data exfiltration via HTTP.

Exposure Window

• First file deployed: 2025-10-06 (session-handoff-2025-10-07-part2.md)
• Last file deployed: 2026-02-11 (research-timeline.md)
• Total window: ~128 days (2025-10-06 to 2026-02-11)
• Files removed: 2026-02-11T21:15:00Z (approximate)

Sensitive Files Present on Production

Critical (credentials, access, financial)

File	Content
CREDENTIAL_ROTATION_PROCEDURES.md	Credential rotation procedures and schedules
CREDENTIAL_VAULT_SPECIFICATION.md	Vault architecture and credential storage design
VPS_ACCESS_REFERENCE.md	VPS access instructions and connection details
VPS_RECOVERY_REFERENCE.md	Server recovery procedures
STRIPE_LIVE_MODE_DEPLOYMENT.md	Live Stripe payment configuration
STRIPE_SANDBOX_SETUP_COMPLETE.md	Stripe sandbox configuration
STRIPE_ACCOUNT_NAME_FIX.md	Stripe account details
STRIPE_BANK_NAME_MATCHING.md	Bank account matching details
STRIPE_PAYOUT_DIAGNOSTIC.md	Payout diagnostic information
STRIPE_FIX_FOR_JOHN_STROH.md	Personal financial account details
FIND_STRIPE_BANK_HOLDER_NAME.md	Bank holder name lookup
KOHA_STRIPE_SETUP.md	Koha payment integration details
stripe-analysis/ (6 files)	Stripe security audits and account analysis

High (security architecture, incidents)

File	Content
SECURITY-AUDIT-2025-10-09.md	Full security audit findings
SECURITY_AUDIT_REPORT.md	Security audit report
SECURITY_AUDIT_TEMPLATE_VPS.md	VPS audit template (reveals assessment criteria)
SECURITY_INCIDENT_REPORT_2025-12-09.md	Previous security incident details
INCIDENT_RECOVERY_2026-01-19.md	Incident recovery procedures
KOHA-SECURITY-AUDIT-2025-10-09.md	Koha security audit
DOCUMENT_SECURITY_GOVERNANCE.md	Security governance architecture
plans/security-implementation-roadmap.md	Security roadmap (reveals gaps)
plans/security-implementation-tracker.md	Security implementation status
framework-incidents/ (3 files)	Framework violation and bypass incidents

Medium (internal architecture, session data)

Category	Count	Examples
Session handoffs	~20 files	Internal session state, priorities, work plans
Deployment documentation	~8 files	Deployment guides, checklists, rsync patterns
Phase 2 plans	~10 files	Infrastructure plans, cost estimates, roadmaps
Framework documentation	~15 files	Framework architecture, improvements, rules
Internal reports	~10 files	Business intelligence, stakeholder recruitment

Low (operational, non-sensitive)

Category	Count
Glossary, FAQ, general docs	~20 files
Architecture diagrams (mermaid)	2 files
Research data (published)	~10 files
Markdown source (published docs)	~15 files

Total: 356 files across 35 directories

Root Cause

The deployment script (scripts/deploy.sh) uses rsync with a .rsyncignore file to exclude sensitive content during full deployments. The .rsyncignore used a denylist approach for the docs/ directory — attempting to enumerate every sensitive file pattern:

docs/session-handoff-*.md
docs/SESSION_MANAGEMENT_*.md
docs/draft-emails-*.md
docs/precis-*.md
docs/SECURITY_AUDIT_REPORT.md
docs/FRAMEWORK_FAILURE_*.md
docs/PHASE-2-*.md
docs/IMPLEMENTATION_PROGRESS_*.md
docs/DOCUMENT_SECURITY_GOVERNANCE.md

This failed because:

1. Denylist incompleteness: The list covered ~12 patterns but the directory contained 356 files across dozens of naming conventions. Files like CREDENTIAL_ROTATION_PROCEDURES.md, VPS_ACCESS_REFERENCE.md, STRIPE_*.md, and all subdirectories were never added to the exclusion list.
2. No production requirement: The docs/ directory is not used by any production code. Express serves public/ only. There was no reason for any docs/ file to be on the server.
3. Recurrence pattern: The user reports this is the third or fourth time sensitive files have been found on production in the past year. Each previous fix added specific patterns to the denylist rather than addressing the structural problem.

Remediation

Immediate (2026-02-11)

1. Removed all files: rm -rf /var/www/tractatus/docs/ on production server. Verified directory no longer exists.
2. Application verified: Express app confirmed running (HTTP 200, systemd active) — no dependency on docs/.
3. Rsync exclusion fixed: Changed .rsyncignore from denylist to full directory exclusion:

# BEFORE (denylist - failed repeatedly):
docs/session-handoff-*.md
docs/SESSION_MANAGEMENT_*.md
docs/draft-emails-*.md
# ... 9 more specific patterns

# AFTER (allowlist - structural fix):
docs/
docs/**

4. Dry-run verified: rsync -avzn with updated .rsyncignore confirms zero docs/ files would be synced.

Structural (prevent recurrence)

The .rsyncignore now excludes docs/ entirely. Since no production code reads from this directory, this is safe. The public/docs/ directory (SVGs, PDFs) is unaffected — it lives inside public/ which is synced and served normally.

Why This Keeps Happening

This is the structural pattern across recurring incidents:

1. Denylist governance fails. Every time a new sensitive file is created, it must be manually added to the exclusion list. This requires the person creating the file to remember the exclusion list exists, know the correct pattern syntax, and test it. In practice, none of these happen consistently — especially when the file creator is an AI assistant responding to immediate task needs.

2. The deploy script checks the wrong thing. The check-confidential-docs.js pre-commit hook scans for confidential markers but only catches files with explicit confidential headers. It does not catch files that are sensitive by nature (credential procedures, security audits) but lack such headers.

3. Full deploys sync everything not excluded. The rsync --delete approach means the production server is a mirror of local minus exclusions. This is an allowlist problem being solved with a denylist tool.

Mitigating Factors

• Files were not HTTP-accessible (Express serves only public/)
• Nginx does not serve static files from the docs/ path
• All detected probe attempts received 404 responses
• No evidence of server compromise during the exposure window
• The most sensitive files (.env, SSH keys) were correctly excluded by other .rsyncignore rules

Recommendations

1. DONE: Exclude docs/ entirely from rsync
2. Consider: Moving to an allowlist deployment model (only sync public/, src/, package.json, etc.) rather than syncing everything-minus-exclusions
3. Consider: Adding a post-deploy verification step that checks for unexpected directories on production
4. Consider: Regular automated audit comparing production filesystem against expected deployment manifest

---

Resolved: 2026-02-11T21:15:00Z Verified by: Dry-run rsync, production HTTP tests, application health check