tractatus/deployment-quickstart/UMAMI_SETUP_GUIDE.md
TheFlow d32be2c673 feat(api): implement research inquiry endpoint and Umami analytics
HIGH PRIORITY: Fixes production 404 error on research inquiry form

Research Inquiry API:
- Add POST /api/research-inquiry endpoint for form submissions
- Add admin endpoints for inquiry management (list, get, assign, respond, delete)
- Create ResearchInquiry model with MongoDB integration
- Add to moderation queue for human review (strategic quadrant)
- Include rate limiting (5 req/min) and CSRF protection
- Tested locally: endpoint responding, data saving to DB

Umami Analytics (Privacy-First):
- Add Docker Compose config for Umami + PostgreSQL
- Create nginx reverse proxy config with SSL support
- Implement privacy-first tracking script (DNT, opt-out, no cookies)
- Integrate tracking across 26 public HTML pages
- Exclude admin pages from tracking (privacy boundary)
- Add comprehensive deployment guide (UMAMI_SETUP_GUIDE.md)
- Environment variables added to .env.example

Files Created (9):
- src/models/ResearchInquiry.model.js
- src/controllers/research.controller.js
- src/routes/research.routes.js
- public/js/components/umami-tracker.js
- deployment-quickstart/nginx-analytics.conf
- deployment-quickstart/UMAMI_SETUP_GUIDE.md
- scripts/add-umami-tracking.sh
- scripts/add-tracking-python.py
- SESSION_SUMMARY_ANALYTICS_RESEARCH_INQUIRY.md

Files Modified (29):
- src/routes/index.js (research routes)
- deployment-quickstart/docker-compose.yml (umami services)
- deployment-quickstart/.env.example (umami config)
- 26 public HTML pages (tracking script)

Values Alignment:
 Privacy-First Design (cookie-free, DNT honored, opt-out available)
 Human Agency (research inquiries require human review)
 Data Sovereignty (self-hosted analytics, no third-party sharing)
 GDPR Compliance (no personal data in analytics)
 Transparency (open-source tools, documented setup)

Testing Status:
 Research inquiry: Locally tested, data verified in MongoDB
 Umami analytics: Pending production deployment

Next Steps:
1. Deploy to production (./scripts/deploy.sh)
2. Test research form on live site
3. Deploy Umami following UMAMI_SETUP_GUIDE.md
4. Update umami-tracker.js with website ID after setup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 01:31:02 +13:00

483 lines
14 KiB
Markdown

# Umami Analytics Setup Guide
Complete guide for deploying privacy-preserving Umami analytics for the Tractatus project.
---
## Overview
**Umami** is a privacy-first, GDPR-compliant, open-source web analytics alternative that:
- ✅ No cookies
- ✅ No personal data collection
- ✅ No cross-site tracking
- ✅ GDPR/CCPA compliant by default
- ✅ Lightweight (<2KB tracking script)
- Self-hosted (full data sovereignty)
---
## Prerequisites
1. **Server Requirements:**
- Docker and Docker Compose installed
- Domain/subdomain configured: `analytics.agenticgovernance.digital`
- SSL certificate (via Certbot/Let's Encrypt)
2. **Environment Variables:**
- Copy `.env.example` to `.env`
- Generate secrets with: `openssl rand -base64 32`
---
## Step 1: Configure Environment Variables
Edit `.env` and set the following:
```bash
# Umami Analytics Configuration
UMAMI_APP_SECRET=<generate-with-openssl-rand-base64-32>
UMAMI_DB_NAME=umami
UMAMI_DB_USER=umami
UMAMI_DB_PASSWORD=<generate-secure-password>
UMAMI_PORT=3000
UMAMI_TRACKER_SCRIPT=umami
UMAMI_DISABLE_TELEMETRY=1
# Enable analytics in main app
ANALYTICS_ENABLED=true
```
---
## Step 2: Deploy Umami with Docker Compose
From the `deployment-quickstart` directory:
```bash
# Start Umami and PostgreSQL containers
docker-compose up -d umami umami-db
# Check container status
docker-compose ps
# View logs
docker-compose logs -f umami
```
**Expected output:**
```
tractatus-umami | Server running on port 3000
tractatus-umami-db | database system is ready to accept connections
```
---
## Step 3: Initial Umami Setup
1. **Access Umami dashboard (locally first):**
```bash
# Test locally before DNS/nginx setup
ssh -L 3000:localhost:3000 ubuntu@vps-93a693da.vps.ovh.net
# Then open: http://localhost:3000
```
2. **First login:**
- Username: `admin`
- Password: `umami`
- **IMMEDIATELY change password!**
3. **Create website:**
- Click "Add website"
- Name: `Tractatus Framework`
- Domain: `agenticgovernance.digital`
- Timezone: Your preference
- **Copy the tracking code** (we'll use the website ID)
4. **Get tracking script details:**
- Website ID: Will look like `a1b2c3d4-e5f6-7890-abcd-ef1234567890`
- Tracking script: `<script async src="https://analytics.agenticgovernance.digital/script.js" data-website-id="YOUR-WEBSITE-ID"></script>`
---
## Step 4: Configure Nginx Reverse Proxy with SSL
### A. DNS Configuration
Add an A record for the analytics subdomain:
```
Type: A
Name: analytics
Value: <your-vps-ip-address>
TTL: 300 (or default)
```
Verify DNS propagation:
```bash
dig analytics.agenticgovernance.digital
```
### B. Install Nginx (if not already installed)
```bash
sudo apt update
sudo apt install nginx certbot python3-certbot-nginx
```
### C. Copy Nginx Configuration
```bash
# On VPS
sudo cp /path/to/deployment-quickstart/nginx-analytics.conf /etc/nginx/sites-available/analytics.agenticgovernance.digital
# Enable site
sudo ln -s /etc/nginx/sites-available/analytics.agenticgovernance.digital /etc/nginx/sites-enabled/
# Test configuration
sudo nginx -t
# Reload nginx
sudo systemctl reload nginx
```
### D. Obtain SSL Certificate
```bash
# Use Certbot to automatically configure SSL
sudo certbot --nginx -d analytics.agenticgovernance.digital
# Follow prompts:
# - Enter email address
# - Agree to Terms of Service
# - Choose redirect (option 2: redirect HTTP to HTTPS)
```
### E. Verify SSL Auto-Renewal
```bash
# Test renewal process (dry run)
sudo certbot renew --dry-run
# Certbot auto-renewal is enabled by default via systemd timer
sudo systemctl status certbot.timer
```
---
## Step 5: Integrate Tracking Script Across All Pages
### A. Create Tracking Script Component
Create `public/js/components/umami-tracker.js`:
```javascript
/**
* Umami Analytics - Privacy-First Tracking
* No cookies, no personal data, GDPR-compliant
*/
(function() {
'use strict';
// Only load if analytics is enabled and not in development
if (window.location.hostname === 'localhost' || window.location.hostname === '127.0.0.1') {
console.log('[Analytics] Disabled in development environment');
return;
}
// Check if user has opted out (respect DNT header and localStorage preference)
const dnt = navigator.doNotTrack || window.doNotTrack || navigator.msDoNotTrack;
const optedOut = localStorage.getItem('umami.disabled') === 'true';
if (dnt === '1' || dnt === 'yes' || optedOut) {
console.log('[Analytics] Tracking disabled (DNT or user preference)');
return;
}
// Umami configuration
const UMAMI_WEBSITE_ID = 'YOUR-WEBSITE-ID-HERE'; // Replace with actual website ID
const UMAMI_SRC = 'https://analytics.agenticgovernance.digital/script.js';
// Load Umami tracking script
const script = document.createElement('script');
script.async = true;
script.defer = true;
script.src = UMAMI_SRC;
script.setAttribute('data-website-id', UMAMI_WEBSITE_ID);
script.setAttribute('data-domains', 'agenticgovernance.digital');
script.setAttribute('data-auto-track', 'true');
// Error handling
script.onerror = function() {
console.warn('[Analytics] Failed to load Umami tracking script');
};
// Append script to head
document.head.appendChild(script);
console.log('[Analytics] Umami tracker loaded');
})();
```
### B. Add Script to All HTML Pages
Add the following to the `<head>` section of **every HTML page**:
```html
<!-- Privacy-Preserving Analytics (Umami - GDPR Compliant, No Cookies) -->
<script src="/js/components/umami-tracker.js"></script>
```
**Files to update:**
- `public/index.html`
- `public/about.html`
- `public/advocate.html`
- `public/researcher.html`
- `public/implementer.html`
- `public/leader.html`
- `public/docs.html`
- `public/blog.html`
- `public/blog-post.html`
- `public/case-submission.html`
- `public/media-inquiry.html`
- `public/privacy.html`
- `public/gdpr.html`
- `public/demos/*.html`
**Exclude from tracking:**
- `public/admin/*.html` (admin panel should not be tracked)
---
## Step 6: Update Privacy Policy
Add the following section to `public/privacy.html`:
### Analytics Section
```markdown
## Website Analytics
We use **Umami Analytics**, a privacy-first, open-source analytics tool to understand how visitors use our website. Umami is fully GDPR and CCPA compliant.
**What Umami collects (all anonymized):**
- Page views
- Referrer sources (where visitors came from)
- Browser type (e.g., Chrome, Firefox)
- Device type (desktop, mobile, tablet)
- Country (derived from IP address, not stored)
- Operating system (general categories only)
**What Umami does NOT collect:**
- Individual IP addresses
- Personal identifiable information (PII)
- Cookies or persistent identifiers
- Cross-site tracking data
- Individual user behavior
**Your rights:**
- Analytics are cookie-free and anonymous
- You can opt out by enabling Do Not Track (DNT) in your browser
- You can disable analytics by visiting [Opt-Out Page]
- View our analytics transparency: [Public Dashboard] (if enabled)
**Data sovereignty:**
- All analytics data is self-hosted on our servers
- No third-party access to analytics data
- Data retention: 12 months (configurable)
For more information, see [Umami's privacy policy](https://umami.is/privacy).
```
---
## Step 7: Testing
### A. Test Tracking (Local)
1. Open browser DevTools (F12)
2. Navigate to Network tab
3. Visit: https://agenticgovernance.digital
4. Look for request to: `https://analytics.agenticgovernance.digital/api/send`
5. Should see 200 OK response
### B. Test Dashboard
1. Login to: https://analytics.agenticgovernance.digital
2. Navigate to Websites → Tractatus Framework
3. Should see real-time visitor data appear within 1-2 minutes
### C. Test Do Not Track (DNT)
1. Enable DNT in browser settings
2. Reload page
3. Verify no tracking request is sent
4. Check console: "Tracking disabled (DNT or user preference)"
---
## Step 8: Optional - Public Dashboard
To enable public transparency:
1. In Umami dashboard, go to Website Settings
2. Enable "Share URL"
3. Copy share URL
4. Add link to website footer or privacy page:
```html
<a href="https://analytics.agenticgovernance.digital/share/YOUR-SHARE-ID" target="_blank">
View Analytics (Public)
</a>
```
---
## Security Checklist
- [ ] Changed default Umami admin password
- [ ] Generated secure `UMAMI_APP_SECRET`
- [ ] Set strong `UMAMI_DB_PASSWORD`
- [ ] SSL certificate installed and auto-renewal enabled
- [ ] Nginx security headers configured
- [ ] Firewall rules allow ports 80, 443
- [ ] Docker containers running with restart policy
- [ ] Backup strategy for PostgreSQL data
- [ ] Privacy policy updated
- [ ] DNT (Do Not Track) respected
- [ ] Admin panel excluded from tracking
---
## Maintenance
### Backup Umami Database
```bash
# Backup PostgreSQL data
docker-compose exec umami-db pg_dump -U umami umami > umami-backup-$(date +%Y%m%d).sql
# Restore from backup
cat umami-backup-20250128.sql | docker-compose exec -T umami-db psql -U umami umami
```
### Update Umami
```bash
# Pull latest image
docker-compose pull umami
# Restart container
docker-compose up -d umami
```
### Monitor Logs
```bash
# Umami application logs
docker-compose logs -f umami
# PostgreSQL logs
docker-compose logs -f umami-db
# Nginx logs
sudo tail -f /var/log/nginx/umami-access.log
sudo tail -f /var/log/nginx/umami-error.log
```
---
## Troubleshooting
### Issue: Umami dashboard not accessible
**Solution:**
1. Check container status: `docker-compose ps`
2. Check logs: `docker-compose logs umami`
3. Verify port 3000 is accessible: `curl http://localhost:3000/api/heartbeat`
4. Check nginx configuration: `sudo nginx -t`
### Issue: Tracking script not loading
**Solution:**
1. Check browser console for errors
2. Verify DNS: `dig analytics.agenticgovernance.digital`
3. Test direct access: `curl https://analytics.agenticgovernance.digital/script.js`
4. Check CSP headers (may block external scripts)
### Issue: No data appearing in dashboard
**Solution:**
1. Verify website ID in tracking script matches dashboard
2. Check browser DevTools → Network tab for `/api/send` request
3. Ensure DNT is not enabled
4. Clear browser cache and test in incognito
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────┐
│ User's Browser │
│ https://agenticgovernance.digital │
│ ┌─────────────────────────────────────────┐ │
│ │ <script src="...analytics.../script.js" │ │
│ │ data-website-id="..." /> │ │
│ └─────────────────────────────────────────┘ │
└───────────────────────┬─────────────────────────────────────┘
│ HTTPS (anonymized pageview)
┌─────────────────────────────────────────────────────────────┐
│ Nginx Reverse Proxy (Port 443) │
│ analytics.agenticgovernance.digital │
│ - SSL Termination (Let's Encrypt) │
│ - Security Headers │
│ - Proxy to Umami Container │
└───────────────────────┬─────────────────────────────────────┘
│ HTTP (internal)
┌─────────────────────────────────────────────────────────────┐
│ Docker Container: tractatus-umami (Port 3000) │
│ - Umami Application (Node.js/Next.js) │
│ - Tracking API (/api/send) │
│ - Dashboard UI (/dashboard) │
└───────────────────────┬─────────────────────────────────────┘
│ PostgreSQL Connection
┌─────────────────────────────────────────────────────────────┐
│ Docker Container: tractatus-umami-db (Port 5432) │
│ - PostgreSQL 15 │
│ - Database: umami │
│ - Volume: umami_db_data (persistent storage) │
└─────────────────────────────────────────────────────────────┘
```
---
## Values Alignment
This implementation aligns with Tractatus core values:
1. **Privacy-First Design**: Cookie-free, no personal data collection
2. **Transparency**: Open-source, public dashboard option
3. **Data Sovereignty**: Self-hosted, no third-party data sharing
4. **User Respect**: Honors DNT, provides opt-out
5. **No Proprietary Lock-in**: Open-source (MIT license), portable data
---
## Resources
- Umami Documentation: https://umami.is/docs
- Umami GitHub: https://github.com/umami-software/umami
- GDPR Compliance: https://umami.is/docs/guides/gdpr-compliance
- Nginx Configuration: https://nginx.org/en/docs/
- Let's Encrypt: https://letsencrypt.org/
---
**Last Updated:** 2025-10-29
**Maintained by:** Tractatus Project