tractatus/deployment-quickstart/UMAMI_SETUP_GUIDE.md
TheFlow d32be2c673 feat(api): implement research inquiry endpoint and Umami analytics
HIGH PRIORITY: Fixes production 404 error on research inquiry form

Research Inquiry API:
- Add POST /api/research-inquiry endpoint for form submissions
- Add admin endpoints for inquiry management (list, get, assign, respond, delete)
- Create ResearchInquiry model with MongoDB integration
- Add to moderation queue for human review (strategic quadrant)
- Include rate limiting (5 req/min) and CSRF protection
- Tested locally: endpoint responding, data saving to DB

Umami Analytics (Privacy-First):
- Add Docker Compose config for Umami + PostgreSQL
- Create nginx reverse proxy config with SSL support
- Implement privacy-first tracking script (DNT, opt-out, no cookies)
- Integrate tracking across 26 public HTML pages
- Exclude admin pages from tracking (privacy boundary)
- Add comprehensive deployment guide (UMAMI_SETUP_GUIDE.md)
- Environment variables added to .env.example

Files Created (9):
- src/models/ResearchInquiry.model.js
- src/controllers/research.controller.js
- src/routes/research.routes.js
- public/js/components/umami-tracker.js
- deployment-quickstart/nginx-analytics.conf
- deployment-quickstart/UMAMI_SETUP_GUIDE.md
- scripts/add-umami-tracking.sh
- scripts/add-tracking-python.py
- SESSION_SUMMARY_ANALYTICS_RESEARCH_INQUIRY.md

Files Modified (29):
- src/routes/index.js (research routes)
- deployment-quickstart/docker-compose.yml (umami services)
- deployment-quickstart/.env.example (umami config)
- 26 public HTML pages (tracking script)

Values Alignment:
 Privacy-First Design (cookie-free, DNT honored, opt-out available)
 Human Agency (research inquiries require human review)
 Data Sovereignty (self-hosted analytics, no third-party sharing)
 GDPR Compliance (no personal data in analytics)
 Transparency (open-source tools, documented setup)

Testing Status:
 Research inquiry: Locally tested, data verified in MongoDB
 Umami analytics: Pending production deployment

Next Steps:
1. Deploy to production (./scripts/deploy.sh)
2. Test research form on live site
3. Deploy Umami following UMAMI_SETUP_GUIDE.md
4. Update umami-tracker.js with website ID after setup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 01:31:02 +13:00

14 KiB

Umami Analytics Setup Guide

Complete guide for deploying privacy-preserving Umami analytics for the Tractatus project.


Overview

Umami is a privacy-first, GDPR-compliant, open-source web analytics alternative that:

  • No cookies
  • No personal data collection
  • No cross-site tracking
  • GDPR/CCPA compliant by default
  • Lightweight (<2KB tracking script)
  • Self-hosted (full data sovereignty)

Prerequisites

  1. Server Requirements:

    • Docker and Docker Compose installed
    • Domain/subdomain configured: analytics.agenticgovernance.digital
    • SSL certificate (via Certbot/Let's Encrypt)
  2. Environment Variables:

    • Copy .env.example to .env
    • Generate secrets with: openssl rand -base64 32

Step 1: Configure Environment Variables

Edit .env and set the following:

# Umami Analytics Configuration
UMAMI_APP_SECRET=<generate-with-openssl-rand-base64-32>
UMAMI_DB_NAME=umami
UMAMI_DB_USER=umami
UMAMI_DB_PASSWORD=<generate-secure-password>
UMAMI_PORT=3000
UMAMI_TRACKER_SCRIPT=umami
UMAMI_DISABLE_TELEMETRY=1

# Enable analytics in main app
ANALYTICS_ENABLED=true

Step 2: Deploy Umami with Docker Compose

From the deployment-quickstart directory:

# Start Umami and PostgreSQL containers
docker-compose up -d umami umami-db

# Check container status
docker-compose ps

# View logs
docker-compose logs -f umami

Expected output:

tractatus-umami       | Server running on port 3000
tractatus-umami-db    | database system is ready to accept connections

Step 3: Initial Umami Setup

  1. Access Umami dashboard (locally first):

    # Test locally before DNS/nginx setup
    ssh -L 3000:localhost:3000 ubuntu@vps-93a693da.vps.ovh.net
    # Then open: http://localhost:3000
    
  2. First login:

    • Username: admin
    • Password: umami
    • IMMEDIATELY change password!
  3. Create website:

    • Click "Add website"
    • Name: Tractatus Framework
    • Domain: agenticgovernance.digital
    • Timezone: Your preference
    • Copy the tracking code (we'll use the website ID)
  4. Get tracking script details:

    • Website ID: Will look like a1b2c3d4-e5f6-7890-abcd-ef1234567890
    • Tracking script: <script async src="https://analytics.agenticgovernance.digital/script.js" data-website-id="YOUR-WEBSITE-ID"></script>

Step 4: Configure Nginx Reverse Proxy with SSL

A. DNS Configuration

Add an A record for the analytics subdomain:

Type: A
Name: analytics
Value: <your-vps-ip-address>
TTL: 300 (or default)

Verify DNS propagation:

dig analytics.agenticgovernance.digital

B. Install Nginx (if not already installed)

sudo apt update
sudo apt install nginx certbot python3-certbot-nginx

C. Copy Nginx Configuration

# On VPS
sudo cp /path/to/deployment-quickstart/nginx-analytics.conf /etc/nginx/sites-available/analytics.agenticgovernance.digital

# Enable site
sudo ln -s /etc/nginx/sites-available/analytics.agenticgovernance.digital /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload nginx
sudo systemctl reload nginx

D. Obtain SSL Certificate

# Use Certbot to automatically configure SSL
sudo certbot --nginx -d analytics.agenticgovernance.digital

# Follow prompts:
# - Enter email address
# - Agree to Terms of Service
# - Choose redirect (option 2: redirect HTTP to HTTPS)

E. Verify SSL Auto-Renewal

# Test renewal process (dry run)
sudo certbot renew --dry-run

# Certbot auto-renewal is enabled by default via systemd timer
sudo systemctl status certbot.timer

Step 5: Integrate Tracking Script Across All Pages

A. Create Tracking Script Component

Create public/js/components/umami-tracker.js:

/**
 * Umami Analytics - Privacy-First Tracking
 * No cookies, no personal data, GDPR-compliant
 */

(function() {
  'use strict';

  // Only load if analytics is enabled and not in development
  if (window.location.hostname === 'localhost' || window.location.hostname === '127.0.0.1') {
    console.log('[Analytics] Disabled in development environment');
    return;
  }

  // Check if user has opted out (respect DNT header and localStorage preference)
  const dnt = navigator.doNotTrack || window.doNotTrack || navigator.msDoNotTrack;
  const optedOut = localStorage.getItem('umami.disabled') === 'true';

  if (dnt === '1' || dnt === 'yes' || optedOut) {
    console.log('[Analytics] Tracking disabled (DNT or user preference)');
    return;
  }

  // Umami configuration
  const UMAMI_WEBSITE_ID = 'YOUR-WEBSITE-ID-HERE'; // Replace with actual website ID
  const UMAMI_SRC = 'https://analytics.agenticgovernance.digital/script.js';

  // Load Umami tracking script
  const script = document.createElement('script');
  script.async = true;
  script.defer = true;
  script.src = UMAMI_SRC;
  script.setAttribute('data-website-id', UMAMI_WEBSITE_ID);
  script.setAttribute('data-domains', 'agenticgovernance.digital');
  script.setAttribute('data-auto-track', 'true');

  // Error handling
  script.onerror = function() {
    console.warn('[Analytics] Failed to load Umami tracking script');
  };

  // Append script to head
  document.head.appendChild(script);

  console.log('[Analytics] Umami tracker loaded');
})();

B. Add Script to All HTML Pages

Add the following to the <head> section of every HTML page:

<!-- Privacy-Preserving Analytics (Umami - GDPR Compliant, No Cookies) -->
<script src="/js/components/umami-tracker.js"></script>

Files to update:

  • public/index.html
  • public/about.html
  • public/advocate.html
  • public/researcher.html
  • public/implementer.html
  • public/leader.html
  • public/docs.html
  • public/blog.html
  • public/blog-post.html
  • public/case-submission.html
  • public/media-inquiry.html
  • public/privacy.html
  • public/gdpr.html
  • public/demos/*.html

Exclude from tracking:

  • public/admin/*.html (admin panel should not be tracked)

Step 6: Update Privacy Policy

Add the following section to public/privacy.html:

Analytics Section

## Website Analytics

We use **Umami Analytics**, a privacy-first, open-source analytics tool to understand how visitors use our website. Umami is fully GDPR and CCPA compliant.

**What Umami collects (all anonymized):**
- Page views
- Referrer sources (where visitors came from)
- Browser type (e.g., Chrome, Firefox)
- Device type (desktop, mobile, tablet)
- Country (derived from IP address, not stored)
- Operating system (general categories only)

**What Umami does NOT collect:**
- Individual IP addresses
- Personal identifiable information (PII)
- Cookies or persistent identifiers
- Cross-site tracking data
- Individual user behavior

**Your rights:**
- Analytics are cookie-free and anonymous
- You can opt out by enabling Do Not Track (DNT) in your browser
- You can disable analytics by visiting [Opt-Out Page]
- View our analytics transparency: [Public Dashboard] (if enabled)

**Data sovereignty:**
- All analytics data is self-hosted on our servers
- No third-party access to analytics data
- Data retention: 12 months (configurable)

For more information, see [Umami's privacy policy](https://umami.is/privacy).

Step 7: Testing

A. Test Tracking (Local)

  1. Open browser DevTools (F12)
  2. Navigate to Network tab
  3. Visit: https://agenticgovernance.digital
  4. Look for request to: https://analytics.agenticgovernance.digital/api/send
  5. Should see 200 OK response

B. Test Dashboard

  1. Login to: https://analytics.agenticgovernance.digital
  2. Navigate to Websites → Tractatus Framework
  3. Should see real-time visitor data appear within 1-2 minutes

C. Test Do Not Track (DNT)

  1. Enable DNT in browser settings
  2. Reload page
  3. Verify no tracking request is sent
  4. Check console: "Tracking disabled (DNT or user preference)"

Step 8: Optional - Public Dashboard

To enable public transparency:

  1. In Umami dashboard, go to Website Settings
  2. Enable "Share URL"
  3. Copy share URL
  4. Add link to website footer or privacy page:
    <a href="https://analytics.agenticgovernance.digital/share/YOUR-SHARE-ID" target="_blank">
      View Analytics (Public)
    </a>
    

Security Checklist

  • Changed default Umami admin password
  • Generated secure UMAMI_APP_SECRET
  • Set strong UMAMI_DB_PASSWORD
  • SSL certificate installed and auto-renewal enabled
  • Nginx security headers configured
  • Firewall rules allow ports 80, 443
  • Docker containers running with restart policy
  • Backup strategy for PostgreSQL data
  • Privacy policy updated
  • DNT (Do Not Track) respected
  • Admin panel excluded from tracking

Maintenance

Backup Umami Database

# Backup PostgreSQL data
docker-compose exec umami-db pg_dump -U umami umami > umami-backup-$(date +%Y%m%d).sql

# Restore from backup
cat umami-backup-20250128.sql | docker-compose exec -T umami-db psql -U umami umami

Update Umami

# Pull latest image
docker-compose pull umami

# Restart container
docker-compose up -d umami

Monitor Logs

# Umami application logs
docker-compose logs -f umami

# PostgreSQL logs
docker-compose logs -f umami-db

# Nginx logs
sudo tail -f /var/log/nginx/umami-access.log
sudo tail -f /var/log/nginx/umami-error.log

Troubleshooting

Issue: Umami dashboard not accessible

Solution:

  1. Check container status: docker-compose ps
  2. Check logs: docker-compose logs umami
  3. Verify port 3000 is accessible: curl http://localhost:3000/api/heartbeat
  4. Check nginx configuration: sudo nginx -t

Issue: Tracking script not loading

Solution:

  1. Check browser console for errors
  2. Verify DNS: dig analytics.agenticgovernance.digital
  3. Test direct access: curl https://analytics.agenticgovernance.digital/script.js
  4. Check CSP headers (may block external scripts)

Issue: No data appearing in dashboard

Solution:

  1. Verify website ID in tracking script matches dashboard
  2. Check browser DevTools → Network tab for /api/send request
  3. Ensure DNT is not enabled
  4. Clear browser cache and test in incognito

Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                    User's Browser                           │
│  https://agenticgovernance.digital                          │
│  ┌─────────────────────────────────────────┐                │
│  │  <script src="...analytics.../script.js" │                │
│  │  data-website-id="..." />                │                │
│  └─────────────────────────────────────────┘                │
└───────────────────────┬─────────────────────────────────────┘
                        │
                        │ HTTPS (anonymized pageview)
                        ▼
┌─────────────────────────────────────────────────────────────┐
│              Nginx Reverse Proxy (Port 443)                 │
│        analytics.agenticgovernance.digital                  │
│  - SSL Termination (Let's Encrypt)                          │
│  - Security Headers                                         │
│  - Proxy to Umami Container                                 │
└───────────────────────┬─────────────────────────────────────┘
                        │
                        │ HTTP (internal)
                        ▼
┌─────────────────────────────────────────────────────────────┐
│         Docker Container: tractatus-umami (Port 3000)       │
│  - Umami Application (Node.js/Next.js)                      │
│  - Tracking API (/api/send)                                 │
│  - Dashboard UI (/dashboard)                                │
└───────────────────────┬─────────────────────────────────────┘
                        │
                        │ PostgreSQL Connection
                        ▼
┌─────────────────────────────────────────────────────────────┐
│      Docker Container: tractatus-umami-db (Port 5432)       │
│  - PostgreSQL 15                                            │
│  - Database: umami                                          │
│  - Volume: umami_db_data (persistent storage)               │
└─────────────────────────────────────────────────────────────┘

Values Alignment

This implementation aligns with Tractatus core values:

  1. Privacy-First Design: Cookie-free, no personal data collection
  2. Transparency: Open-source, public dashboard option
  3. Data Sovereignty: Self-hosted, no third-party data sharing
  4. User Respect: Honors DNT, provides opt-out
  5. No Proprietary Lock-in: Open-source (MIT license), portable data

Resources


Last Updated: 2025-10-29 Maintained by: Tractatus Project