HIGH PRIORITY: Fixes production 404 error on research inquiry form
Research Inquiry API:
- Add POST /api/research-inquiry endpoint for form submissions
- Add admin endpoints for inquiry management (list, get, assign, respond, delete)
- Create ResearchInquiry model with MongoDB integration
- Add to moderation queue for human review (strategic quadrant)
- Include rate limiting (5 req/min) and CSRF protection
- Tested locally: endpoint responding, data saving to DB
Umami Analytics (Privacy-First):
- Add Docker Compose config for Umami + PostgreSQL
- Create nginx reverse proxy config with SSL support
- Implement privacy-first tracking script (DNT, opt-out, no cookies)
- Integrate tracking across 26 public HTML pages
- Exclude admin pages from tracking (privacy boundary)
- Add comprehensive deployment guide (UMAMI_SETUP_GUIDE.md)
- Environment variables added to .env.example
Files Created (9):
- src/models/ResearchInquiry.model.js
- src/controllers/research.controller.js
- src/routes/research.routes.js
- public/js/components/umami-tracker.js
- deployment-quickstart/nginx-analytics.conf
- deployment-quickstart/UMAMI_SETUP_GUIDE.md
- scripts/add-umami-tracking.sh
- scripts/add-tracking-python.py
- SESSION_SUMMARY_ANALYTICS_RESEARCH_INQUIRY.md
Files Modified (29):
- src/routes/index.js (research routes)
- deployment-quickstart/docker-compose.yml (umami services)
- deployment-quickstart/.env.example (umami config)
- 26 public HTML pages (tracking script)
Values Alignment:
✅ Privacy-First Design (cookie-free, DNT honored, opt-out available)
✅ Human Agency (research inquiries require human review)
✅ Data Sovereignty (self-hosted analytics, no third-party sharing)
✅ GDPR Compliance (no personal data in analytics)
✅ Transparency (open-source tools, documented setup)
Testing Status:
✅ Research inquiry: Locally tested, data verified in MongoDB
⏳ Umami analytics: Pending production deployment
Next Steps:
1. Deploy to production (./scripts/deploy.sh)
2. Test research form on live site
3. Deploy Umami following UMAMI_SETUP_GUIDE.md
4. Update umami-tracker.js with website ID after setup
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements comprehensive system for tracking governance framework false negatives:
Backend:
- src/models/MissedBreach.model.js - Schema with severity, cost tracking, miss reasons
- src/controllers/missedBreach.controller.js - CRUD operations and statistics
- src/routes/missedBreach.routes.js - Admin-only API endpoints
- src/routes/index.js - Route integration at /api/admin/missed-breaches
Functionality:
- Report missed breaches with classification (NO_RULE_EXISTS, RULE_TOO_NARROW, etc.)
- Track actual/estimated costs of missed violations
- Calculate effectiveness rate: detected / (detected + missed)
- Breakdown by miss reason with examples
- Link to original audit logs where available
Statistics:
- Total missed breaches by severity
- Average time to detection
- Cost impact analysis
- Effectiveness comparison vs audit logs
Purpose:
- Measure true framework detection rate (not just blocked actions)
- Identify blind spots in governance rules
- Calculate realistic cost avoidance (avoiding "framework theater")
- Support research integrity claims with empirical data
Related: Cross-environment audit sync (production metrics)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements privacy-preserving synchronization of production audit logs
to development for comprehensive governance research analysis.
Backend Components:
- SyncMetadata.model.js: Track sync state and statistics
- audit-sanitizer.util.js: Privacy sanitization utility
- Redacts credentials, API keys, user identities
- Sanitizes file paths and violation content
- Preserves statistical patterns for research
- sync-prod-audit-logs.js: CLI sync script
- Incremental sync with deduplication
- Dry-run mode for testing
- Configurable date range
- AuditLog.model.js: Enhanced schema with environment tracking
- environment field (development/production/staging)
- sync_metadata tracking (original_id, synced_from, etc.)
- New indexes for cross-environment queries
- audit.controller.js: New /api/admin/audit-export endpoint
- Privacy-sanitized export for cross-environment sync
- Environment filter support in getAuditLogs
- MemoryProxy.service.js: Environment tagging in auditDecision()
- Tags new logs with NODE_ENV or override
- Sets is_local flag for tracking
Frontend Components:
- audit-analytics.html: Environment filter dropdown
- audit-analytics.js: Environment filter query parameter handling
Research Benefits:
- Combine dev and prod governance statistics
- Longitudinal analysis across environments
- Validate framework consistency
- Privacy-preserving data sharing
Security:
- API-based export (not direct DB access)
- Admin-only endpoints with JWT authentication
- Comprehensive credential redaction
- One-way sync (production → development)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Problem:
- Blog publishing has governance checks (inst_016/017/018/079)
- Media responses and templates had NO checks
- Inconsistent: same risks, different enforcement
Solution - Unified Framework Enforcement:
1. Created ContentGovernanceChecker.service.js (shared service)
2. Enforced in media responses (blocks at API level)
3. Enforced in response templates (scans on create)
4. Scanner for existing templates
Impact:
✅ Blog posts: Framework checks (existing)
✅ Media inquiry responses: Framework checks (NEW)
✅ Response templates: Framework checks (NEW)
✅ Future: Newsletter content ready for checks
Files Changed:
1. src/services/ContentGovernanceChecker.service.js (NEW)
- Unified content scanner for all external communications
- Checks: inst_016 (stats), inst_017 (guarantees), inst_018 (claims), inst_079 (dark patterns)
- Returns detailed violation reports with context
2. src/controllers/media.controller.js
- Added governance check in respondToInquiry()
- Blocks responses with violations (400 error)
- Logs violations with media outlet context
3. src/models/ResponseTemplate.model.js
- Added governance check in create()
- Stores check results in template record
- Prevents violating templates from being created
4. scripts/scan-response-templates.js (NEW)
- Scans all existing templates for violations
- Displays detailed violation reports
- --fix flag to mark violating templates as inactive
Testing:
✅ ContentGovernanceChecker: All pattern tests pass
✅ Clean content: Passes validation
✅ Fabricated stats: Detected (inst_016)
✅ Absolute guarantees: Detected (inst_017)
✅ Dark patterns: Detected (inst_079)
✅ Template scanner: Works (0 templates in DB)
Enforcement Points:
- Blog posts: publishPost() → blocked at API
- Media responses: respondToInquiry() → blocked at API
- Templates: create() → checked before insertion
- Newsletter: ready for future implementation
Architectural Consistency:
If blog needs governance, ALL external communications need governance.
References:
- inst_016: No fabricated statistics
- inst_017: No absolute guarantees
- inst_018: No unverified production claims
- inst_079: No dark patterns/manipulative urgency
- inst_063: External communications consistency
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed unused function parameters by prefixing with underscore
- Removed unused imports and variables
- Applied eslint --fix for automatic style fixes
- Property shorthand
- String template literals
- Prefer const over let where appropriate
- Spacing and formatting
Reduces lint errors from 108+ to 78 (61 unused vars, 17 other issues)
Related to CI lint failures in previous commit
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Line 49 has sessionId with unique: true (creates index automatically)
- Line 75 had redundant SessionSchema.index({ sessionId: 1 })
- Removed explicit index to eliminate Mongoose duplicate warning
- PageViewSchema had 'index: true' on sessionId field (line 16)
- AND compound index PageViewSchema.index({ sessionId: 1, timestamp: -1 })
- Compound index already covers sessionId queries (leftmost prefix)
- Removed redundant single-field index to eliminate Mongoose warning
- SessionSchema had both 'unique: true' and 'index: true'
- unique already creates an index, making index redundant
- Resolves Mongoose warning about duplicate schema index
- BlogPost uses native MongoDB (not Mongoose), causing MissingSchemaError
- Removed all .populate('blogPostId') calls that tried to reference non-existent Mongoose model
- Manually fetch blog post data in controllers when needed
- Updated getSubmissions, getSubmissionById, getSubmissionByBlogPost, exportSubmission
- Updated SubmissionTracking static methods: getByStatus, getByPublication
- Standalone submissions (like Le Monde) now display without errors
- Create Economist SubmissionTracking package correctly:
* mainArticle = full blog post content
* coverLetter = 216-word SIR— letter
* Links to blog post via blogPostId
- Archive 'Letter to The Economist' from blog posts (it's the cover letter)
- Fix date display on article cards (use published_at)
- Target publication already displaying via blue badge
Database changes:
- Make blogPostId optional in SubmissionTracking model
- Economist package ID: 68fa85ae49d4900e7f2ecd83
- Le Monde package ID: 68fa2abd2e6acd5691932150
Next: Enhanced modal with tabs, validation, export
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Extends SubmissionTracking model to support complete bilingual submission
packages with version control for multiple languages.
Schema additions:
- documents.coverLetter.versions[] - Language-versioned content
- documents.mainArticle.versions[] - With translation metadata
- documents.authorBio.versions[]
- documents.technicalBrief.versions[]
Helper methods:
- getDocument(docType, language, fallbackToDefault)
- setDocumentVersion(docType, language, content, metadata)
- getAvailableLanguages(docType)
- isPackageComplete(language)
- exportPackage(language)
Scripts:
- load-lemonde-package.js - Loads complete Le Monde submission package
Le Monde Package:
- Publication target: Rank 10, high-value French intellectual publication
- Theme: Post-Weberian organizational theory for AI age
- Content: Wittgenstein + Weber critique + indigenous data sovereignty
- Format: 187-word letter (within 150-200 requirement)
- Languages: English (original) + French (translated)
- Database ID: 68fa2abd2e6acd5691932150
- Status: Ready for submission
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add missing space after comma in SubmissionTracking model
- Replace string concatenation with template literal in blog controller
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
SUMMARY:
Completed migration from deprecated 'public: true/false' field to modern
'visibility' field across entire codebase. Ensures single source of truth
for document visibility state.
MIGRATION EXECUTION:
✓ Created migration script with dry-run support
✓ Migrated 120 documents in database (removed deprecated field)
✓ Post-migration: 0 documents with 'public' field, 127 with 'visibility'
✓ Zero data loss - all documents already had visibility set correctly
CODE CHANGES:
1. Database Migration (scripts/migrate-public-to-visibility.js):
- Created safe migration with dry-run mode
- Handles documents with both fields (cleanup)
- Post-migration verification built-in
- Execution: node scripts/migrate-public-to-visibility.js --execute
2. Document Model (src/models/Document.model.js):
- Removed 'public' field from create() method
- Updated findByQuadrant() to use visibility: 'public'
- Updated findByAudience() to use visibility: 'public'
- Updated search() to use visibility: 'public'
3. API Controller (src/controllers/documents.controller.js):
- Removed legacy filter: { public: true, visibility: { $exists: false } }
- listDocuments() now uses clean filter: visibility: 'public'
- searchDocuments() now uses clean filter: visibility: 'public'
4. Scripts Updated:
- upload-document.js: Removed public: true
- seed-architectural-safeguards-document.js: Removed public: true
- import-5-archives.js: Removed public: true
- verify-34-documents.js: Updated query filter to use visibility
- query-all-documents.js: Updated query filter to use visibility
VERIFICATION:
✓ 0 remaining 'public: true/false' usages in src/ and scripts/
✓ All documents use visibility field exclusively
✓ API queries now filter on visibility only
✓ Backward compatibility code removed
DATA MODEL:
Before: { public: true, visibility: 'public' } (redundant)
After: { visibility: 'public' } (single source of truth)
BENEFITS:
- Cleaner data model
- Single source of truth for visibility
- Simplified API logic
- Removed backward compatibility overhead
- Consistent with document security model
FRAMEWORK COMPLIANCE:
Addresses SCHEDULED_TASKS.md item "Legacy public Field Migration"
Completes Sprint 2 Medium Priority task
NEXT STEPS (Optional):
- Deploy migration to production
- Monitor for any edge cases
- Consider adding visibility to database indexes
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Update project dependencies, documentation, and supporting files:
- i18n improvements for multilingual support
- Admin dashboard enhancements
- Documentation updates for Koha/Stripe and deployment
- Server middleware and model updates
- Package dependency updates
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
SECURITY IMPROVEMENTS:
- Change default visibility from 'public' to 'internal' (prevents accidental exposure)
- Add visibility validation (public/internal/confidential/archived)
- Require valid category for public documents
- Add workflow_status tracking (draft/review/published)
PUBLISH WORKFLOW:
- New Document.publish(id, options) method with comprehensive validation
- New Document.unpublish(id, reason) method with audit trail
- New Document.listByWorkflowStatus(status) for workflow management
API ENDPOINTS (Admin only):
- POST /api/documents/:id/publish - Explicit publish with category validation
- POST /api/documents/:id/unpublish - Revert to internal with reason
- GET /api/documents/drafts - List unpublished documents
WORLD-CLASS UX:
- Clear validation messages with actionable guidance
- Lists available categories in error messages
- Tracks publish/unpublish history for audit trail
BACKWARD COMPATIBLE:
- Existing public documents unaffected
- Migration scripts automatically use safer defaults
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- GovernanceRule: Remove duplicate category index (uses compound index)
- VerificationLog: Remove duplicate verifiedAt index (uses compound + TTL)
- VariableValue: Remove duplicate category index (standalone index exists)
Eliminates 3 Mongoose duplicate index warnings on server startup
Server now starts with zero warnings
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
**Cache-Busting Improvements:**
- Switched from timestamp-based to semantic versioning (v1.0.2)
- Updated all HTML files: index.html, docs.html, leader.html
- CSS: tailwind.css?v=1.0.2
- JS: navbar.js, document-cards.js, docs-app.js v1.0.2
- Professional versioning approach for production stability
**systemd Service Implementation:**
- Created tractatus-dev.service for development environment
- Created tractatus-prod.service for production environment
- Added install-systemd.sh script for easy deployment
- Security hardening: NoNewPrivileges, PrivateTmp, ProtectSystem
- Resource limits: 1GB dev, 2GB prod memory limits
- Proper logging integration with journalctl
- Automatic restart on failure (RestartSec=10)
**Why systemd over pm2:**
1. Native Linux integration, no additional dependencies
2. Better OS-level security controls (ProtectSystem, ProtectHome)
3. Superior logging with journalctl integration
4. Standard across Linux distributions
5. More robust process management for production
**Usage:**
# Development:
sudo ./scripts/install-systemd.sh dev
# Production:
sudo ./scripts/install-systemd.sh prod
# View logs:
sudo journalctl -u tractatus -f
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Multi-Currency Implementation:
- Add currency configuration with 10 supported currencies (NZD, USD, EUR, GBP, AUD, CAD, JPY, CHF, SGD, HKD)
- Create client-side and server-side currency utilities for conversion and formatting
- Implement currency selector UI component with auto-detection and localStorage persistence
- Update Donation model to store multi-currency transactions with NZD equivalents
- Update Koha service to handle currency conversion and exchange rate tracking
- Update donation form UI to display prices in selected currency
- Update transparency dashboard to show donations with currency indicators
- Update Stripe setup documentation with currency_options configuration guide
Privacy Policy:
- Create comprehensive privacy policy page (GDPR compliant)
- Add shared footer component with privacy policy link
- Update all Koha pages with footer component
Technical Details:
- Exchange rates stored at donation time for historical accuracy
- All donations tracked in both original currency and NZD for transparency
- Base currency: NZD (New Zealand Dollar)
- Uses Stripe currency_options for monthly subscriptions
- Dynamic currency for one-time donations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed download icon size (1.25rem instead of huge black icons)
- Uploaded all 12 PDFs to production server
- Restored table of contents rendering for all documents
- Fixed modal cards with proper CSS and event handlers
- Replaced all docs-viewer.html links with docs.html
- Added nginx redirect from /docs/* to /docs.html
- Fixed duplicate headers in modal sections
- Improved cache-busting with timestamp versioning
All documentation features now working correctly:
✅ Card-based document viewer with modals
✅ PDF downloads with proper icons
✅ Table of contents navigation
✅ Consistent URL structure
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Models Created (7/10):
- Document.model.js: Framework docs with quadrant classification
- BlogPost.model.js: AI-curated blog with moderation
- MediaInquiry.model.js: Press/media triage workflow
- ModerationQueue.model.js: Human oversight queue with priority
- User.model.js: Admin authentication with bcrypt
- CaseSubmission.model.js: Community case studies with AI review
- Resource.model.js: Curated directory with alignment scores
Features:
- Full CRUD operations for each model
- Tractatus quadrant integration
- AI analysis fields for curation
- Human approval workflows
- Status tracking and filtering
- Security (password hashing, sanitized returns)
Deferred (Phase 2-3):
- Citation.model.js
- Translation.model.js
- KohaDonation.model.js
Status: Core models complete, ready for Express server
