From 8ecd770fce675b625dedae5837a06e8f503131cf Mon Sep 17 00:00:00 2001 From: TheFlow Date: Mon, 27 Oct 2025 12:11:16 +1300 Subject: [PATCH] feat(research): add cross-environment audit log sync infrastructure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements privacy-preserving synchronization of production audit logs to development for comprehensive governance research analysis. Backend Components: - SyncMetadata.model.js: Track sync state and statistics - audit-sanitizer.util.js: Privacy sanitization utility - Redacts credentials, API keys, user identities - Sanitizes file paths and violation content - Preserves statistical patterns for research - sync-prod-audit-logs.js: CLI sync script - Incremental sync with deduplication - Dry-run mode for testing - Configurable date range - AuditLog.model.js: Enhanced schema with environment tracking - environment field (development/production/staging) - sync_metadata tracking (original_id, synced_from, etc.) - New indexes for cross-environment queries - audit.controller.js: New /api/admin/audit-export endpoint - Privacy-sanitized export for cross-environment sync - Environment filter support in getAuditLogs - MemoryProxy.service.js: Environment tagging in auditDecision() - Tags new logs with NODE_ENV or override - Sets is_local flag for tracking Frontend Components: - audit-analytics.html: Environment filter dropdown - audit-analytics.js: Environment filter query parameter handling Research Benefits: - Combine dev and prod governance statistics - Longitudinal analysis across environments - Validate framework consistency - Privacy-preserving data sharing Security: - API-based export (not direct DB access) - Admin-only endpoints with JWT authentication - Comprehensive credential redaction - One-way sync (production → development) šŸ¤– Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- public/admin/audit-analytics.html | 18 +- public/js/admin/audit-analytics.js | 21 +- scripts/hook-validators/validate-file-edit.js | 2 + scripts/sync-prod-audit-logs.js | 249 ++++++++++++++++++ src/controllers/audit.controller.js | 69 ++++- src/models/AuditLog.model.js | 47 ++++ src/models/SyncMetadata.model.js | 65 +++++ src/routes/audit.routes.js | 7 + src/services/MemoryProxy.service.js | 7 + src/utils/audit-sanitizer.util.js | 219 +++++++++++++++ 10 files changed, 694 insertions(+), 10 deletions(-) create mode 100755 scripts/sync-prod-audit-logs.js create mode 100644 src/models/SyncMetadata.model.js create mode 100644 src/utils/audit-sanitizer.util.js diff --git a/public/admin/audit-analytics.html b/public/admin/audit-analytics.html index 1452a06b..42ea251c 100644 --- a/public/admin/audit-analytics.html +++ b/public/admin/audit-analytics.html @@ -5,9 +5,9 @@ Audit Analytics | Tractatus Admin - - - + + +