Commit graph

14 commits

Author SHA1 Message Date
TheFlow
d98eff8c32 fix: Optimize docs page API (7MB→19KB), fix categories, add MI translations
- Add summary projection to Document.list() excluding heavy content fields
- Fix 23 documents with invalid categories (framework/governance/reference)
- Archive 9 duplicate documents (kept canonical short-slug versions)
- Add Te Reo Māori UI translations to docs-app.js and document-cards.js
- Refactor language checks to be extensible (no more hardcoded EN/DE/FR)
- Remove unused font preloads from docs.html (fixes browser warning)
- Add resources to valid publish categories

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:14:02 +13:00
TheFlow
afcfc27502 feat: Complete Phase 2 Agent Lightning website integration
- Added Agent Lightning research section to researcher.html with Demo 2 results
- Created comprehensive /integrations/agent-lightning.html page
- Added Agent Lightning link in homepage hero section
- Updated Discord invite links (Tractatus + semantipy) across all pages
- Added feedback.js script to all key pages for live demonstration

Phase 2 of Master Plan complete: Discord setup → Website completion

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 14:38:20 +13:00
TheFlow
950b55f754 fix: add fallback for separate language documents in API
When requesting a translation via ?lang=de or ?lang=fr, the API now:
1. First checks for embedded translations (document.translations.de/fr)
2. Falls back to checking for separate documents with -de/-fr suffix

This allows the glossary translations (glossary-de, glossary-fr) to work
with the standard /api/documents/glossary?lang=de endpoint.

Fixes the 404 error when switching languages on /docs.html page.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-01 10:03:58 +13:00
TheFlow
1531a47cc0 fix(i18n): disable card view for translations to show translated content
Problem:
- Card view uses sections array which contains English text
- Translated documents showed English content in cards
- Only document title was translated

Solution:
- Set sections = undefined for translated documents
- Forces frontend to use traditional full-document view
- Traditional view displays content_html which IS translated

Result:
- Translated documents now show fully translated content
- Card view disabled for translations (traditional view instead)
- All content (title + body) now displays in German/French

Testing:
- German: "Einführung in den Tractatus-Rahmen", "Was ist Tractatus?"
- content_html confirmed 17KB of translated German text

🌐 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 01:56:00 +13:00
TheFlow
65a859ed00 fix(i18n): workaround for mangled markdown in translations
Problem:
- DeepL API with tag_handling='html' mangled markdown structure
- Translated markdown lost H2 headers and line breaks
- Sections couldn't be extracted from translated content
- Frontend showed no cards for translated documents

Root Cause:
- DeepL's HTML tag handling treated markdown as HTML
- Result: HTML entities (&gt;), no line breaks, corrupted structure

Workaround Solution:
- Use English document sections (preserved structure)
- Display translated document title
- Card titles in English, but card content uses translated HTML
- This allows cards to render correctly while preserving UX

Files Changed:
- src/utils/sections.util.js: Section extraction utilities (created)
- src/controllers/documents.controller.js: Return English sections for translations

Limitations:
- Card section titles remain in English
- Full translated content still displays correctly
- TODO: Re-translate with proper markdown preservation

🌐 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 01:48:28 +13:00
TheFlow
8c22811110 feat(docs): intelligent section recategorization + i18n infrastructure
This commit includes two major improvements to the documentation system:

## 1. Section Recategorization (UX Fix)

**Problem**: 64 sections (24%) were incorrectly marked as "critical" and
displayed at the bottom of documents, burying important foundational content.

**Solution**:
- Created intelligent recategorization script analyzing titles, excerpts,
  and document context
- Reduced "critical" from 64 → 2 sections (97% reduction)
- Properly categorized content by purpose:
  - Conceptual: 63 → 138 (+119%) - foundations, "why this matters"
  - Practical: 3 → 46 (+1433%) - how-to guides, examples
  - Technical: 111 → 50 (-55%) - true implementation details

**UI Improvements**:
- Reordered category display: Critical → Conceptual → Practical → Technical → Reference
- Changed Critical color from amber to red for better visual distinction
- All 22 documents recategorized (173 sections updated)

## 2. i18n Infrastructure (Phase 2)

**Backend**:
- DeepL API integration service with quota management and error handling
- Translation API routes (GET /api/documents/:slug?lang=de, POST /api/documents/:id/translate)
- Document model already supports translations field (no schema changes)

**Frontend**:
- docs-app.js enhanced with language detection and URL parameter support
- Automatic fallback to English when translation unavailable
- Integration with existing i18n-simple.js system

**Scripts**:
- translate-all-documents.js: Batch translation workflow (dry-run support)
- audit-section-categories.js: Category distribution analysis

**URL Strategy**: Query parameter approach (?lang=de, ?lang=fr)

**Status**: Backend complete, ready for DeepL API key configuration

**Files Modified**:
- Frontend: document-cards.js, docs-app.js
- Backend: documents.controller.js, documents.routes.js, DeepL.service.js
- Scripts: 3 new governance/i18n scripts

**Database**: 173 sections recategorized via script (already applied)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 00:48:27 +13:00
TheFlow
ac2db33732 fix(submissions): restructure Economist package and fix article display
- Create Economist SubmissionTracking package correctly:
  * mainArticle = full blog post content
  * coverLetter = 216-word SIR— letter
  * Links to blog post via blogPostId
- Archive 'Letter to The Economist' from blog posts (it's the cover letter)
- Fix date display on article cards (use published_at)
- Target publication already displaying via blue badge

Database changes:
- Make blogPostId optional in SubmissionTracking model
- Economist package ID: 68fa85ae49d4900e7f2ecd83
- Le Monde package ID: 68fa2abd2e6acd5691932150

Next: Enhanced modal with tabs, validation, export

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 08:47:42 +13:00
TheFlow
60f239c0bf refactor: deep cleanup - remove all website code from framework repo
REMOVED: 77 website-specific files from src/ and public/

Website Models (9):
- Blog, CaseSubmission, Document, Donation, MediaInquiry,
  ModerationQueue, NewsletterSubscription, Resource, User

Website Services (6):
- BlogCuration, MediaTriage, Koha, ClaudeAPI, ClaudeMdAnalyzer,
  AdaptiveCommunicationOrchestrator

Website Controllers (9):
- blog, cases, documents, koha, media, newsletter, auth, admin, variables

Website Routes (10):
- blog, cases, documents, koha, media, newsletter, auth, admin, test, demo

Website Middleware (4):
- auth, csrf-protection, file-security, response-sanitization

Website Utils (3):
- document-section-parser, jwt, markdown

Website JS (36):
- Website components, docs viewers, page features, i18n, Koha

RETAINED Framework Code:
- 6 core services (Boundary, ContextPressure, CrossReference,
  InstructionPersistence, Metacognitive, PluralisticDeliberation)
- 4 support services (AnthropicMemoryClient, MemoryProxy,
  RuleOptimizer, VariableSubstitution)
- 9 framework models (governance, audit, deliberation, project state)
- 3 framework controllers (rules, projects, audit)
- 7 framework routes (rules, governance, projects, audit, hooks, sync)
- 6 framework middleware (error, validation, security, governance)
- Minimal admin UI (rule manager, dashboard, hooks dashboard)
- Framework demos and documentation

PURPOSE: Tractatus-framework repo is now PURELY framework code.
All website/project code remains in internal repo only.

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-21 21:22:40 +13:00
TheFlow
13b6910198 refactor(data): migrate legacy public field to modern visibility field
SUMMARY:
Completed migration from deprecated 'public: true/false' field to modern
'visibility' field across entire codebase. Ensures single source of truth
for document visibility state.

MIGRATION EXECUTION:
✓ Created migration script with dry-run support
✓ Migrated 120 documents in database (removed deprecated field)
✓ Post-migration: 0 documents with 'public' field, 127 with 'visibility'
✓ Zero data loss - all documents already had visibility set correctly

CODE CHANGES:

1. Database Migration (scripts/migrate-public-to-visibility.js):
   - Created safe migration with dry-run mode
   - Handles documents with both fields (cleanup)
   - Post-migration verification built-in
   - Execution: node scripts/migrate-public-to-visibility.js --execute

2. Document Model (src/models/Document.model.js):
   - Removed 'public' field from create() method
   - Updated findByQuadrant() to use visibility: 'public'
   - Updated findByAudience() to use visibility: 'public'
   - Updated search() to use visibility: 'public'

3. API Controller (src/controllers/documents.controller.js):
   - Removed legacy filter: { public: true, visibility: { $exists: false } }
   - listDocuments() now uses clean filter: visibility: 'public'
   - searchDocuments() now uses clean filter: visibility: 'public'

4. Scripts Updated:
   - upload-document.js: Removed public: true
   - seed-architectural-safeguards-document.js: Removed public: true
   - import-5-archives.js: Removed public: true
   - verify-34-documents.js: Updated query filter to use visibility
   - query-all-documents.js: Updated query filter to use visibility

VERIFICATION:
✓ 0 remaining 'public: true/false' usages in src/ and scripts/
✓ All documents use visibility field exclusively
✓ API queries now filter on visibility only
✓ Backward compatibility code removed

DATA MODEL:
Before: { public: true, visibility: 'public' } (redundant)
After:  { visibility: 'public' } (single source of truth)

BENEFITS:
- Cleaner data model
- Single source of truth for visibility
- Simplified API logic
- Removed backward compatibility overhead
- Consistent with document security model

FRAMEWORK COMPLIANCE:
Addresses SCHEDULED_TASKS.md item "Legacy public Field Migration"
Completes Sprint 2 Medium Priority task

NEXT STEPS (Optional):
- Deploy migration to production
- Monitor for any edge cases
- Consider adding visibility to database indexes

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-19 13:49:21 +13:00
TheFlow
b076b801c4 feat(security): implement document publish workflow with safe defaults
SECURITY IMPROVEMENTS:
- Change default visibility from 'public' to 'internal' (prevents accidental exposure)
- Add visibility validation (public/internal/confidential/archived)
- Require valid category for public documents
- Add workflow_status tracking (draft/review/published)

PUBLISH WORKFLOW:
- New Document.publish(id, options) method with comprehensive validation
- New Document.unpublish(id, reason) method with audit trail
- New Document.listByWorkflowStatus(status) for workflow management

API ENDPOINTS (Admin only):
- POST /api/documents/:id/publish - Explicit publish with category validation
- POST /api/documents/:id/unpublish - Revert to internal with reason
- GET /api/documents/drafts - List unpublished documents

WORLD-CLASS UX:
- Clear validation messages with actionable guidance
- Lists available categories in error messages
- Tracks publish/unpublish history for audit trail

BACKWARD COMPATIBLE:
- Existing public documents unaffected
- Migration scripts automatically use safer defaults

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-19 12:34:51 +13:00
TheFlow
5034078d43 feat: implement Priority 3 - Enhanced search with faceted filtering
Add comprehensive search functionality to docs.html with:
- Faceted filters (quadrant, persistence, audience)
- Real-time search with 300ms debounce
- Search history with localStorage (last 10 searches)
- Keyboard navigation (Ctrl+K, arrows, Enter, Esc)
- Search tips modal with usage guide
- Result highlighting with query term emphasis
- Performance optimized (<500ms response time)

Backend enhancements:
- Enhanced /api/documents/search endpoint with filter support
- Combined text search + metadata filtering
- Returns pagination and filter state

Frontend additions:
- Search UI in docs.html (search bar, 3 filter dropdowns)
- docs-search-enhanced.js module with all functionality
- Search results panel with document cards
- Search tips modal with keyboard shortcuts

CSP Compliance:
- No inline event handlers or scripts
- All event listeners attached via external JS
- Pre-action check validated all files

Reference: docs/FEATURE_RICH_UI_IMPLEMENTATION_PLAN.md lines 123-156
Priority: 3 of 10 (8-10 hour estimated, completed)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-11 18:06:15 +13:00
TheFlow
6ceac8f449 feat: implement documentation reorganization with archives
Documentation Reorganization (Option A - Full):
- Reduced public docs from 47 to 11 (76% reduction)
- 31 documents archived (project tracking, outdated)
- 5 documents marked confidential (security, payments)
- Clear 3-tier structure: Getting Started, Framework Details, Case Studies

Archives Infrastructure:
- Added visibility: 'archived' | 'public' | 'confidential' | 'internal'
- Added category: 'conceptual' | 'practical' | 'reference' | 'archived' | 'project-tracking'
- Added order field for explicit document ordering (1-11 for public)
- Added archiveNote field for explaining why documents were archived
- New endpoint: GET /api/documents/archived
- New controller method: listArchivedDocuments()
- UI: Archives section (collapsed by default) at bottom of docs list

Public Documentation (11 documents, well-organized):
  1. Architectural Overview (reference)
  2. Core Concepts (conceptual) - needs Phase 5 update
  3. Implementation Guide (practical) - needs MongoDB rewrite
  4. Core Values & Principles (conceptual)
  5. Case Studies (practical)
  6. Business Case Template (practical)
  7. Glossary (reference) - needs Phase 5 terms
  8-11. Recent Case Studies (practical)

Model Updates:
- src/models/Document.model.js: Added visibility, category, order, archiveNote fields
- src/models/Document.model.js: Added listArchived() static method
- Default sort by order (1-999) instead of date

Controller Updates:
- src/controllers/documents.controller.js: Added listArchivedDocuments()
- Filter excludes archived docs from main list by default

Route Updates:
- src/routes/documents.routes.js: Added GET /api/documents/archived

UI Updates:
- public/js/docs-app.js: New category structure (Getting Started, Framework Details, Reference)
- public/js/docs-app.js: Fetches and displays archived documents in collapsed section
- public/js/docs-app.js: Archives show document count badge
- public/js/docs-app.js: Archive notes displayed below archived document links
- Auto-loads Architectural Overview (order: 1) on page load

Scripts Created:
- scripts/archive-outdated-documents.js: Archive 10 outdated documents
- scripts/update-document-metadata.js: Set order/category for 7 core docs
- scripts/archive-all-internal-documents.js: Mass archive 23 internal docs

Documentation:
- docs/DOCUMENT_AUDIT_2025-10-11.md: Comprehensive audit of all 47 documents
- docs/DOCUMENT_REORGANIZATION_SUMMARY.md: Executive summary with before/after

Next Steps (Phase 2 - Content Updates):
- Update Core Concepts for Phase 5 MongoDB architecture
- Rewrite Implementation Guide for MongoDB deployment
- Update Glossary with Phase 5 terms (MongoDB, MemoryProxy, API Memory)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-11 01:26:14 +13:00
TheFlow
426fde1ac5 feat(infra): semantic versioning and systemd service implementation
**Cache-Busting Improvements:**
- Switched from timestamp-based to semantic versioning (v1.0.2)
- Updated all HTML files: index.html, docs.html, leader.html
- CSS: tailwind.css?v=1.0.2
- JS: navbar.js, document-cards.js, docs-app.js v1.0.2
- Professional versioning approach for production stability

**systemd Service Implementation:**
- Created tractatus-dev.service for development environment
- Created tractatus-prod.service for production environment
- Added install-systemd.sh script for easy deployment
- Security hardening: NoNewPrivileges, PrivateTmp, ProtectSystem
- Resource limits: 1GB dev, 2GB prod memory limits
- Proper logging integration with journalctl
- Automatic restart on failure (RestartSec=10)

**Why systemd over pm2:**
1. Native Linux integration, no additional dependencies
2. Better OS-level security controls (ProtectSystem, ProtectHome)
3. Superior logging with journalctl integration
4. Standard across Linux distributions
5. More robust process management for production

**Usage:**
  # Development:
  sudo ./scripts/install-systemd.sh dev

  # Production:
  sudo ./scripts/install-systemd.sh prod

  # View logs:
  sudo journalctl -u tractatus -f

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-09 09:16:22 +13:00
TheFlow
0d75492c60 feat: add API routes, controllers, and migration tools
Implemented complete backend API foundation with authentication, document
management, blog operations, and admin functionality. Added migration tools
for database seeding and document import.

**Controllers (4 files):**
- auth.controller.js: User authentication (login, getCurrentUser, logout)
- documents.controller.js: Document CRUD operations
- blog.controller.js: Blog post management with admin/public access
- admin.controller.js: Admin dashboard (stats, moderation queue, activity)

**Routes (5 files):**
- auth.routes.js: Authentication endpoints
- documents.routes.js: Document API endpoints
- blog.routes.js: Blog API endpoints
- admin.routes.js: Admin API endpoints
- index.js: Central routing configuration with API documentation

**Migration Tools (2 scripts):**
- seed-admin.js: Create admin user for system access
- migrate-documents.js: Import markdown documents with metadata extraction,
  slug generation, and dry-run support. Successfully migrated 8 documents
  from anthropic-submission directory.

**Server Updates:**
- Integrated all API routes under /api namespace
- Updated homepage to reflect completed API implementation
- Maintained security middleware (Helmet, CORS, rate limiting)

**Testing:**
 Server starts successfully on port 9000
 Authentication flow working (login, token validation)
 Document endpoints tested (list, get by slug)
 Admin stats endpoint verified (requires authentication)
 Migration completed: 8 documents imported

**Database Status:**
- Documents collection: 8 technical papers
- Users collection: 1 admin user
- All indexes operational

This completes the core backend API infrastructure. Next steps: build
Tractatus governance services (InstructionClassifier, CrossReferenceValidator,
BoundaryEnforcer).

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-07 00:36:40 +13:00