tractatus/docs/DOCUMENT_REORGANIZATION_SUMMARY.md
TheFlow 2fc6e0a593 feat: implement documentation reorganization with archives
Documentation Reorganization (Option A - Full):
- Reduced public docs from 47 to 11 (76% reduction)
- 31 documents archived (project tracking, outdated)
- 5 documents marked confidential (security, payments)
- Clear 3-tier structure: Getting Started, Framework Details, Case Studies

Archives Infrastructure:
- Added visibility: 'archived' | 'public' | 'confidential' | 'internal'
- Added category: 'conceptual' | 'practical' | 'reference' | 'archived' | 'project-tracking'
- Added order field for explicit document ordering (1-11 for public)
- Added archiveNote field for explaining why documents were archived
- New endpoint: GET /api/documents/archived
- New controller method: listArchivedDocuments()
- UI: Archives section (collapsed by default) at bottom of docs list

Public Documentation (11 documents, well-organized):
  1. Architectural Overview (reference)
  2. Core Concepts (conceptual) - needs Phase 5 update
  3. Implementation Guide (practical) - needs MongoDB rewrite
  4. Core Values & Principles (conceptual)
  5. Case Studies (practical)
  6. Business Case Template (practical)
  7. Glossary (reference) - needs Phase 5 terms
  8-11. Recent Case Studies (practical)

Model Updates:
- src/models/Document.model.js: Added visibility, category, order, archiveNote fields
- src/models/Document.model.js: Added listArchived() static method
- Default sort by order (1-999) instead of date

Controller Updates:
- src/controllers/documents.controller.js: Added listArchivedDocuments()
- Filter excludes archived docs from main list by default

Route Updates:
- src/routes/documents.routes.js: Added GET /api/documents/archived

UI Updates:
- public/js/docs-app.js: New category structure (Getting Started, Framework Details, Reference)
- public/js/docs-app.js: Fetches and displays archived documents in collapsed section
- public/js/docs-app.js: Archives show document count badge
- public/js/docs-app.js: Archive notes displayed below archived document links
- Auto-loads Architectural Overview (order: 1) on page load

Scripts Created:
- scripts/archive-outdated-documents.js: Archive 10 outdated documents
- scripts/update-document-metadata.js: Set order/category for 7 core docs
- scripts/archive-all-internal-documents.js: Mass archive 23 internal docs

Documentation:
- docs/DOCUMENT_AUDIT_2025-10-11.md: Comprehensive audit of all 47 documents
- docs/DOCUMENT_REORGANIZATION_SUMMARY.md: Executive summary with before/after

Next Steps (Phase 2 - Content Updates):
- Update Core Concepts for Phase 5 MongoDB architecture
- Rewrite Implementation Guide for MongoDB deployment
- Update Glossary with Phase 5 terms (MongoDB, MemoryProxy, API Memory)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-11 01:26:14 +13:00

6.7 KiB

Tractatus Documentation Reorganization Summary

Date: 2025-10-11


Why This Reorganization?

Answer to your question: I needed to install mongoose and @anthropic-ai/sdk because I deployed TOO MUCH at once. The commit included:

  • The .md file ( needed)
  • Framework service enhancements ( shouldn't have deployed)
  • MongoDB models requiring mongoose ( side effect)
  • Services requiring @anthropic-ai/sdk ( side effect)

Lesson: Documentation deployment should NOT require framework dependencies. This indicates poor separation of concerns in the deployment process.


Current State: 17 Documents → Too Many, Poorly Organized

PUBLIC DOCUMENTATION (17 documents, no categorization)
├── ❌ No clear "start here" path
├── ❌ Outdated docs contradict current implementation
├── ❌ Project tracking mixed with end-user docs
├── ❌ Research proposals confuse visitors
└── ❌ New visitors overwhelmed

Proposed State: 7 + 10 Archives → Clear, Current, World-Class UX

PUBLIC DOCUMENTATION (7 documents, clear structure)
│
├── 📘 GETTING STARTED (order: 1-3)
│   ├── 1. Architectural Overview ✅ CURRENT
│   ├── 2. Core Concepts ⚠️ UPDATE
│   └── 3. Implementation Guide ⚠️ UPDATE
│
├── 📗 FRAMEWORK DETAILS (order: 4-6)
│   ├── 4. Core Values & Principles ✅ CURRENT
│   ├── 5. Case Studies ✅ CURRENT
│   └── 6. Business Case Template ✅ CURRENT
│
└── 📕 REFERENCE (order: 7)
    └── 7. Glossary of Terms ⚠️ UPDATE

📦 ARCHIVES (10 documents, collapsed section at bottom)
│
├── Historical Architecture Proposals (4)
│   ├── Introduction to Tractatus (superseded)
│   ├── LLM Architecture Spec (pre-Phase 5)
│   ├── Executive Brief (pre-Phase 5)
│   └── Organizational Theory (academic)
│
├── Development Tools (1)
│   └── Framework Enforcement for Claude Code
│
├── Project Tracking - Phase 5 (2)
│   ├── Session 1 Summary
│   └── Session 2 Summary
│
└── Research Proposals (3)
    ├── Feasibility Research Scope
    ├── Concurrent Session Architecture
    └── Rule Proliferation

Document Actions Summary

Action Count Documents
KEEP AS-IS 4 Architectural Overview, Core Values, Case Studies, Business Template
⚠️ UPDATE 3 Core Concepts, Implementation Guide, Glossary
📦 ARCHIVE 10 See Archives section above
Total 17 → 7 public + 10 archived

What Needs Updating?

1. Core Concepts (CRITICAL)

Problem: References 5 services, now 6. No MongoDB mention. Fix:

  • Add MongoDB persistence layer
  • Add 6th service (BlogCuration)
  • Add API Memory integration
  • Reference architectural overview for details

2. Implementation Guide (CRITICAL)

Problem: Shows filesystem-only setup, pre-MongoDB. Fix:

  • MongoDB setup instructions
  • Environment variables (MONGODB_URI, MONGODB_DB)
  • Remove .claude/instruction-history.json references
  • Add production deployment section
  • Update code examples

3. Glossary (MEDIUM)

Problem: Missing Phase 5 terms. Fix:

  • Add: MongoDB, MemoryProxy, API Memory, Hybrid Architecture
  • Update: Persistence mechanism definitions

UX Improvements

Before (Current)

Visitor lands on docs page:
"17 documents... which do I read first?"
"Some talk about filesystem, some about MongoDB... which is current?"
"Why are there session summaries here?"
"Is this a research proposal or actual documentation?"
→ Confusion, high bounce rate

After (Proposed)

Visitor lands on docs page:
📘 GETTING STARTED
   → Clear entry point: Architectural Overview
   → Only 3 documents to get started

📗 FRAMEWORK DETAILS
   → 3 focused documents for deeper understanding

📕 REFERENCE
   → Glossary for quick lookups

📦 ARCHIVES (collapsed)
   → Historical content available but not prominent

→ Clear path, low friction, professional UX

Implementation Checklist

Phase 1: Archives Infrastructure ⏱️ ~1 hour

  • Add visibility enum: public, archived
  • Add category enum: conceptual, practical, reference, archived, project-tracking, research-proposal
  • Update docs UI to show archives section (collapsed by default)
  • Test archives section rendering

Phase 2: Update Current Docs ⏱️ ~2-3 hours

  • Update Core Concepts (MongoDB, 6 services, API Memory)
  • Rewrite Implementation Guide (MongoDB setup, env vars, deployment)
  • Update Glossary (add Phase 5 terms)
  • Test all updated docs for accuracy

Phase 3: Archive Outdated Docs ⏱️ ~30 minutes

  • Set visibility: archived for 10 documents
  • Add archiveNote field explaining why archived
  • Set proper categories for all archived docs
  • Test archived documents still accessible via direct link

Phase 4: Metadata & Ordering ⏱️ ~30 minutes

  • Set category for all 7 public documents
  • Set audience appropriately
  • Set order 1-7 for public documents
  • Set order: 999 for archived documents

Phase 5: Testing & Validation ⏱️ ~30 minutes

  • Verify docs page shows 7 main documents only
  • Verify archives section collapsed by default
  • Verify all PDFs download correctly
  • Verify mobile responsiveness
  • Get user feedback

Total Estimated Effort: 4-6 hours


Success Metrics

Metric Before After Goal
Public docs count 17 7 60% reduction
Outdated docs visible 10 0 100% reduction
Clear categorization 0/17 7/7 100% improvement
"Start here" clarity None Clear Obvious entry point
Updated for Phase 5 1/17 7/7 100% current

Decision Required

Option A: Full Reorganization (Recommended)

  • Implement all 5 phases
  • 4-6 hours effort
  • Result: World-class UX

Option B: Minimal Update

  • Just update Core Concepts, Implementation Guide, Glossary
  • 2-3 hours effort
  • Result: Better accuracy, but still cluttered

Option C: Archive Only

  • Just hide outdated docs
  • 1 hour effort
  • Result: Less clutter, but remaining docs still inaccurate

Recommendation: Option A - Full reorganization for world-class UX


Next Steps

  1. Review this summary and audit document
  2. Approve reorganization plan (or request modifications)
  3. Schedule implementation (4-6 hour window)
  4. Execute phases 1-5
  5. Test and validate
  6. Monitor user feedback

Questions?

  • Which documents should stay/go?
  • Different categorization scheme?
  • Timing preferences?
  • Other concerns?