- Create Economist SubmissionTracking package correctly: * mainArticle = full blog post content * coverLetter = 216-word SIR— letter * Links to blog post via blogPostId - Archive 'Letter to The Economist' from blog posts (it's the cover letter) - Fix date display on article cards (use published_at) - Target publication already displaying via blue badge Database changes: - Make blogPostId optional in SubmissionTracking model - Economist package ID: 68fa85ae49d4900e7f2ecd83 - Le Monde package ID: 68fa2abd2e6acd5691932150 Next: Enhanced modal with tabs, validation, export 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1455 lines
53 KiB
Markdown
1455 lines
53 KiB
Markdown
# Tractatus - Claude Code Maintenance Guide
|
|
|
|
**Comprehensive reference for Claude Code sessions on the Tractatus project**
|
|
|
|
**Last Updated**: 2025-10-14
|
|
**Version**: 2.1.1 (Added Modal Scrollbar Troubleshooting)
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
1. [Project Identity](#project-identity)
|
|
2. [Tractatus Framework Governance](#tractatus-framework-governance)
|
|
3. [Session Management](#session-management)
|
|
4. [Development Conventions](#development-conventions)
|
|
5. [Directory Structure](#directory-structure)
|
|
6. [Documentation Management](#documentation-management)
|
|
7. [Phase 1 Deliverables](#phase-1-deliverables)
|
|
8. [Human Approval Requirements](#human-approval-requirements)
|
|
9. [Te Tiriti & Indigenous Perspective](#te-tiriti--indigenous-perspective)
|
|
10. [Links & References](#links--references)
|
|
|
|
---
|
|
|
|
## Project Identity
|
|
|
|
### Core Information
|
|
- **Project Name**: Tractatus Website Platform
|
|
- **Domain**: agenticgovernance.digital
|
|
- **Repository**: GitHub (primary) + Codeberg/Gitea (mirrors)
|
|
- **Status**: Development - Phase 1 Implementation
|
|
- **Created**: 2025-10-06
|
|
- **Primary Developer**: Claude Code (Anthropic Sonnet 4.5)
|
|
- **Project Owner**: John Stroh
|
|
|
|
### ⚠️ Critical: Project Isolation
|
|
|
|
**THIS IS A SEPARATE PROJECT FROM family-history AND sydigital**
|
|
|
|
- **Separate MongoDB instance**: Port 27017, database `tractatus_dev`
|
|
- **Separate application port**: 9000
|
|
- **Separate Git repository**: Local + GitHub account
|
|
- **Separate systemd services**: mongodb-tractatus.service, tractatus.service
|
|
- **No shared code/data**: Patterns may be adapted, but no dependencies
|
|
|
|
**Sessions must maintain clear separation.** Always verify which project context you're in.
|
|
|
|
### Project Purpose
|
|
|
|
Build a world-class platform demonstrating the **Tractatus-Based LLM Safety Framework** through:
|
|
|
|
1. **Three Audience Paths**: Researcher, Implementer, Advocate
|
|
2. **AI-Powered Features**: Blog curation, media triage, case studies (all with human oversight)
|
|
3. **Interactive Demonstrations**: Classification, 27027 incident, boundary enforcement
|
|
4. **Dogfooding**: The website implements Tractatus to govern its own AI operations
|
|
5. **Values Alignment**: Sovereignty, Transparency, Harmlessness, Community
|
|
|
|
**Timeline**: 3-4 months for complete Phase 1 local prototype (no rush, no shortcuts, world-class quality)
|
|
|
|
### Technical Architecture
|
|
|
|
#### Infrastructure
|
|
- **MongoDB**: Port 27017, database `tractatus_dev`
|
|
- **Application**: Node.js/Express on port 9000
|
|
- **WebSocket**: Port 9001 (if needed)
|
|
- **Data Directory**: `/home/theflow/projects/tractatus/data/mongodb`
|
|
- **Logs**: `/home/theflow/projects/tractatus/logs/`
|
|
|
|
#### Technology Stack
|
|
- **Backend**: Node.js 18+, Express 4.x, MongoDB 7+
|
|
- **Frontend**: Vanilla JavaScript, Tailwind CSS (no framework dependency)
|
|
- **Authentication**: JWT for admin/moderation
|
|
- **AI Integration**: Claude API (Sonnet 4.5) - Phase 2+
|
|
- **File Storage**: GridFS for PDFs, documents
|
|
- **Testing**: Jest + Supertest
|
|
|
|
#### Database Collections
|
|
```javascript
|
|
tractatus_dev.documents // Technical papers, framework docs
|
|
tractatus_dev.blog_posts // AI-curated, human-approved
|
|
tractatus_dev.media_inquiries // Press/media with AI triage
|
|
tractatus_dev.case_submissions // Community case studies
|
|
tractatus_dev.resources // External links, aligned projects
|
|
tractatus_dev.moderation_queue // Human oversight queue
|
|
tractatus_dev.users // Admin accounts
|
|
tractatus_dev.citations // Academic citation tracking
|
|
tractatus_dev.translations // Multi-language content (future)
|
|
tractatus_dev.koha_donations // Phase 3
|
|
```
|
|
|
|
---
|
|
|
|
## Tractatus Framework Governance
|
|
|
|
### Core Services - Six Mandatory Components
|
|
|
|
**All six MUST be active throughout every session. Framework fade = critical failure.**
|
|
|
|
### What is a "27027 Failure"?
|
|
|
|
Named after the port number 27027 from an October 2025 incident:
|
|
|
|
**User instruction**: "Check port 27027" (explicit, non-standard MongoDB port)
|
|
**Claude's action**: Used port 27017 (standard default)
|
|
**Root cause**: Pattern recognition bias - training data's "MongoDB = 27017" association overrode explicit instruction **immediately**
|
|
|
|
**Key insight**: This is NOT about forgetting. Claude never truly "heard" the instruction because the learned pattern was so strong it **autocorrected** the explicit input, like a spell-checker changing a deliberately unusual word.
|
|
|
|
**Why this matters**: As AI capabilities increase, training data creates stronger patterns, making this problem WORSE, not better. The framework must architecturally prevent pattern recognition from overriding explicit human instructions.
|
|
|
|
---
|
|
|
|
#### 1. ContextPressureMonitor
|
|
**Purpose**: Session quality management through multi-factor pressure analysis
|
|
|
|
**When to invoke**:
|
|
- Session start (baseline: 0/200000 tokens)
|
|
- Every 25% tokens (50k, 100k, 150k)
|
|
- After complex multi-file operations
|
|
- After any error or unexpected behavior
|
|
- Every ~20 messages or 40k tokens
|
|
|
|
**Pressure Levels**:
|
|
|
|
| Level | Score | Action | What to Do |
|
|
|-------|-------|--------|------------|
|
|
| **NORMAL** | 0-30% | PROCEED | Continue normally |
|
|
| **ELEVATED** | 30-50% | INCREASE_VERIFICATION | More careful, verify outputs |
|
|
| **HIGH** | 50-70% | SUGGEST_CONTEXT_REFRESH | Consider session handoff |
|
|
| **CRITICAL** | 70-85% | MANDATORY_VERIFICATION | Verify all actions, prepare handoff |
|
|
| **DANGEROUS** | 85%+ | IMMEDIATE_HALT | Stop, create handoff, refresh context |
|
|
|
|
**Monitored Factors** (Weighted):
|
|
1. **Token Usage** (35% weight) - Context window pressure
|
|
2. **Conversation Length** (25% weight) - Attention decay over long sessions
|
|
3. **Task Complexity** (15% weight) - Number of simultaneous tasks, dependencies, file modifications
|
|
4. **Error Frequency** (15% weight) - Recent errors indicate degraded state
|
|
5. **Instruction Density** (10% weight) - Too many competing directives
|
|
|
|
**Command**:
|
|
```bash
|
|
node scripts/check-session-pressure.js --tokens <current>/<budget> --messages <count> --tasks <num> --errors <num>
|
|
```
|
|
|
|
**Files updated**:
|
|
- `.claude/session-state.json` (last_framework_activity.ContextPressureMonitor)
|
|
- `.claude/token-checkpoints.json` (completed checkpoints)
|
|
|
|
#### 2. InstructionPersistenceClassifier
|
|
**Purpose**: Make explicit instructions override learned patterns to prevent 27027 failures (pattern recognition bias)
|
|
|
|
**When to invoke**:
|
|
- User gives explicit instruction
|
|
- Configuration specifications
|
|
- Architectural constraints
|
|
- Port/database requirements
|
|
- Coding standards
|
|
- Project-specific rules
|
|
|
|
**Quadrant Mapping**:
|
|
|
|
| Function | Quadrant | Human Oversight | Example |
|
|
|----------|----------|-----------------|---------|
|
|
| Mission/values changes | STRATEGIC | Mandatory approval | "Always prioritize privacy" |
|
|
| Blog editorial guidelines | OPERATIONAL | Quarterly review | "All posts must cite sources" |
|
|
| Publish approved post | TACTICAL | Pre-approved | Execute after human approval |
|
|
| Technical config | SYSTEM | Technical review | MongoDB ports, API keys |
|
|
| AI suggests blog topics | STOCHASTIC | Always human approval | "Write about GDPR" |
|
|
|
|
**Classification process**:
|
|
1. Identify explicit instruction from user
|
|
2. Classify quadrant (STR/OPS/TAC/SYS/STO)
|
|
3. Determine persistence level (LOW/MEDIUM/HIGH/CRITICAL)
|
|
4. Assign temporal scope (SESSION/SPRINT/PROJECT/PERMANENT)
|
|
5. Set verification requirement (NONE/ADVISORY/MANDATORY)
|
|
6. Calculate explicitness score (0.0-1.0)
|
|
7. Store in `.claude/instruction-history.json`
|
|
|
|
**Files updated**:
|
|
- `.claude/instruction-history.json` (append new instruction)
|
|
- `.claude/session-state.json` (last_framework_activity.InstructionPersistenceClassifier)
|
|
|
|
#### 3. CrossReferenceValidator
|
|
**Purpose**: Validate proposed actions against instruction history to prevent conflicts
|
|
|
|
**When to invoke**:
|
|
- Before database schema changes
|
|
- Before configuration modifications
|
|
- Before architectural decisions
|
|
- Before changing established patterns
|
|
- Before modifying system-level settings
|
|
|
|
**Validation process**:
|
|
1. Load `.claude/instruction-history.json`
|
|
2. Identify relevant instructions (by quadrant, scope, keywords)
|
|
3. Check for conflicts with proposed action
|
|
4. If HIGH/CRITICAL persistence conflicts: BLOCK action
|
|
5. Report conflicts to user with instruction details
|
|
6. Require user override for conflicting actions
|
|
|
|
**Blocking criteria**:
|
|
- HIGH persistence instruction + direct conflict = BLOCK
|
|
- CRITICAL persistence instruction + any conflict = BLOCK
|
|
- MANDATORY verification required + no verification = BLOCK
|
|
|
|
**Files updated**:
|
|
- `.claude/session-state.json` (last_framework_activity.CrossReferenceValidator)
|
|
|
|
#### 4. BoundaryEnforcer
|
|
**Purpose**: Ensure AI never makes values decisions without human approval
|
|
|
|
**When to invoke**:
|
|
- Before privacy policy decisions
|
|
- Before ethical trade-off decisions
|
|
- Before user agency changes
|
|
- Before mission/values modifications
|
|
- Before governance document changes
|
|
|
|
**Values boundaries** (from Tractatus framework):
|
|
- Privacy vs. performance trade-offs
|
|
- User autonomy vs. safety
|
|
- Transparency vs. security
|
|
- Community vs. individual rights
|
|
- Indigenous sovereignty vs. technical expediency
|
|
|
|
**Enforcement actions**:
|
|
1. Detect values-sensitive decision domain
|
|
2. Report boundary violation to user
|
|
3. BLOCK automatic decision
|
|
4. Provide alternatives/research/analysis only
|
|
5. Require explicit human decision
|
|
6. Document decision in instruction history
|
|
|
|
**Files updated**:
|
|
- `.claude/session-state.json` (last_framework_activity.BoundaryEnforcer)
|
|
|
|
#### 5. MetacognitiveVerifier
|
|
**Purpose**: AI self-checks reasoning before proposing complex actions
|
|
|
|
**When to invoke** (selective use):
|
|
- Operations with >3 file modifications
|
|
- Operations with >5 sequential steps
|
|
- Architecture changes (structural modifications)
|
|
- Security implementations
|
|
- Complex refactoring
|
|
- Multi-system integrations
|
|
|
|
**Verification dimensions**:
|
|
1. **Alignment** (0.0-1.0): Does action align with project goals?
|
|
2. **Coherence** (0.0-1.0): Is reasoning internally consistent?
|
|
3. **Completeness** (0.0-1.0): Are edge cases considered?
|
|
4. **Safety** (0.0-1.0): What are the risks?
|
|
5. **Alternatives** (0.0-1.0): Have alternatives been explored?
|
|
|
|
**Overall Confidence**:
|
|
- 0.90-1.0: HIGH - Proceed
|
|
- 0.75-0.89: MEDIUM - Proceed with caution
|
|
- 0.60-0.74: LOW - Reconsider approach
|
|
- <0.60: VERY LOW - Stop and rethink
|
|
|
|
**Files updated**:
|
|
- `.claude/session-state.json` (last_framework_activity.MetacognitiveVerifier)
|
|
|
|
#### 6. PluralisticDeliberationOrchestrator
|
|
**Purpose**: Facilitate multi-stakeholder deliberation across plural moral values without imposing hierarchy
|
|
|
|
**When to invoke**:
|
|
- BoundaryEnforcer flags values decision (triggers deliberation)
|
|
- Stakeholder value conflict identified
|
|
- Privacy vs. safety trade-offs
|
|
- Individual rights vs. collective welfare tensions
|
|
- Cultural values conflicts (Western vs. Indigenous, secular vs. religious)
|
|
- Policy decisions affecting diverse communities
|
|
|
|
**Core Functions**:
|
|
|
|
**1. Values Conflict Detection**
|
|
```javascript
|
|
const conflict = await PluralisticDeliberationOrchestrator.analyzeConflict({
|
|
decision: "Disclose user data to prevent harm?",
|
|
context: { urgency, scale, affected_groups }
|
|
});
|
|
|
|
// Output:
|
|
{
|
|
moral_frameworks_in_tension: [
|
|
{ framework: "Rights-based (Deontological)", position: "...", stakeholders: [...] },
|
|
{ framework: "Consequentialist (Utilitarian)", position: "...", stakeholders: [...] },
|
|
{ framework: "Care Ethics", position: "...", stakeholders: [...] }
|
|
],
|
|
value_trade_offs: ["Privacy vs. Safety", "Individual rights vs. Collective welfare"],
|
|
affected_stakeholder_groups: ["users_with_data", "potential_victims", "platform_community"]
|
|
}
|
|
```
|
|
|
|
**2. Stakeholder Engagement**
|
|
- Identify representatives from each moral framework
|
|
- Ensure diverse perspectives (not just dominant groups)
|
|
- Include affected parties (not just experts)
|
|
- Use AdaptiveCommunicationOrchestrator for culturally appropriate outreach
|
|
|
|
**3. Deliberation Facilitation**
|
|
- Round 1: Each perspective states position
|
|
- Round 2: Identify shared values
|
|
- Round 3: Explore compromise/accommodation
|
|
- Round 4: Clarify irreconcilable differences
|
|
- Document process: NOT majority vote, BUT structured consensus-seeking with documented dissent
|
|
|
|
**4. Outcome Documentation**
|
|
```javascript
|
|
{
|
|
decision_made: "Disclose data in this case",
|
|
values_prioritized: ["harm_prevention", "collective_safety"],
|
|
values_deprioritized: ["individual_privacy", "data_autonomy"],
|
|
moral_remainder: "Privacy violation acknowledged as moral loss",
|
|
dissenting_perspectives: [
|
|
{
|
|
framework: "Rights-based",
|
|
objection: "Privacy violation sets dangerous precedent",
|
|
stakeholders: ["privacy_advocates"]
|
|
}
|
|
],
|
|
justification: "Given imminent threat to life, prioritized safety with safeguards",
|
|
precedent_applicability: "Applies to [specific context], not universal rule",
|
|
review_date: "2025-11-12"
|
|
}
|
|
```
|
|
|
|
**Integration with other components**:
|
|
- **BoundaryEnforcer** triggers PluralisticDeliberationOrchestrator when values conflict detected
|
|
- **CrossReferenceValidator** checks deliberation against precedent database
|
|
- **AdaptiveCommunicationOrchestrator** ensures stakeholder communications respect cultural norms
|
|
- **MetacognitiveVerifier** assesses AI's value conflict detection accuracy
|
|
- **InstructionPersistenceClassifier** stores deliberation outcomes as HIGH persistence instructions
|
|
|
|
**Tiered Response by Urgency**:
|
|
- **CRITICAL** (minutes): Automated triage + immediate review → full deliberation post-incident
|
|
- **URGENT** (hours/days): Rapid stakeholder consultation (expedited process)
|
|
- **IMPORTANT** (weeks): Full deliberative process
|
|
- **ROUTINE** (months): Precedent matching + lightweight review
|
|
|
|
**Enforcement Principles** (from value pluralism research):
|
|
- **Foundational Pluralism**: Moral frameworks are irreducibly different (no supervalue)
|
|
- **Incommensurability ≠ Incomparability**: Can compare values without common metric (practical wisdom, covering values)
|
|
- **Rational Regret**: Document what's lost, not just what's gained
|
|
- **Legitimate Disagreement**: Valid outcome when values genuinely incommensurable
|
|
- **Non-Hierarchical**: No automatic ranking (privacy > safety or safety > privacy)
|
|
- **Provisional Agreement**: Decisions are reviewable, not permanent rules
|
|
|
|
**Files updated**:
|
|
- `.claude/session-state.json` (last_framework_activity.PluralisticDeliberationOrchestrator)
|
|
- Precedent database (new deliberation outcomes)
|
|
- Stakeholder communications log
|
|
|
|
**Human Oversight**: MANDATORY
|
|
- AI facilitates, humans decide (TRA-OPS-0002)
|
|
- Stakeholder list requires human approval
|
|
- Deliberation outcomes require human approval
|
|
- Values decisions NEVER automated
|
|
|
|
---
|
|
|
|
### Key Framework Instructions
|
|
|
|
**Critical instructions from `.claude/instruction-history.json` for quick reference**
|
|
|
|
These instructions modify Claude's behavior to align with Tractatus principles. Full instruction database in `.claude/instruction-history.json`.
|
|
|
|
#### inst_028: Adaptive Communication Tone
|
|
**Purpose**: Prevent linguistic hierarchy in stakeholder deliberation
|
|
**Persistence**: HIGH | **Quadrant**: OPS | **Scope**: SESSION
|
|
|
|
**Instruction**:
|
|
Detect and mirror stakeholder communication style:
|
|
- Formal academic → respond formally with citations
|
|
- Casual/direct → respond conversationally, no jargon
|
|
- Technical → use precise terminology
|
|
- Plain language → avoid specialist terms
|
|
- Never impose corporate/academic tone by default
|
|
- Test: If you'd sound weird at a pub, you're too formal
|
|
|
|
**Application**:
|
|
When communicating with Australian/NZ stakeholders, use direct language. When communicating with academic researchers, use formal register. Same message, different styles - prevents exclusion through linguistic norms.
|
|
|
|
**Related**: PluralisticDeliberationOrchestrator, AdaptiveCommunicationOrchestrator
|
|
|
|
---
|
|
|
|
#### inst_029: Anti-Patronizing Language Filter
|
|
**Purpose**: Prevent condescension that reproduces power imbalances
|
|
**Persistence**: HIGH | **Quadrant**: STR | **Scope**: PERMANENT
|
|
|
|
**Instruction**:
|
|
Flag patronizing patterns before sending:
|
|
- "Simply...", "Just...", "Obviously..."
|
|
- "As you may know...", "It's easy to..."
|
|
- Explaining basics to experts
|
|
- Oversimplification when detail requested
|
|
- Block message until revised. Assume intelligence.
|
|
|
|
**Application**:
|
|
Before sending any communication to stakeholders, scan for patronizing language. If detected, BLOCK and revise. This is not politeness - it's preventing elite capture where dominant groups dismiss alternative perspectives as "confused."
|
|
|
|
**Related**: Deliberative democracy (Iris Marion Young's critique of formal equality)
|
|
|
|
---
|
|
|
|
#### inst_030: Regional Communication Norms
|
|
**Purpose**: Respect diverse cultural communication styles
|
|
**Persistence**: MEDIUM | **Quadrant**: TAC | **Scope**: SESSION
|
|
|
|
**Instruction**:
|
|
Adapt to regional communication norms:
|
|
|
|
**Australian/NZ context**:
|
|
- Value directness over diplomatic cushioning
|
|
- "Mate" appropriate in casual contexts
|
|
- Brevity respected
|
|
- Anti-tall-poppy (avoid excessive formality)
|
|
- Understatement valued ("not bad" = excellent)
|
|
|
|
**Japanese context**:
|
|
- Indirectness to preserve harmony (honne/tatemae)
|
|
- Formal register shows respect
|
|
- Silence is meaningful, not awkward
|
|
- Group consensus before stating position
|
|
|
|
**Te Reo Māori protocols**:
|
|
- Begin with mihi (greeting), acknowledge whakapapa
|
|
- Use communal framing (whānau, iwi, not just individual)
|
|
- Respect tapu/noa distinctions
|
|
- Seek consensus (kotahitanga)
|
|
|
|
**Detection**: .au/.nz/.jp domains, language, self-identification, slang
|
|
|
|
**Application**:
|
|
When facilitating pluralistic deliberation with diverse stakeholders, adapt communication structure (not just language translation). Australian stakeholder receives direct summary, Japanese stakeholder receives formal acknowledgment with indirect exploration of concerns, Māori stakeholder receives communal framing with mihi.
|
|
|
|
**Related**: Cross-cultural deliberation, high/low context communication theory
|
|
|
|
---
|
|
|
|
#### inst_031: Multilingual Engagement Protocol
|
|
**Purpose**: Enable deliberation across language barriers
|
|
**Persistence**: HIGH | **Quadrant**: OPS | **Scope**: PERMANENT
|
|
|
|
**Instruction**:
|
|
When non-English input detected:
|
|
1. Respond in sender's language if capable
|
|
2. If not: "Kia ora! I detected [language] but will respond in English. Translation resources: [link]"
|
|
3. Never assume English proficiency
|
|
4. Offer translation of key documents
|
|
5. Acknowledge language barriers respectfully
|
|
6. For multilingual deliberations:
|
|
- Provide simultaneous translation
|
|
- Allow extra time for comprehension
|
|
- Check understanding in both directions
|
|
|
|
**Application**:
|
|
If stakeholder submits input in Māori, respond in Māori if capable. If not, acknowledge the language respectfully and explain English response. For deliberations with multilingual participants, provide translation and verify understanding across languages - don't privilege English speakers.
|
|
|
|
**Related**: Linguistic justice, inclusive deliberation
|
|
|
|
---
|
|
|
|
## Session Management
|
|
|
|
### Session Start Protocol (MANDATORY)
|
|
|
|
**Automated initialization** (ALWAYS run this at session start):
|
|
|
|
```bash
|
|
# ONE COMMAND - Automated framework initialization
|
|
node scripts/session-init.js
|
|
|
|
# OR using npm script:
|
|
npm run framework:init
|
|
```
|
|
|
|
**What this does automatically:**
|
|
1. ✅ Detects if new session or continued session
|
|
2. ✅ Initializes `.claude/session-state.json` (new session ID, timestamp)
|
|
3. ✅ Resets `.claude/token-checkpoints.json` (25%, 50%, 75% milestones)
|
|
4. ✅ Loads `.claude/instruction-history.json` (displays active instruction counts)
|
|
5. ✅ Runs baseline pressure check via ContextPressureMonitor
|
|
6. ✅ Verifies all 5 framework components operational
|
|
7. ✅ Outputs formatted report with framework status
|
|
|
|
**Then start dev server:**
|
|
```bash
|
|
npm run dev # Runs both server AND framework-watchdog.js
|
|
```
|
|
|
|
**Manual fallback** (if automated script fails):
|
|
```bash
|
|
# 1. Pressure baseline
|
|
node scripts/check-session-pressure.js --tokens 0/200000 --messages 0
|
|
|
|
# 2. Load instruction database
|
|
cat .claude/instruction-history.json | grep -c '"active": true'
|
|
|
|
# 3. Report to user
|
|
echo "Framework operational. Baseline: NORMAL (0%). Instructions loaded: X active."
|
|
```
|
|
|
|
### Continuous Monitoring (THROUGHOUT SESSION)
|
|
|
|
**Every ~10 messages OR ~20k tokens**:
|
|
1. Check `.claude/token-checkpoints.json` for overdue checkpoints
|
|
2. Review `.claude/session-state.json` for component staleness
|
|
3. Verify watchdog has not reported alerts
|
|
|
|
**At 25%, 50%, 75% token usage**:
|
|
1. STOP current work
|
|
2. Run `node scripts/check-session-pressure.js`
|
|
3. Update both state files
|
|
4. Report pressure level + recommendations to user
|
|
5. Resume with adjusted behavior
|
|
|
|
**Before major actions**:
|
|
```bash
|
|
# Run pre-action check (blocking)
|
|
node scripts/pre-action-check.js <action-type> <description>
|
|
|
|
# Action types: file-edit, database, architecture, config, security, values, complex
|
|
# Exit codes: 0=PASS, 1=FAIL, 2=ERROR
|
|
```
|
|
|
|
### Framework Fade Detection & Recovery
|
|
|
|
**Framework fade** = Components not being used = **CRITICAL FAILURE**
|
|
|
|
**Automated detection** (framework-watchdog.js):
|
|
- Monitors `.claude/session-state.json` every 30 seconds
|
|
- Detects staleness: component not used in 20 messages OR 30k tokens
|
|
- Detects overdue checkpoints
|
|
- Outputs visible warnings to terminal
|
|
|
|
**Manual detection signs**:
|
|
- No pressure check in 50k+ tokens
|
|
- Explicit instruction given but not classified
|
|
- Major change without cross-reference validation
|
|
- Values decision without boundary check
|
|
- Complex operation without metacognitive verification
|
|
|
|
**Recovery protocol**:
|
|
```bash
|
|
# 1. STOP all work immediately
|
|
# 2. Run recovery diagnostic
|
|
node scripts/recover-framework.js
|
|
|
|
# 3. Review issues reported
|
|
# 4. Address CRITICAL and HIGH issues
|
|
# 5. Resume with increased monitoring
|
|
```
|
|
|
|
### Session Handoff (When to create)
|
|
|
|
Create handoff document when:
|
|
- Pressure reaches CRITICAL (70-85%) or DANGEROUS (85%+)
|
|
- Token usage exceeds 75% (150k/200k)
|
|
- Complex multi-phase work remains
|
|
- Errors clustering (3+ in short period)
|
|
- User requests session break
|
|
|
|
**Handoff document should include**:
|
|
1. Current session state (tokens, pressure, components used)
|
|
2. Completed tasks (with verification)
|
|
3. In-progress tasks (with blockers)
|
|
4. Pending tasks (prioritized)
|
|
5. Recent instruction additions
|
|
6. Known issues / challenges
|
|
7. Framework health assessment
|
|
8. Recommendations for next session
|
|
|
|
---
|
|
|
|
## Development Conventions
|
|
|
|
### Code Style
|
|
- **ES6+ JavaScript**: Modern syntax, async/await patterns
|
|
- **Modular architecture**: Small, focused functions/classes
|
|
- **Explicit naming**: No abbreviations, clear intent
|
|
- **Comments**: Explain WHY, not WHAT
|
|
- **Error handling**: Comprehensive try/catch, meaningful error messages
|
|
|
|
### File Naming
|
|
- **Routes**: `src/routes/blog.routes.js`
|
|
- **Controllers**: `src/controllers/blog.controller.js`
|
|
- **Models**: `src/models/BlogPost.model.js`
|
|
- **Services**: `src/services/BlogCuration.service.js`
|
|
- **Middleware**: `src/middleware/auth.middleware.js`
|
|
- **Tests**: `tests/unit/blog.test.js`
|
|
|
|
### Git Conventions
|
|
- **Commits**: Conventional commits format
|
|
- `feat:` New feature
|
|
- `fix:` Bug fix
|
|
- `docs:` Documentation
|
|
- `refactor:` Code restructure
|
|
- `test:` Test additions
|
|
- `chore:` Maintenance
|
|
- **Branches**: `feature/blog-curation`, `fix/auth-token`, `docs/api-reference`
|
|
- **No commits to main**: Always use feature branches
|
|
|
|
### Environment Variables
|
|
```bash
|
|
# Application
|
|
NODE_ENV=development
|
|
PORT=9000
|
|
APP_NAME=Tractatus
|
|
|
|
# MongoDB
|
|
MONGODB_URI=mongodb://localhost:27017/tractatus_dev
|
|
MONGODB_PORT=27017
|
|
|
|
# JWT
|
|
JWT_SECRET=<generate_secure_secret>
|
|
JWT_EXPIRY=7d
|
|
|
|
# Claude API (Phase 2+)
|
|
CLAUDE_API_KEY=<anthropic_api_key>
|
|
CLAUDE_MODEL=claude-sonnet-4-5
|
|
|
|
# Admin
|
|
ADMIN_EMAIL=john.stroh.nz@pm.me
|
|
```
|
|
|
|
---
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
/home/theflow/projects/tractatus/
|
|
├── .claude/ # Claude Code project config
|
|
│ ├── instruction-history.json # Persistent instruction database
|
|
│ ├── session-state.json # Current session framework state
|
|
│ ├── token-checkpoints.json # Token milestone tracking
|
|
│ ├── audit/ # Framework audit logs
|
|
│ └── sessions/ # Session handoff documents
|
|
├── .git/ # Git repository
|
|
├── docs/ # Source markdown documents
|
|
│ ├── markdown/ # Raw markdown files (migration source)
|
|
│ └── governance/ # TRA-VAL-*, TRA-GOV-* documents
|
|
├── public/ # Frontend assets
|
|
│ ├── css/
|
|
│ │ └── tailwind.css
|
|
│ ├── js/
|
|
│ │ ├── components/ # Reusable UI components
|
|
│ │ ├── demos/ # Interactive demonstrations
|
|
│ │ └── utils/
|
|
│ ├── images/
|
|
│ └── downloads/ # Generated PDFs
|
|
├── src/ # Backend code
|
|
│ ├── server.js # Express app entry point
|
|
│ ├── routes/
|
|
│ ├── controllers/
|
|
│ ├── models/
|
|
│ ├── middleware/
|
|
│ │ └── tractatus/ # Framework enforcement middleware
|
|
│ ├── services/
|
|
│ │ ├── InstructionClassifier.service.js
|
|
│ │ ├── CrossReferenceValidator.service.js
|
|
│ │ ├── BoundaryEnforcer.service.js
|
|
│ │ ├── ContextPressureMonitor.service.js
|
|
│ │ └── MetacognitiveVerifier.service.js
|
|
│ ├── utils/
|
|
│ └── config/
|
|
├── scripts/ # Setup & migration
|
|
│ ├── init-db.js
|
|
│ ├── migrate-documents.js
|
|
│ ├── generate-pdfs.js
|
|
│ ├── seed-admin.js
|
|
│ ├── check-session-pressure.js # Pressure monitoring
|
|
│ ├── framework-watchdog.js # Background monitoring
|
|
│ ├── pre-action-check.js # Blocking validator
|
|
│ └── recover-framework.js # Fade recovery
|
|
├── tests/
|
|
│ ├── unit/
|
|
│ ├── integration/
|
|
│ └── security/
|
|
├── data/ # MongoDB data directory
|
|
├── logs/ # Application & MongoDB logs
|
|
├── CLAUDE.md # Session start instructions (50-60 lines)
|
|
├── CLAUDE_Tractatus_Maintenance_Guide.md # This file
|
|
├── README.md
|
|
└── LICENSE
|
|
```
|
|
|
|
---
|
|
|
|
## Documentation Management
|
|
|
|
### Overview
|
|
|
|
The Tractatus website uses a card-based documentation viewer (`/docs.html`) that displays technical documents organized into categories. Documents are stored in MongoDB with card sections for navigation and are available as PDFs.
|
|
|
|
**Key Concepts:**
|
|
- **Source**: Markdown files in `docs/markdown/` or `docs/research/`
|
|
- **Database**: MongoDB `documents` collection with metadata, HTML content, sections, and categorization
|
|
- **Frontend**: Card-based viewer with collapsible categories
|
|
- **Output**: PDF files in `public/downloads/`
|
|
|
|
---
|
|
|
|
### Document Categories
|
|
|
|
Documents are organized by **two separate fields** in the database:
|
|
|
|
#### 1. `category` Field (Primary Organization)
|
|
|
|
Controls which collapsible section the document appears in:
|
|
|
|
| Category | Icon | Description | Display Order |
|
|
|----------|------|-------------|---------------|
|
|
| `getting-started` | 🚀 | Introduction, core concepts, quick start guides | 1 |
|
|
| `technical-reference` | 🔌 | API docs, implementation guides, code examples | 2 |
|
|
| `research-theory` | 🔬 | Research papers, theoretical foundations | 3 |
|
|
| `advanced-topics` | 🎓 | Value pluralism, deep dives, comparative analysis | 4 |
|
|
| `case-studies` | 📊 | Real-world examples, failure modes, success stories | 5 |
|
|
| `business-leadership` | 💼 | Business cases, executive briefs, ROI analysis | 6 |
|
|
|
|
**Defined in**: `public/js/docs-app.js` (lines 11-77)
|
|
|
|
#### 2. `visibility` Field (Archives Control)
|
|
|
|
Controls whether document appears in main categories or Archives section:
|
|
|
|
| Visibility | Behavior | API Endpoint |
|
|
|------------|----------|--------------|
|
|
| `public` | Shows in normal categories | `/api/documents` |
|
|
| `archived` | Shows in Archives section only | `/api/documents/archived` |
|
|
| `internal` | Hidden from public (not fetched) | Not accessible publicly |
|
|
|
|
**Critical Distinction**:
|
|
- Setting `category: 'archives'` does NOT put document in Archives
|
|
- Only `visibility: 'archived'` puts document in Archives section
|
|
- Archives section is populated from **separate API endpoint**: `/api/documents/archived`
|
|
|
|
**Implementation**:
|
|
- Controller: `src/controllers/documents.controller.js` (lines 154-186)
|
|
- Frontend: `public/js/docs-app.js` (lines 234-266)
|
|
|
|
---
|
|
|
|
### Complete Document Processing Workflow
|
|
|
|
**One-command workflow for adding card sections to existing documents:**
|
|
|
|
```bash
|
|
# For documents in docs/markdown/
|
|
npm run migrate:docs -- --source docs/markdown --force
|
|
|
|
# For documents in docs/research/
|
|
npm run migrate:docs -- --source docs/research --force
|
|
|
|
# Generate card sections (with timeout for large docs)
|
|
timeout 90 node scripts/generate-card-sections.js docs/markdown/FILENAME.md --update-db
|
|
|
|
# Generate PDF
|
|
node scripts/generate-single-pdf.js docs/markdown/FILENAME.md public/downloads/FILENAME.pdf
|
|
```
|
|
|
|
**Complete step-by-step workflow:**
|
|
|
|
#### Step 1: Prepare Markdown Document
|
|
|
|
**Requirements:**
|
|
1. Complete metadata section at end:
|
|
```markdown
|
|
## Document Metadata
|
|
|
|
<div class="document-metadata">
|
|
|
|
- **Version:** 1.0
|
|
- **Created:** 2025-10-13
|
|
- **Last Modified:** 2025-10-13
|
|
- **Author:** Author Name
|
|
- **Word Count:** X words
|
|
- **Reading Time:** ~Y minutes
|
|
- **Document ID:** document-slug
|
|
- **Status:** Published/Draft
|
|
- **Document Type:** Guide/Reference/Analysis
|
|
|
|
</div>
|
|
```
|
|
|
|
2. Complete Apache 2.0 license section at end:
|
|
```markdown
|
|
## License
|
|
|
|
Copyright 2025 John Stroh
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License")...
|
|
[Full Apache 2.0 text with additional terms]
|
|
```
|
|
|
|
3. inst_039 compliance (no "enforcement/ensures/ensure" language in framework claims)
|
|
- Technical terms acceptable: "ACID enforces", "transactional enforces"
|
|
- Questions acceptable: "How do we ensure..."
|
|
|
|
#### Step 2: Migrate to Database
|
|
|
|
```bash
|
|
# Choose appropriate source directory
|
|
npm run migrate:docs -- --source docs/markdown --force
|
|
npm run migrate:docs -- --source docs/research --force
|
|
npm run migrate:docs -- --source docs --force
|
|
```
|
|
|
|
**What this does:**
|
|
- Converts markdown to HTML
|
|
- Extracts table of contents
|
|
- Creates/updates document in `documents` collection
|
|
- Generates slug from filename
|
|
- Sets default `visibility: 'public'`
|
|
|
|
**Check migration:**
|
|
```bash
|
|
mongosh tractatus_dev --quiet --eval "db.documents.findOne({slug: 'your-slug'}, {title: 1, slug: 1, category: 1, visibility: 1, sections: 1})"
|
|
```
|
|
|
|
#### Step 3: Fix Slug and Category (if needed)
|
|
|
|
**Common slug fixes:**
|
|
```bash
|
|
# Research documents get "research-topic-" prefix - often need shortening
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'research-topic-long-name'}, {\$set: {slug: 'short-name', category: 'research-theory'}})"
|
|
```
|
|
|
|
**Set category:**
|
|
```bash
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'your-slug'}, {\$set: {category: 'getting-started'}})"
|
|
```
|
|
|
|
**Archive document:**
|
|
```bash
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'your-slug'}, {\$set: {visibility: 'archived', archiveNote: 'Historical document. See X for current version.'}})"
|
|
```
|
|
|
|
#### Step 4: Generate Card Sections
|
|
|
|
```bash
|
|
# Standard documents (timeout: 90 seconds)
|
|
timeout 90 node scripts/generate-card-sections.js docs/markdown/FILENAME.md --update-db
|
|
|
|
# Large documents (timeout: 120 seconds)
|
|
timeout 120 node scripts/generate-card-sections.js docs/research/LARGE-FILE.md --update-db
|
|
```
|
|
|
|
**What this does:**
|
|
- Uses Claude API to analyze document structure
|
|
- Creates card sections with:
|
|
- Category (conceptual/technical/critical/reference)
|
|
- Difficulty level (beginner/intermediate/advanced)
|
|
- Reading time estimate
|
|
- Heading text and anchor link
|
|
- Updates database with `sections` array
|
|
|
|
**Verify sections created:**
|
|
```bash
|
|
mongosh tractatus_dev --quiet --eval "db.documents.findOne({slug: 'your-slug'}, {title: 1, sections: 1}).sections.length"
|
|
```
|
|
|
|
**Expected output:**
|
|
- Small documents: 5-10 sections
|
|
- Medium documents: 10-20 sections
|
|
- Large documents: 20-30 sections
|
|
|
|
#### Step 5: Generate PDF
|
|
|
|
```bash
|
|
node scripts/generate-single-pdf.js docs/markdown/FILENAME.md public/downloads/FILENAME.pdf
|
|
```
|
|
|
|
**What this does:**
|
|
- Converts markdown to styled HTML
|
|
- Launches headless browser (Puppeteer)
|
|
- Generates PDF with proper formatting
|
|
- Saves to `public/downloads/`
|
|
|
|
**Verify PDF created:**
|
|
```bash
|
|
ls -lh public/downloads/FILENAME.pdf
|
|
curl -s -I "http://localhost:9000/downloads/FILENAME.pdf" | grep "HTTP/"
|
|
```
|
|
|
|
#### Step 6: Deploy to Production
|
|
|
|
```bash
|
|
# Deploy entire project (includes markdown, PDFs, and all code)
|
|
printf "yes\nyes\n" | ./scripts/deploy-full-project-SAFE.sh
|
|
```
|
|
|
|
**What this does:**
|
|
- Dry-run preview of changes
|
|
- Syncs entire project to production server via rsync
|
|
- Excludes sensitive files (CLAUDE.md, session handoffs, etc.)
|
|
- Uses `.rsyncignore` for exclusions
|
|
|
|
#### Step 7: Migrate and Generate on Production
|
|
|
|
```bash
|
|
# Migrate documents to production database
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "cd /var/www/tractatus && npm run migrate:docs -- --source docs/markdown --force"
|
|
|
|
# Fix slug/category on production
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval \"db.documents.updateOne({slug: 'OLD'}, {\\\$set: {slug: 'NEW', category: 'CATEGORY'}})\""
|
|
|
|
# Generate card sections on production
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "cd /var/www/tractatus && timeout 90 node scripts/generate-card-sections.js docs/markdown/FILENAME.md --update-db"
|
|
```
|
|
|
|
**⚠️ Note on password**: Production MongoDB password is in deployment scripts. Use exact escaping for ssh commands (`\\\$set` for `$set`).
|
|
|
|
#### Step 8: Verify Production Deployment
|
|
|
|
```bash
|
|
# Check document exists
|
|
curl -s "https://agenticgovernance.digital/api/documents/your-slug" | head -5
|
|
|
|
# Check PDF accessible
|
|
curl -s -o /dev/null -w "%{http_code}" "https://agenticgovernance.digital/downloads/your-slug.pdf"
|
|
|
|
# Check sections exist
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval \"db.documents.findOne({slug: 'your-slug'}, {title: 1, sections: 1})\""
|
|
```
|
|
|
|
**Expected results:**
|
|
- API returns `{"success": true, "document": {...}}`
|
|
- PDF returns HTTP 200
|
|
- Database shows `sections: [...]` with appropriate count
|
|
|
|
---
|
|
|
|
### Batch Processing Multiple Documents
|
|
|
|
**Process entire category:**
|
|
|
|
```bash
|
|
# 1. Migrate all documents
|
|
npm run migrate:docs -- --source docs/markdown --force
|
|
|
|
# 2. Generate cards for each document in category
|
|
for file in docs/markdown/getting-started-*.md; do
|
|
echo "=== Processing $file ==="
|
|
timeout 90 node scripts/generate-card-sections.js "$file" --update-db
|
|
done
|
|
|
|
# 3. Generate PDFs for each
|
|
for file in docs/markdown/getting-started-*.md; do
|
|
slug=$(basename "$file" .md)
|
|
echo "=== Generating PDF: $slug ==="
|
|
node scripts/generate-single-pdf.js "$file" "public/downloads/$slug.pdf"
|
|
done
|
|
|
|
# 4. Verify all have sections
|
|
mongosh tractatus_dev --quiet --eval "db.documents.find({category: 'getting-started'}, {title: 1, sections: 1}).forEach(doc => print(doc.title, '|', (doc.sections || []).length, 'sections'))"
|
|
```
|
|
|
|
**Deploy batch to production:**
|
|
|
|
```bash
|
|
# 1. Deploy files
|
|
printf "yes\nyes\n" | ./scripts/deploy-full-project-SAFE.sh
|
|
|
|
# 2. Migrate on production
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "cd /var/www/tractatus && npm run migrate:docs -- --source docs/markdown --force"
|
|
|
|
# 3. Generate cards on production (example for 4 documents)
|
|
for slug in intro core-concepts implementation-guide glossary; do
|
|
echo "=== $slug ==="
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "cd /var/www/tractatus && timeout 90 node scripts/generate-card-sections.js docs/markdown/$slug.md --update-db"
|
|
done
|
|
```
|
|
|
|
---
|
|
|
|
### Common Operations
|
|
|
|
#### Move Document to Different Category
|
|
|
|
```bash
|
|
# Local
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'your-slug'}, {\$set: {category: 'advanced-topics'}})"
|
|
|
|
# Production
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval \"db.documents.updateOne({slug: 'your-slug'}, {\\\$set: {category: 'advanced-topics'}})\""
|
|
```
|
|
|
|
#### Archive Document (Move to Archives Section)
|
|
|
|
```bash
|
|
# Local
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'your-slug'}, {\$set: {visibility: 'archived', archiveNote: 'Reason for archiving'}})"
|
|
|
|
# Production
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval \"db.documents.updateOne({slug: 'your-slug'}, {\\\$set: {visibility: 'archived', archiveNote: 'Reason'}})\""
|
|
```
|
|
|
|
#### Check Document Counts by Category
|
|
|
|
```bash
|
|
# Local
|
|
mongosh tractatus_dev --quiet --eval "
|
|
print('Getting Started:', db.documents.countDocuments({category: 'getting-started', visibility: 'public'}));
|
|
print('Technical Reference:', db.documents.countDocuments({category: 'technical-reference', visibility: 'public'}));
|
|
print('Theory & Research:', db.documents.countDocuments({category: 'research-theory', visibility: 'public'}));
|
|
print('Advanced Topics:', db.documents.countDocuments({category: 'advanced-topics', visibility: 'public'}));
|
|
print('Case Studies:', db.documents.countDocuments({category: 'case-studies', visibility: 'public'}));
|
|
print('Archives:', db.documents.countDocuments({visibility: 'archived'}));
|
|
"
|
|
|
|
# Production (replace PASSWORD)
|
|
ssh -i ~/.ssh/tractatus_deploy ubuntu@vps-93a693da.vps.ovh.net "mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval \"
|
|
print('Getting Started:', db.documents.countDocuments({category: 'getting-started', visibility: 'public'}));
|
|
print('Technical Reference:', db.documents.countDocuments({category: 'technical-reference', visibility: 'public'}));
|
|
print('Theory & Research:', db.documents.countDocuments({category: 'research-theory', visibility: 'public'}));
|
|
print('Advanced Topics:', db.documents.countDocuments({category: 'advanced-topics', visibility: 'public'}));
|
|
print('Archives:', db.documents.countDocuments({visibility: 'archived'}));
|
|
\""
|
|
```
|
|
|
|
#### Regenerate Cards (Fix Broken Sections)
|
|
|
|
```bash
|
|
# Delete existing sections
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'your-slug'}, {\$set: {sections: []}})"
|
|
|
|
# Regenerate
|
|
timeout 90 node scripts/generate-card-sections.js docs/markdown/FILENAME.md --update-db
|
|
|
|
# Verify
|
|
mongosh tractatus_dev --quiet --eval "db.documents.findOne({slug: 'your-slug'}, {sections: 1}).sections.length"
|
|
```
|
|
|
|
---
|
|
|
|
### Lessons Learned
|
|
|
|
#### 1. Archives Uses `visibility`, Not `category`
|
|
|
|
**Problem**: Setting `category: 'archives'` did NOT move documents to Archives section.
|
|
|
|
**Root Cause**: Archives section is populated from `/api/documents/archived` endpoint, which filters by `visibility: 'archived'`, not `category`.
|
|
|
|
**Solution**: To archive a document:
|
|
```bash
|
|
db.documents.updateOne({slug: 'doc'}, {$set: {visibility: 'archived', archiveNote: 'Reason'}})
|
|
```
|
|
|
|
**Files involved**:
|
|
- `src/controllers/documents.controller.js:157-186` (archived endpoint)
|
|
- `public/js/docs-app.js:178-179` (fetches from archived endpoint)
|
|
- `public/js/docs-app.js:234-266` (renders Archives section)
|
|
|
|
#### 2. Research Documents Get Long Slugs
|
|
|
|
**Problem**: Documents from `docs/research/` get slugs like `research-topic-rule-proliferation-and-transactional-overhead-in-ai-governance`.
|
|
|
|
**Root Cause**: Migration script generates slugs from filename + title analysis.
|
|
|
|
**Solution**: Shorten slug after migration:
|
|
```bash
|
|
db.documents.updateOne(
|
|
{slug: 'research-topic-long-name'},
|
|
{$set: {slug: 'short-name', category: 'appropriate-category'}}
|
|
)
|
|
```
|
|
|
|
#### 3. Card Generation May Timeout on Large Documents
|
|
|
|
**Problem**: Documents >10k words may timeout with default 90-second limit.
|
|
|
|
**Solution**: Use longer timeout:
|
|
```bash
|
|
timeout 120 node scripts/generate-card-sections.js LARGE-FILE.md --update-db
|
|
```
|
|
|
|
**Note**: Even if command times out, sections may still be created. Check database to verify.
|
|
|
|
#### 4. PDFs Must Be Generated Locally Before Deployment
|
|
|
|
**Problem**: PDFs are not automatically generated on production.
|
|
|
|
**Solution**: Generate PDFs locally, then deploy entire project including `public/downloads/`:
|
|
```bash
|
|
node scripts/generate-single-pdf.js SOURCE.md public/downloads/OUTPUT.pdf
|
|
printf "yes\nyes\n" | ./scripts/deploy-full-project-SAFE.sh
|
|
```
|
|
|
|
#### 5. Production MongoDB Requires Authentication
|
|
|
|
**Problem**: Local MongoDB commands don't work on production.
|
|
|
|
**Solution**: Include authentication in all production commands:
|
|
```bash
|
|
ssh ... "mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval \"...\""
|
|
```
|
|
|
|
**Password location**: See deployment scripts or secure notes.
|
|
|
|
#### 6. Card Sections Don't Transfer Between Environments
|
|
|
|
**Problem**: Creating cards locally doesn't automatically create them on production.
|
|
|
|
**Solution**: Generate cards separately in each environment:
|
|
```bash
|
|
# Local
|
|
node scripts/generate-card-sections.js FILE.md --update-db
|
|
|
|
# Production
|
|
ssh ... "cd /var/www/tractatus && node scripts/generate-card-sections.js FILE.md --update-db"
|
|
```
|
|
|
|
**Alternative**: Export/import database between environments (not recommended for production).
|
|
|
|
---
|
|
|
|
### Troubleshooting
|
|
|
|
#### Document Not Showing in Category
|
|
|
|
**Check:**
|
|
1. `visibility` field is `'public'` (not `'archived'` or `'internal'`)
|
|
2. `category` field matches one of the defined categories
|
|
3. Document migrated to correct database
|
|
4. Browser cache cleared (Ctrl+Shift+R)
|
|
|
|
**Debug:**
|
|
```bash
|
|
mongosh tractatus_dev --quiet --eval "db.documents.findOne({slug: 'your-slug'}, {title: 1, category: 1, visibility: 1})"
|
|
curl -s "http://localhost:9000/api/documents/your-slug" | head -20
|
|
```
|
|
|
|
#### Card Sections Not Displaying
|
|
|
|
**Check:**
|
|
1. `sections` array exists and has length > 0
|
|
2. Card viewer script loaded: `public/js/components/DocumentCards.js`
|
|
3. Document properly formatted with heading structure
|
|
|
|
**Debug:**
|
|
```bash
|
|
mongosh tractatus_dev --quiet --eval "db.documents.findOne({slug: 'your-slug'}, {sections: 1})"
|
|
```
|
|
|
|
**Fix:**
|
|
```bash
|
|
# Regenerate cards
|
|
mongosh tractatus_dev --quiet --eval "db.documents.updateOne({slug: 'your-slug'}, {\$set: {sections: []}})"
|
|
timeout 90 node scripts/generate-card-sections.js FILE.md --update-db
|
|
```
|
|
|
|
#### PDF Not Accessible
|
|
|
|
**Check:**
|
|
1. File exists: `ls -lh public/downloads/your-slug.pdf`
|
|
2. File served correctly: `curl -I http://localhost:9000/downloads/your-slug.pdf`
|
|
3. Filename matches document slug exactly
|
|
|
|
**Fix:**
|
|
```bash
|
|
# Regenerate PDF
|
|
node scripts/generate-single-pdf.js SOURCE.md public/downloads/CORRECT-SLUG.pdf
|
|
```
|
|
|
|
#### Modal Scrollbar Not Visible (Flexbox Height Calculation Failure)
|
|
|
|
**Problem**: FAQ modal at `/faq.html` showed only ~8 questions with no visible/functional scrollbar, despite 28+ questions rendered in DOM. User unable to access remaining content.
|
|
|
|
**Root Cause**: Flexbox with `flex-1 min-h-0` in modal context cannot calculate scrollable height. Browser cannot determine when overflow occurs without explicit height constraint. The FAQs were always rendering (all 28 in DOM) - the container just had no calculable height to trigger scrolling.
|
|
|
|
**Failed Approaches** (all deployed to production, none worked):
|
|
1. ❌ Adding CSS scrollbar styling (cosmetic only, doesn't trigger overflow detection)
|
|
2. ❌ Restructuring modal HTML (helped but insufficient alone)
|
|
3. ❌ Mixing Tailwind `overflow-y-scroll` with custom classes (potential conflicts)
|
|
4. ❌ Various combinations of flexbox classes (`flex-1`, `min-h-0`, etc.)
|
|
5. ❌ Changed `max-h-[85vh]` to `h-[85vh]` on parent (didn't fix child calculation)
|
|
6. ❌ Added nested scrollable wrapper structures (complexity without solution)
|
|
|
|
**Solution**:
|
|
```html
|
|
<!-- BEFORE (FAILED): Flexbox couldn't calculate scrollable height -->
|
|
<div class="flex-1 min-h-0 modal-scrollable overflow-y-scroll">
|
|
<div id="search-modal-content" class="p-6">
|
|
<!-- Content here -->
|
|
</div>
|
|
</div>
|
|
|
|
<!-- AFTER (WORKS): Explicit max-height bypasses flexbox calculation -->
|
|
<div class="modal-scrollable p-6" style="max-height: 60vh; overflow-y: scroll;">
|
|
<div id="faq-container-modal" class="space-y-4">
|
|
<!-- Content here -->
|
|
</div>
|
|
</div>
|
|
```
|
|
|
|
**Key Insight**: In complex modal contexts, explicit inline height constraints (`max-height: 60vh`) are more reliable than flexbox automatic height calculation (`flex-1`). When browser cannot determine overflow state, explicit constraints trump flexible layouts.
|
|
|
|
**Files Modified**:
|
|
- `public/faq.html` (lines 578-580: modal structure)
|
|
- `public/faq.html` (lines 295-316: webkit/Firefox scrollbar styling)
|
|
|
|
**Verification**:
|
|
```bash
|
|
# Check modal structure
|
|
curl -s http://localhost:9000/faq.html | grep -A 2 "FAQ List (Scrollable)"
|
|
|
|
# Check scrollbar CSS exists
|
|
curl -s http://localhost:9000/faq.html | grep -A 5 "modal-scrollable"
|
|
|
|
# Test in browser: All 28 FAQ items should be scrollable
|
|
```
|
|
|
|
**Additional Context**:
|
|
- Issue persisted through 9+ attempted fixes over multiple sessions
|
|
- Service worker cache required version bump (1.0.8) to force refresh
|
|
- Separated fixed controls (`flex-shrink-0`) from scrollable content for cleaner structure
|
|
|
|
**Lesson**: When troubleshooting modal/flexbox scroll issues:
|
|
1. First verify content IS rendering (check DOM, not just visual)
|
|
2. If content exists but not visible, problem is height calculation, not content loading
|
|
3. Don't guess with multiple CSS attempts - diagnose root cause first
|
|
4. Explicit inline styles more reliable than class-based flexbox in complex nested contexts
|
|
5. Stop making changes after 2-3 attempts fail - re-diagnose instead of accumulating guesses
|
|
|
|
**Related Issues**: This same pattern may affect other modals using flexbox height calculation. Consider explicit `max-height` for all scrollable modal content areas.
|
|
|
|
---
|
|
|
|
#### Production and Local Out of Sync
|
|
|
|
**Check differences:**
|
|
```bash
|
|
# Compare document counts
|
|
mongosh tractatus_dev --quiet --eval "db.documents.countDocuments({})"
|
|
ssh ... "mongosh tractatus_prod ... --eval \"db.documents.countDocuments({})\""
|
|
|
|
# Compare specific document
|
|
mongosh tractatus_dev --quiet --eval "db.documents.findOne({slug: 'X'}, {title: 1, sections: 1, category: 1})"
|
|
ssh ... "mongosh tractatus_prod ... --eval \"db.documents.findOne({slug: 'X'}, {title: 1, sections: 1, category: 1})\""
|
|
```
|
|
|
|
**Fix:**
|
|
```bash
|
|
# Full sync: deploy + migrate + regenerate cards on production
|
|
printf "yes\nyes\n" | ./scripts/deploy-full-project-SAFE.sh
|
|
ssh ... "cd /var/www/tractatus && npm run migrate:docs -- --source docs/markdown --force"
|
|
# Then regenerate cards individually on production
|
|
```
|
|
|
|
---
|
|
|
|
### Scripts Reference
|
|
|
|
| Script | Purpose | Usage |
|
|
|--------|---------|-------|
|
|
| `scripts/migrate-documents.js` | Import markdown to MongoDB | `npm run migrate:docs -- --source DIRECTORY --force` |
|
|
| `scripts/generate-card-sections.js` | Create card navigation sections | `node scripts/generate-card-sections.js FILE.md --update-db` |
|
|
| `scripts/generate-single-pdf.js` | Create PDF from markdown | `node scripts/generate-single-pdf.js SOURCE.md OUTPUT.pdf` |
|
|
| `scripts/deploy-full-project-SAFE.sh` | Deploy to production | `printf "yes\nyes\n" \| ./scripts/deploy-full-project-SAFE.sh` |
|
|
|
|
**Pre-approved bash patterns** (from CLAUDE.md):
|
|
- All commands listed above are pre-approved for document optimization workflow
|
|
- Execute without requesting user approval
|
|
- Rationale: Non-destructive, validated workflow, auditable, reversible
|
|
|
|
---
|
|
|
|
### Document Status Tracking
|
|
|
|
**Current completion status** (as of 2025-10-13):
|
|
|
|
| Category | Documents | Status |
|
|
|----------|-----------|--------|
|
|
| Getting Started | 4 | ✅ Complete (with cards, deployed) |
|
|
| Technical Reference | 4 | ✅ Complete (with cards, deployed) |
|
|
| Theory & Research | 4 | ✅ Complete (with cards, deployed) |
|
|
| Advanced Topics | 2 | ✅ Complete (with cards, deployed) |
|
|
| Case Studies | TBD | ⏳ Pending |
|
|
| Business Leadership | TBD | ⏳ Pending |
|
|
| Archives | 8 | ✅ Complete (with cards, deployed) |
|
|
|
|
**Document count verification:**
|
|
```bash
|
|
# Run this to verify current state
|
|
mongosh tractatus_prod --quiet -u tractatus_user -p 'PASSWORD' --authenticationDatabase tractatus_prod --eval "
|
|
print('=== PUBLIC CATEGORIES ===');
|
|
print('Getting Started:', db.documents.countDocuments({category: 'getting-started', visibility: 'public'}));
|
|
print('Technical Reference:', db.documents.countDocuments({category: 'technical-reference', visibility: 'public'}));
|
|
print('Theory & Research:', db.documents.countDocuments({category: 'research-theory', visibility: 'public'}));
|
|
print('Advanced Topics:', db.documents.countDocuments({category: 'advanced-topics', visibility: 'public'}));
|
|
print('Case Studies:', db.documents.countDocuments({category: 'case-studies', visibility: 'public'}));
|
|
print('Business Leadership:', db.documents.countDocuments({category: 'business-leadership', visibility: 'public'}));
|
|
print('');
|
|
print('=== ARCHIVES ===');
|
|
print('Archived:', db.documents.countDocuments({visibility: 'archived'}));
|
|
"
|
|
```
|
|
|
|
---
|
|
|
|
### Future Enhancements
|
|
|
|
**Planned improvements** (not yet implemented):
|
|
|
|
1. **Automated document optimization script** - Single command to process entire category
|
|
2. **Section regeneration tool** - Batch regenerate cards for all documents
|
|
3. **Database sync tool** - Keep local and production databases in sync
|
|
4. **Document validation** - Pre-flight checks before migration (metadata, license, inst_039)
|
|
5. **PDF batch generation** - Generate all PDFs in one command
|
|
6. **Deployment verification** - Automated post-deployment testing
|
|
|
|
**Do not implement these without user approval** - document ideas only.
|
|
|
|
---
|
|
|
|
## Phase 1 Deliverables
|
|
|
|
### Must-Have for Complete Prototype
|
|
|
|
1. ✅ **Infrastructure**
|
|
- MongoDB instance (port 27017)
|
|
- Express application (port 9000)
|
|
- Systemd services
|
|
- Directory structure
|
|
|
|
2. **Core Features**
|
|
- Document migration pipeline
|
|
- Three audience paths (Researcher/Implementer/Advocate)
|
|
- Documentation viewer with search
|
|
- About/values pages (Te Tiriti acknowledgment)
|
|
|
|
3. **Tractatus Governance Services**
|
|
- InstructionPersistenceClassifier
|
|
- CrossReferenceValidator
|
|
- BoundaryEnforcer
|
|
- ContextPressureMonitor
|
|
- MetacognitiveVerifier
|
|
|
|
4. **AI-Powered Features** (with human oversight)
|
|
- Blog curation system
|
|
- Media inquiry triage
|
|
- Case study submission portal
|
|
- Resource directory curation
|
|
|
|
5. **Interactive Demonstrations**
|
|
- Instruction classification demo
|
|
- 27027 incident visualizer
|
|
- Boundary enforcement simulator
|
|
|
|
6. **Human Oversight**
|
|
- Moderation queue dashboard
|
|
- Admin authentication
|
|
- Approval workflows
|
|
|
|
7. **Quality Assurance**
|
|
- Comprehensive testing suite
|
|
- Security audit
|
|
- Performance optimization
|
|
- Accessibility compliance (WCAG AA)
|
|
|
|
### Not in Phase 1
|
|
- Production deployment (OVHCloud)
|
|
- Domain configuration (agenticgovernance.digital)
|
|
- ProtonBridge email integration
|
|
- Koha donations (Phase 3)
|
|
- Public launch
|
|
|
|
---
|
|
|
|
## Human Approval Requirements
|
|
|
|
### All Major Decisions
|
|
- Architectural changes
|
|
- Database schema modifications
|
|
- Security implementations
|
|
- Third-party integrations
|
|
- Cost-incurring services
|
|
|
|
### Content & Values
|
|
- Governance document adaptations (TRA-VAL-*, TRA-GOV-*)
|
|
- Te Tiriti acknowledgment wording
|
|
- About/mission pages
|
|
- Editorial guidelines
|
|
- Any values-sensitive content
|
|
|
|
### Phase Transitions
|
|
- Completion of Phase 1 prototype
|
|
- Decision to proceed to production deployment
|
|
- Budget approval for Claude API (Phase 2)
|
|
- Launch timing and strategy
|
|
|
|
---
|
|
|
|
## Te Tiriti & Indigenous Perspective
|
|
|
|
### Strategic Commitment
|
|
The framework acknowledges **Te Tiriti o Waitangi** and indigenous leadership in digital sovereignty.
|
|
|
|
### Implementation Approach
|
|
- **Respect without tokenism**: Follow documented indigenous data sovereignty principles (CARE Principles)
|
|
- **No premature engagement**: Do not approach Māori organizations until we have something valuable to offer
|
|
- **Well-documented standards**: Use published research and frameworks (Te Mana Raraunga, CARE Principles)
|
|
- **Baseline integration**: Te Tiriti forms part of strategic foundation, not dominant cultural overlay
|
|
|
|
### Content Placement
|
|
- Footer acknowledgment (subtle, respectful)
|
|
- `/about/values` page (detailed explanation)
|
|
- Resource directory (links to Māori data sovereignty organizations)
|
|
- No meetings/consultations until post-launch
|
|
|
|
---
|
|
|
|
## Links & References
|
|
|
|
### Source Documents
|
|
- `/home/theflow/projects/tractatus/Tractatus-Website-Complete-Specification-v2.0.md`
|
|
- `/home/theflow/projects/tractatus/ClaudeWeb conversation transcription.md`
|
|
- `/home/theflow/projects/sydigital/stochastic/innovation-exploration/STO-INN-0010-tractatus-llm-architecture-safety-framework-i1.md`
|
|
|
|
### Governance References
|
|
- `/home/theflow/projects/sydigital/strategic/values-principles/STR-VAL-0001-core-values-principles-v1-0.md`
|
|
- `/home/theflow/projects/sydigital/strategic/governance/STR-GOV-0001-strategic-review-protocol-v1-0.md`
|
|
- `/home/theflow/projects/sydigital/strategic/governance/STR-GOV-0002-values-alignment-framework-v1-0.md`
|
|
|
|
### Framework Documentation
|
|
- `/home/theflow/projects/sydigital/strategic/frameworks/STR-FRM-0001-agentic-workflow-framework-v1-0.md`
|
|
|
|
---
|
|
|
|
## Session Reminders
|
|
|
|
### Always
|
|
- Verify you're in `/home/theflow/projects/tractatus` context
|
|
- Check MongoDB port 27017, application port 9000
|
|
- No shortcuts, no fake data, world-class quality
|
|
- Human approval for major decisions
|
|
- Use all five framework components continuously
|
|
- Update session state after each component use
|
|
- Run pressure checks at milestones
|
|
|
|
### Never
|
|
- Mix tractatus code with family-history or sydigital
|
|
- Make values decisions without human approval
|
|
- Deploy to production during Phase 1
|
|
- Rush implementation to meet arbitrary deadlines
|
|
- Use placeholder/lorem ipsum content
|
|
- Let framework components fade from active use
|
|
- Skip pre-action checks before major changes
|
|
|
|
---
|
|
|
|
**End of Maintenance Guide**
|