diff --git a/README.md b/README.md index 7b9a1d38..6e705438 100644 --- a/README.md +++ b/README.md @@ -185,7 +185,7 @@ const verification = verifier.verify({ During development, Claude (running with Tractatus governance) fabricated financial statistics on the landing page: - $3.77M in annual savings (no basis) - 1,315% ROI (completely invented) -- False claims of being "production-ready" +- False readiness claims (unverified maturity statements) **The framework structured the response:** diff --git a/docs/markdown/comparison-matrix.md b/docs/markdown/comparison-matrix.md index a28a9d92..3c4b3b22 100644 --- a/docs/markdown/comparison-matrix.md +++ b/docs/markdown/comparison-matrix.md @@ -519,7 +519,7 @@ AuditLogger (log decision) **CLAUDE.md:** Manual project documentation **Tractatus:** Automated governance enforcement -**Together:** Production-ready AI with architectural safety design +**Together:** Research-stage AI with architectural safety design **The Trade-Off:** - **Cost:** <10ms overhead, 1-2 days integration, MongoDB requirement diff --git a/docs/markdown/implementation-guide-v1.1.md b/docs/markdown/implementation-guide-v1.1.md index 72832681..259c3a1c 100644 --- a/docs/markdown/implementation-guide-v1.1.md +++ b/docs/markdown/implementation-guide-v1.1.md @@ -2,7 +2,7 @@ **Version**: 1.1 **Last Updated**: 2025-10-11 -**Status**: Production-Ready (Phase 5 Complete) +**Status**: Research Implementation (Phase 5 Complete) --- diff --git a/docs/markdown/introduction-to-the-tractatus-framework.md b/docs/markdown/introduction-to-the-tractatus-framework.md index b6fbc4b2..9dda8c25 100644 --- a/docs/markdown/introduction-to-the-tractatus-framework.md +++ b/docs/markdown/introduction-to-the-tractatus-framework.md @@ -174,7 +174,7 @@ Applied to AI safety: - Novel framework for values pluralism in AI ### Software Teams Deploying LLMs -- Production-ready code (tested, documented) +- Reference implementation code (tested, documented) - Immediate safety improvements - Integration guides for existing systems - Prevents known failure modes @@ -196,7 +196,7 @@ Applied to AI safety: **What Tractatus does NOT do:** - Train better LLMs (uses existing models as-is) - Ensure "aligned" AI behavior -- Eliminate all possible failures +- Reduce risk of failures - Replace human judgment **What Tractatus DOES do:** diff --git a/docs/markdown/introduction.md b/docs/markdown/introduction.md index 1f3c048a..5958cf1b 100644 --- a/docs/markdown/introduction.md +++ b/docs/markdown/introduction.md @@ -195,7 +195,7 @@ AI systems gradually make decisions in values-sensitive domains without realizin ### Implementers -- Production-ready code (Node.js, tested, documented) +- Reference implementation code (Node.js, tested, documented) - Integration guides for existing systems - Immediate safety improvements diff --git a/docs/markdown/llm-integration-feasibility-research-scope.md b/docs/markdown/llm-integration-feasibility-research-scope.md index 93eba6d6..34a7a562 100644 --- a/docs/markdown/llm-integration-feasibility-research-scope.md +++ b/docs/markdown/llm-integration-feasibility-research-scope.md @@ -617,10 +617,10 @@ LLM: [Uses 27027 despite training pattern] 4. Evaluate hybrid strategies (RAG for 80% + middleware for 20%) **Success Criteria**: -- 50 rules: <200ms overhead (<15% increase) -- 100 rules: <400ms overhead (<30% increase) -- 200 rules: <800ms overhead (<60% increase) -- Accuracy maintained across all scales (>95%) +- 50 rules: <200ms overhead (target: <15% increase [NEEDS VERIFICATION]) +- 100 rules: <400ms overhead (target: <30% increase [NEEDS VERIFICATION]) +- 200 rules: <800ms overhead (target: <60% increase [NEEDS VERIFICATION]) +- Accuracy target: >95% across all scales [NEEDS VERIFICATION] --- @@ -820,8 +820,8 @@ LLM: [Uses 27027 despite training pattern] **Minimum Viable Integration**: - ✅ Instruction persistence: 100% across 50+ conversation turns - ✅ Override prevention: <2% failure rate (vs. ~15% baseline) -- ✅ Latency impact: <15% increase for 50-rule database -- ✅ Scalability: Support 100 rules with <30% overhead +- ✅ Latency impact: <15% increase [NEEDS VERIFICATION] for 50-rule database +- ✅ Scalability: Support 100 rules with <30% overhead [NEEDS VERIFICATION] - ✅ Multi-tenant: 5-level hierarchy with <10ms conflict resolution **Stretch Goals**: @@ -1336,7 +1336,7 @@ If PoC successful (95%+ enforcement, <20% latency, 100% persistence): **New questions introduced by memory tool approach**: -1. **API Maturity**: Are memory/context editing APIs production-ready or beta? +1. **API Maturity**: Are memory/context editing APIs stable or experimental? 2. **Access Control**: How to implement multi-tenant access to shared memory? 3. **Encryption**: Does memory tool support encrypted storage of sensitive rules? 4. **Versioning**: Can memory tool track rule evolution over time? diff --git a/docs/markdown/phase-5-session2-summary.md b/docs/markdown/phase-5-session2-summary.md index 998b7a36..b9a17ab8 100644 --- a/docs/markdown/phase-5-session2-summary.md +++ b/docs/markdown/phase-5-session2-summary.md @@ -482,7 +482,7 @@ Results: **If you're interested in Phase 5 PoC**: -**Framework Status**: 100% integrated, production-ready +**Framework Status**: 100% integrated, research implementation **Integration Pattern**: Proven and documented for all service types diff --git a/governance/TRA-OPS-0002-blog-editorial-guidelines-v1-0.md b/governance/TRA-OPS-0002-blog-editorial-guidelines-v1-0.md index 8ec59f56..99d60d04 100644 --- a/governance/TRA-OPS-0002-blog-editorial-guidelines-v1-0.md +++ b/governance/TRA-OPS-0002-blog-editorial-guidelines-v1-0.md @@ -158,7 +158,7 @@ Changes: Rewrote introduction, added 3 examples, verified all citations **Format**: - Problem statement (what are we solving?) - Architecture overview (high-level design) -- Code examples (production-ready) +- Code examples (working, tested) - Testing strategies - Performance considerations diff --git a/governance/TRA-OPS-0003-media-inquiry-response-protocol-v1-0.md b/governance/TRA-OPS-0003-media-inquiry-response-protocol-v1-0.md index 5e4dfb46..d7fb171a 100644 --- a/governance/TRA-OPS-0003-media-inquiry-response-protocol-v1-0.md +++ b/governance/TRA-OPS-0003-media-inquiry-response-protocol-v1-0.md @@ -291,7 +291,7 @@ Hi [Name], Thank you for your interest in integrating the Tractatus Framework into [Company/Product]. -The framework is currently in Phase 2 development (soft launch). We expect production-ready packages in [Timeframe]. +The framework is currently in Phase 2 development (soft launch). We expect stable research packages in [Timeframe]. For early adopters, we offer: - Implementation consultation diff --git a/scripts/analyze-violations.js b/scripts/analyze-violations.js new file mode 100644 index 00000000..c84d5479 --- /dev/null +++ b/scripts/analyze-violations.js @@ -0,0 +1,169 @@ +#!/usr/bin/env node + +/** + * Analyze ProhibitedTermsScanner violations + * Filter to only show violations that need user review + */ + +const { execSync } = require('child_process'); + +// Run scanner and capture output (ignore exit code - scanner exits with 1 if violations found) +let output; +try { + output = execSync('node scripts/framework-components/ProhibitedTermsScanner.js --details', { + encoding: 'utf8' + }); +} catch (err) { + // Scanner exits with code 1 when violations found (for pre-commit hooks) + output = err.stdout; +} + +// Parse violations +const violations = []; +const lines = output.split('\n'); + +for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + + if (line.match(/^ \/home\/theflow\/projects\/tractatus\//)) { + const filePath = line.trim().split(':')[0]; + const lineNum = line.trim().split(':')[1]; + + // Get next 4 lines for context + const rule = lines[i + 1]?.match(/Rule: (inst_\d+)/)?.[1]; + const found = lines[i + 2]?.match(/Found: "(.+)"/)?.[1]; + const context = lines[i + 3]?.match(/Context: (.+)/)?.[1]; + const suggestion = lines[i + 4]?.match(/Suggestion: (.+)/)?.[1]; + + if (rule && found) { + violations.push({ + file: filePath.replace('/home/theflow/projects/tractatus/', ''), + line: lineNum, + rule, + found, + context: context || '', + suggestion: suggestion || '' + }); + } + } +} + +// Filter to only relevant violations +const relevantViolations = violations.filter(v => { + const file = v.file; + + // Public-facing UI + if (file.startsWith('public/')) return true; + + // GitHub repository files + if (file === 'README.md') return true; + if (file === 'CLAUDE.md') return true; + if (file.startsWith('governance/')) return true; + if (file.startsWith('docs/markdown/')) return true; // Published docs + + return false; +}); + +// Categorize by type and file +const byFile = {}; +relevantViolations.forEach(v => { + if (!byFile[v.file]) { + byFile[v.file] = { + inst_017: [], + inst_018: [], + inst_016: [] + }; + } + byFile[v.file][v.rule].push(v); +}); + +// Print summary +console.log('\n═══════════════════════════════════════════════════════════'); +console.log(' VIOLATIONS REQUIRING REVIEW'); +console.log('═══════════════════════════════════════════════════════════\n'); + +console.log(`Total violations in codebase: ${violations.length}`); +console.log(`Violations requiring review: ${relevantViolations.length}`); +console.log(`Files to review: ${Object.keys(byFile).length}\n`); + +console.log('───────────────────────────────────────────────────────────'); +console.log('CATEGORIZED BY FILE AND VIOLATION TYPE'); +console.log('───────────────────────────────────────────────────────────\n'); + +const fileCategories = { + 'Public UI Files': [], + 'GitHub Repo Files': [], + 'Published Documentation': [] +}; + +Object.keys(byFile).forEach(file => { + if (file.startsWith('public/')) { + fileCategories['Public UI Files'].push(file); + } else if (file === 'README.md' || file === 'CLAUDE.md' || file.startsWith('governance/')) { + fileCategories['GitHub Repo Files'].push(file); + } else { + fileCategories['Published Documentation'].push(file); + } +}); + +Object.entries(fileCategories).forEach(([category, files]) => { + if (files.length === 0) return; + + console.log(`\n📁 ${category.toUpperCase()}\n`); + + files.forEach(file => { + const fileCounts = byFile[file]; + const total = fileCounts.inst_017.length + fileCounts.inst_018.length + fileCounts.inst_016.length; + + console.log(` ${file} (${total} violations)`); + + if (fileCounts.inst_017.length > 0) { + console.log(` inst_017 (Absolute Assurance): ${fileCounts.inst_017.length}`); + fileCounts.inst_017.slice(0, 3).forEach(v => { + console.log(` Line ${v.line}: "${v.found}" → ${v.suggestion}`); + }); + if (fileCounts.inst_017.length > 3) { + console.log(` ... and ${fileCounts.inst_017.length - 3} more`); + } + } + + if (fileCounts.inst_018.length > 0) { + console.log(` inst_018 (Unverified Claims): ${fileCounts.inst_018.length}`); + fileCounts.inst_018.slice(0, 3).forEach(v => { + console.log(` Line ${v.line}: "${v.found}" → ${v.suggestion}`); + }); + if (fileCounts.inst_018.length > 3) { + console.log(` ... and ${fileCounts.inst_018.length - 3} more`); + } + } + + if (fileCounts.inst_016.length > 0) { + console.log(` inst_016 (Fabricated Statistics): ${fileCounts.inst_016.length}`); + fileCounts.inst_016.slice(0, 3).forEach(v => { + console.log(` Line ${v.line}: "${v.found}" → ${v.suggestion}`); + }); + if (fileCounts.inst_016.length > 3) { + console.log(` ... and ${fileCounts.inst_016.length - 3} more`); + } + } + + console.log(''); + }); +}); + +console.log('═══════════════════════════════════════════════════════════'); +console.log('SUMMARY BY VIOLATION TYPE'); +console.log('═══════════════════════════════════════════════════════════\n'); + +const totals = { + inst_017: relevantViolations.filter(v => v.rule === 'inst_017').length, + inst_018: relevantViolations.filter(v => v.rule === 'inst_018').length, + inst_016: relevantViolations.filter(v => v.rule === 'inst_016').length +}; + +console.log(`inst_017 (Absolute Assurance - "guarantee", "never fails"): ${totals.inst_017}`); +console.log(`inst_018 (Unverified Claims - "production-ready", "battle-tested"): ${totals.inst_018}`); +console.log(`inst_016 (Fabricated Statistics - percentages without sources): ${totals.inst_016}`); +console.log(`\nTotal: ${totals.inst_017 + totals.inst_018 + totals.inst_016}\n`); + +console.log('═══════════════════════════════════════════════════════════\n');