209a3a291a
SUMMARY: Implemented 6 framework refinements identified from incident analysis (inst_049 and inst_025 violations). These enhancements provide architectural enforcement for patterns that previously relied on voluntary compliance. ENHANCEMENTS IMPLEMENTED: 1. Instruction Analytics Script (Priority 8) - scripts/analyze-instruction-violations.js - Analyzes instruction-history.json for usage patterns - Identifies most violated instructions - Calculates enforcement effectiveness (hook vs. voluntary) - Shows 97.2% voluntary compliance, 75% hook enforcement - Recommendations for converting voluntary → architectural 2. Framework Incidents Database (Priority 7) - .claude/framework-incidents.json - Structured tracking of framework violations - INC-001: Ignored user hypothesis (70k tokens wasted) - INC-002: Deployment directory flattening (inst_025 violation) - Statistics: 2 incidents, 75k tokens wasted, 4.5 hours lost 3. Loop Detector Module (Priorities 3 & 4) - scripts/framework-components/LoopDetector.js - Detects "stuck in loop" patterns - Triggers: 3+ edits to same file, repeated action types - Feeds into MetacognitiveVerifier and ContextPressureMonitor - Calculates pressure contribution (5-40 points by severity) 4. Action Pattern Tracker (Priority 3 & 4) - scripts/track-action-patterns.js - Tracks edit/write actions to detect repetition - Alerts after 3 consecutive edits to same file - Maintains action history (last 100 actions) - Recommendations for metacognitive verification 5. Pre-Deployment Validation (Priority 5) - scripts/validate-deployment.js - Validates rsync/scp commands against inst_025 - Detects directory structure flattening - Suggests separate commands for different directories - Prevents 4th documented occurrence of deployment errors 6. User Suggestion Tracker (Priority 6) - scripts/track-user-suggestions.js - Implements inst_049: "Test user hypothesis first" - Tracks user technical hypotheses - Flags untested hypotheses as HIGH priority - Integrates with MetacognitiveVerifier for compliance USAGE: Instruction Analytics: node scripts/analyze-instruction-violations.js Loop Detection: node scripts/track-action-patterns.js --check node scripts/track-action-patterns.js --summary Deployment Validation: node scripts/validate-deployment.js --command "rsync ..." User Suggestions: node scripts/track-user-suggestions.js --add "hypothesis text" node scripts/track-user-suggestions.js --check-untested IMPACT: - Converts 6 voluntary compliance patterns to architectural enforcement - Prevents repeat of documented 75k token waste - Provides visibility into framework effectiveness - Establishes foundation for future hook integration METRICS FROM ANALYTICS: - Active Instructions: 40 - Voluntary Compliance: 97.2% - Hook Enforcement: 75.0% - Recorded Violations: 2 - Tokens Wasted: 75,000 NEXT STEPS: - Integrate LoopDetector into MetacognitiveVerifier.service.js - Add Pre-Deployment Validation to Bash command validator hook - Wire User Suggestion Tracker into BoundaryEnforcer checks - Document successful compliance patterns (7 STRATEGIC instructions at 100%) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
68 lines
2.7 KiB
JSON
68 lines
2.7 KiB
JSON
{
|
|
"version": "1.0",
|
|
"last_updated": "2025-10-20T20:00:00Z",
|
|
"description": "Framework incident tracking database for Tractatus governance violations",
|
|
"incidents": [
|
|
{
|
|
"id": "INC-001",
|
|
"date": "2025-10-20",
|
|
"type": "ignored_user_hypothesis",
|
|
"title": "Ignored User Technical Hypothesis - Tailwind Issue",
|
|
"instruction_violated": "inst_049",
|
|
"severity": "HIGH",
|
|
"components_failed": ["MetacognitiveVerifier", "BoundaryEnforcer", "CrossReferenceValidator"],
|
|
"tokens_wasted": 70000,
|
|
"time_wasted_hours": 4,
|
|
"description": "AI pursued 12 failed debugging attempts instead of testing user's correctly-identified 'Tailwind issue' hypothesis first",
|
|
"resolution": "inst_049 added - 'Test user hypothesis first before alternatives'",
|
|
"lessons": [
|
|
"Test user hypothesis first before pursuing alternatives",
|
|
"After 3 failed attempts, pause and metacognitively verify approach",
|
|
"Pattern override bias can persist across many attempts",
|
|
"User technical expertise must be respected architecturally"
|
|
],
|
|
"prevented_by": "architectural_enforcement",
|
|
"document_reference": "FRAMEWORK_INCIDENT_2025-10-20_IGNORED_USER_HYPOTHESIS.md"
|
|
},
|
|
{
|
|
"id": "INC-002",
|
|
"date": "2025-10-20",
|
|
"type": "pattern_override_bias",
|
|
"title": "Deployment Directory Structure Flattened",
|
|
"instruction_violated": "inst_025",
|
|
"severity": "HIGH",
|
|
"components_failed": ["CrossReferenceValidator", "BashCommandValidator"],
|
|
"tokens_wasted": 5000,
|
|
"time_wasted_hours": 0.5,
|
|
"description": "Used single rsync command for files with different subdirectories, flattening structure. Fourth occurrence of this failure.",
|
|
"resolution": "Added Bash command validator hook to architecturally prevent pattern",
|
|
"lessons": [
|
|
"Training patterns ('efficient single rsync') override explicit instructions",
|
|
"Voluntary compliance fails for pattern override bias",
|
|
"Bash tool needs validator hook like Edit/Write tools",
|
|
"Repeated failures indicate enforcement gap, not education gap"
|
|
],
|
|
"prevented_by": "bash_command_validator_hook",
|
|
"document_reference": "FRAMEWORK_VIOLATION_2025-10-20_INST_025_DEPLOYMENT.md"
|
|
}
|
|
],
|
|
"statistics": {
|
|
"total_incidents": 2,
|
|
"total_tokens_wasted": 75000,
|
|
"total_time_wasted_hours": 4.5,
|
|
"by_severity": {
|
|
"CRITICAL": 0,
|
|
"HIGH": 2,
|
|
"MEDIUM": 0,
|
|
"LOW": 0
|
|
},
|
|
"by_type": {
|
|
"ignored_user_hypothesis": 1,
|
|
"pattern_override_bias": 1
|
|
},
|
|
"most_violated_instructions": [
|
|
{"id": "inst_025", "count": 1},
|
|
{"id": "inst_049", "count": 1}
|
|
]
|
|
}
|
|
}
|