tractatus/docs/SESSION_INIT_API_MEMORY_AUDIT.md

# Session Init API Memory Audit & Optimization Plan

**Date**: 2025-10-10
**Context**: Preparing for first session with Anthropic API Memory system
**Script**: `scripts/session-init.js`

---

## Current Implementation Analysis

### What session-init.js Does Now

1. **Checks Session Status** (lines 66-85)
   - Reads `.claude/session-state.json` to detect new vs. continued session
   - Uses file timestamp and message count to determine status
   - **File-dependent**: Requires file to exist for detection

2. **Initializes Session State** (lines 90-144)
   - Creates/overwrites `.claude/session-state.json`
   - Sets up framework activity tracking
   - **File-based**: All state stored in JSON file

3. **Resets Token Checkpoints** (lines 149-165)
   - Creates/overwrites `.claude/token-checkpoints.json`
   - Sets 25%, 50%, 75% milestones
   - **File-based**: Checkpoint state in JSON

4. **Loads Instruction History** (lines 170-193)
   - Reads `.claude/instruction-history.json` (6k+ tokens)
   - Counts active instructions by persistence/quadrant
   - **File-dependent**: Requires full file read

5. **Runs Pressure Check** (lines 198-218)
   - Executes `check-session-pressure.js` as subprocess
   - Parses output to extract pressure level/score
   - **Independent**: Should work with API Memory

6. **Runs Framework Tests** (lines 288-323)
   - Executes unit tests for 5 framework components
   - Reports pass/fail status
   - **Independent**: Should work with API Memory

---

## API Memory System Implications

### What API Memory SHOULD Provide

1. **Automatic Instruction Persistence**
   - 19 instructions might be available without reading file
   - Framework context might persist across sessions
   - Session state might be remembered

2. **Session Continuity Detection**
   - API Memory knows if this is a continuation
   - Can detect compaction events
   - May preserve session metadata

3. **Reduced File Dependency**
   - Less need to read large JSON files
   - State could be queried from Memory API instead
   - Faster initialization

### What API Memory CANNOT Replace

1. **Local File State** (still needed)
   - Other tools may read these files
   - Backward compatibility
   - Debugging and auditing

2. **Explicit Framework Initialization**
   - Components still need setup
   - Tests still need to run
   - Checkpoints still need tracking

---

## Optimization Recommendations

### Priority 1: API Memory Detection (IMMEDIATE)

**Add at session start:**
```javascript
/**
 * Detect if API Memory system is active
 * This is indicated by system providing context without file reads
 */
function detectAPIMemory() {
  // Check for API Memory system hints
  // (Implementation depends on Anthropic's API Memory design)

  // For now, we can infer from environment or test query
  const hasAPIMemory = process.env.ANTHROPIC_API_MEMORY === 'true' ||
                       process.env.CLAUDE_CODE_VERSION?.includes('memory');

  return {
    available: hasAPIMemory,
    detected: hasAPIMemory ? 'API Memory system detected' : 'File-based system (legacy)'
  };
}
```

**Benefits**:
- Inform user if new system is active
- Adjust behavior based on memory availability
- Collect data on memory system effectiveness

### Priority 2: Conditional File Reads (MEDIUM)

**Optimize instruction loading:**
```javascript
function loadInstructionHistory() {
  // If API Memory available, try lightweight query first
  if (apiMemoryAvailable) {
    try {
      // Query memory for instruction count without full file read
      const memoryStats = queryAPIMemoryStats();
      if (memoryStats.instructionsLoaded) {
        log('  Instructions loaded via API Memory', 'green');
        return memoryStats;
      }
    } catch (err) {
      // Fall back to file read
    }
  }

  // Original file-based implementation
  const history = JSON.parse(fs.readFileSync(INSTRUCTION_HISTORY_PATH, 'utf8'));
  // ... rest of function
}
```

**Benefits**:
- Reduces 6k token file read if Memory has instructions
- Faster initialization
- Still works if Memory system unavailable

### Priority 3: Session Continuity Intelligence (MEDIUM)

**Enhanced session detection:**
```javascript
function isNewSession() {
  // Check API Memory first
  if (apiMemoryAvailable) {
    try {
      const memorySession = queryAPIMemorySession();
      if (memorySession.sessionId) {
        return {
          isNew: false,
          isToday: true,
          sessionState: memorySession,
          source: 'api_memory'
        };
      }
    } catch (err) {
      // Fall back to file
    }
  }

  // Original file-based detection
  // ... existing implementation
}
```

**Benefits**:
- Better post-compaction detection
- Smoother session continuation
- Reduced file dependency

### Priority 4: API Memory Instrumentation (LOW - DATA COLLECTION)

**Add tracking for next session:**
```javascript
function reportAPIMemoryStats() {
  if (!apiMemoryAvailable) return;

  section('API Memory System Status');
  success('API Memory: ACTIVE (Experimental)');

  try {
    const stats = {
      instructionsPersisted: queryInstructionPersistence(),
      sessionContextPreserved: querySessionContext(),
      compactionHandled: queryCompactionStatus()
    };

    log(`  Instructions persisted: ${stats.instructionsPersisted}`, 'cyan');
    log(`  Session context: ${stats.sessionContextPreserved}`, 'cyan');
    log(`  Post-compaction: ${stats.compactionHandled}`, 'cyan');
  } catch (err) {
    warning(`Could not query API Memory stats: ${err.message}`);
  }
}
```

**Benefits**:
- Understand how API Memory works
- Collect data for future optimization
- Identify gaps in Memory system

---

## Implementation Strategy

### Phase 1: Detection (Next Session - 30 mins)
1. Add `detectAPIMemory()` function
2. Report API Memory status in output
3. Set flag for conditional behavior
4. **No breaking changes** - purely informational

### Phase 2: Optimization (Future Session - 2 hours)
1. Implement conditional file reads
2. Add Memory-based session detection
3. Reduce redundant operations
4. **Maintain backward compatibility**

### Phase 3: Instrumentation (Future Session - 1 hour)
1. Add Memory system metrics
2. Track effectiveness
3. Identify optimization opportunities
4. **Data collection for refinement**

---

## Testing Plan

### Next Session Tests

1. **Does API Memory preserve instructions?**
   - Test: Start session, check if 19 instructions accessible without file read
   - Expected: Yes (instructions available via Memory)
   - Fallback: No (still reads from file)

2. **Does API Memory detect continuation?**
   - Test: After compaction, does Memory know previous session?
   - Expected: Yes (session context preserved)
   - Fallback: No (treats as new session)

3. **Does API Memory reduce file operations?**
   - Test: Count file reads in session-init.js
   - Baseline: 3 file reads (session-state, instruction-history, token-checkpoints)
   - Expected: 1-2 file reads (some via Memory)

4. **Does API Memory extend session length?**
   - Test: How many messages before compaction?
   - Baseline: 20-40 messages (file-heavy work)
   - Expected: 30-60 messages (better context management)

---

## Risks & Mitigations

### Risk 1: API Memory Not Yet Active
**Impact**: Script behavior unchanged, no benefit
**Mitigation**: Script still works exactly as before (backward compatible)

### Risk 2: API Memory Incomplete
**Impact**: Some features work, others don't
**Mitigation**: Fallback to file-based system for missing features

### Risk 3: API Memory Overhead
**Impact**: Querying Memory adds latency
**Mitigation**: Timeout on Memory queries, fall back to files

### Risk 4: Breaking Changes
**Impact**: Script fails in new environment
**Mitigation**: Extensive testing, graceful degradation

---

## Recommendations for Next Session

### Immediate Actions
1. ✅ **Run session-init.js as-is** - Don't modify yet
2. ✅ **Observe API Memory behavior** - Note what changes
3. ✅ **Document findings** - What works, what doesn't
4. ⚠️ **Don't optimize prematurely** - Understand system first

### After Understanding API Memory
1. Implement Phase 1 detection (30 mins)
2. Test with file-heavy operations
3. Measure session length improvement
4. Report findings to inform Phase 2

### Long-Term
- Phase 2 optimization (2 hours) - Conditional file reads
- Phase 3 instrumentation (1 hour) - Metrics collection
- Phase 4 enhancement (4 hours) - Full Memory integration (see inst_019)

---

## Current Status

**session-init.js**: ✅ **READY FOR API MEMORY**
- No breaking changes needed
- Will work as-is with or without Memory
- Optimization can be added incrementally
- Maintains backward compatibility

**Next Session Priority**: **OBSERVE & DOCUMENT** API Memory behavior before optimizing.

---

**Prepared by**: Claude (claude-sonnet-4-5-20250929)
**Session**: 2025-10-10-api-memory-transition
**Related**: inst_019 (context pressure monitoring), Phase 3.5 (concurrent sessions)