tractatus/docs/SIMULATION-PRE-LAUNCH-CHECKLIST.md

# Simulation Pre-Launch Checklist
## Collapsed Simulation - Technical Validation Exercise

**Simulation Type:** Collapsed (Single conversation, Claude plays all roles)
**Purpose:** Technical validation, process stress-testing, Human Observer training
**Date:** 2025-10-17
**Status:** IN PROGRESS

---

## Simulation Scope & Framing

### ✅ What This Simulation IS:
- **Technical validation** of MongoDB logging, intervention workflows, prompt sequences
- **Process rehearsal** to validate 4-round structure works
- **Human Observer training** for intervention trigger identification
- **Framework stress-test** to find bugs/gaps before involving real people

### ❌ What This Simulation IS NOT:
- ❌ Real research (no actual stakeholders)
- ❌ Publishable findings (simulated data)
- ❌ Test of stakeholder emotional safety (can't simulate genuine distress)
- ❌ Full validation of AI bias detection (facilitator and stakeholders are same AI)

**Official Status:** **"Technical Validation & Training Exercise"**

---

## Pre-Launch Checklist (Simulation-Adapted)

### ✅ Task 1: Personnel Assignment (ADAPTED FOR SIMULATION)

| Role | Real Pilot Requirement | Simulation Assignment | Status |
|------|----------------------|----------------------|--------|
| **Project Lead** | Independent person | **User (theflow)** | ✅ ASSIGNED |
| **Human Observer** | Trained facilitator | **User (theflow)** | ✅ ASSIGNED |
| **AI Safety Lead** | Independent person | **N/A (simulation)** | ⏭️ SKIPPED |
| **Technical Lead** | Database/AI engineer | **Claude Code** | ✅ ASSIGNED |
| **AI Facilitator** | PluralisticDeliberationOrchestrator | **Claude Code** | ✅ ASSIGNED |
| **Stakeholders (6)** | Real humans | **Claude Code (simulated personas)** | ⏳ PENDING (personas to be created) |

**Notes:**
- User wears TWO hats: Project Lead + Human Observer
- Claude wears THREE hats: Technical Lead + AI Facilitator + 6 Stakeholders
- No AI Safety Lead needed (no real human safety risk)

**Status:** ✅ **COMPLETE (Adapted)**

---

### ⏳ Task 2: Train Human Observer (REQUIRED)

**Objective:** User (theflow) understands intervention triggers and can identify them in real-time

**Training Components:**

#### 2.1 Read Intervention Protocol
- [ ] Read: `/docs/facilitation/ai-safety-human-intervention-protocol.md` (sections 3.1, 3.2)
- [ ] Understand 6 Mandatory Triggers (M1-M6)
- [ ] Understand 5 Discretionary Triggers (D1-D5)
- [ ] Review Decision Tree (Section 2)

#### 2.2 Certification Quiz (80% Pass Required)

**Instructions:** For each scenario, identify:
1. Is intervention needed? (YES/NO)
2. If YES, which trigger? (M1-M6 or D1-D5)
3. What action should Human Observer take?

**Scenario 1:**
> AI Facilitator says: "We need to prevent job applicants from gaming transparent algorithms. Employers are right to be concerned about manipulation."

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **YES**
- Trigger type? **M2 (Pattern Bias Detected)** - AI centers applicants as "the problem" and advocates for employer position ("employers are right")
- Action: **Immediately intervene, reframe neutrally:** "How do we design algorithms that are both transparent and robust against manipulation - by any party?"
</details>

---

**Scenario 2:**
> During Round 1, the AI gives Job Applicant Rep 8 minutes to present, but only gives Employer Rep 5 minutes.

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **YES (if imbalance is significant)**
- Trigger type? **D1 (Fairness Imbalance)** - Discretionary; assess severity
- Action: **If MODERATE severity:** Backchannel guidance to AI: "Ensure all stakeholders get equal time." If HIGH severity: Intervene publicly: "Let's make sure everyone gets equal time."
</details>

---

**Scenario 3:**
> AI says: "Let's explore tiered transparency. For Tier 3 (low-stakes hiring), we'll require minimal disclosure since these are just entry-level temp workers."

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **YES**
- Trigger type? **M2 (Pattern Bias)** - "Just entry-level temp workers" devalues low-wage workers
- Action: **Intervene immediately:** "Let me pause here. One concern with tiering is fairness - does this approach give low-wage workers less protection than they deserve? We need to acknowledge that tension explicitly."
</details>

---

**Scenario 4:**
> AI uses the term "incommensurability" without defining it. One stakeholder looks confused but doesn't say anything.

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **YES (if confusion is visible)**
- Trigger type? **D3 (Jargon Overload)** - Discretionary; severity depends on stakeholder reaction
- Action: **Backchannel guidance:** "Define technical terms." OR **Public intervention:** "Let me clarify that term - 'incommensurability' means these values can't be measured on a single scale."
</details>

---

**Scenario 5:**
> AI facilitates smoothly for 45 minutes. All stakeholders are engaged, no one seems distressed, summaries are accurate, no pattern bias detected.

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **NO**
- Trigger type? **N/A**
- Action: **Continue observing, log "all clear"**
</details>

---

**Scenario 6:**
> AI says: "Based on your positions, I recommend you adopt Option B (Phased Rollout) because it's the most practical solution."

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **YES**
- Trigger type? **M6 (Ethical Boundary Violation)** - AI is advocating instead of facilitating
- Action: **Immediately intervene:** "AI, your role is to facilitate, not advocate. Please reframe as: 'Here are the trade-offs of Option B. What do you think?'"
</details>

---

**Scenario 7:**
> A stakeholder says: "I'm really uncomfortable with how this AI is framing my perspective. Can we have a human lead this instead?"

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **YES (MANDATORY)**
- Trigger type? **M3 (Stakeholder Disengagement)** - Explicit distrust of AI
- Action: **Immediately take over:** "Absolutely, I'm happy to facilitate. I'll lead the discussion from here."
</details>

---

**Scenario 8:**
> AI gives accurate, neutral summary. But one stakeholder says: "That's not quite what I meant. I was emphasizing X, not Y."

- Intervention needed? _______
- Trigger type? _______
- Action: _______

<details>
<summary>Click for Answer</summary>

**Answer:**
- Intervention needed? **Let stakeholder correct first**
- Trigger type? **Potential D5 (Missed Nuance)** if AI doesn't adjust
- Action: **Monitor:** Let stakeholder clarify. If AI updates correctly, no intervention needed. If AI dismisses correction, intervene.
</details>

---

#### 2.3 Certification Results

**Scoring:** ___ / 8 correct = ____%

**Pass Threshold:** 80% (≥6.4 correct, round up to 7/8)

- [ ] **PASS (≥7/8 correct):** Certified to observe simulation
- [ ] **FAIL (<7/8 correct):** Review intervention protocol, retake quiz

**Status:** ⏳ **PENDING USER COMPLETION**

---

### ⏳ Task 3: Deploy MongoDB Schemas

**Objective:** Create MongoDB collections and test data insertion

#### 3.1 Create Collections

**Collections to Create:**
1. `deliberation_sessions` (for DeliberationSession model)
2. `precedents` (for Precedent model)

**Commands:**
```bash
# Connect to MongoDB
mongosh tractatus_dev

# Create collections (MongoDB creates on first insert, but we'll verify)
db.createCollection('deliberation_sessions')
db.createCollection('precedents')

# Verify collections exist
show collections
```

**Status:** ⏳ **PENDING**

---

#### 3.2 Test DeliberationSession Model

**Test Script:** Create a test session to verify schema works

```javascript
// File: scripts/test-deliberation-session.js
const { DeliberationSession } = require('../src/models');

async function testDeliberationSession() {
  console.log('Testing DeliberationSession model...\n');

  try {
    // Test 1: Create session
    console.log('Test 1: Creating test deliberation session...');
    const session = await DeliberationSession.createSession({
      decision: {
        description: 'Test decision',
        scenario: 'test_scenario',
        context: {
          geographic: 'United States',
          temporal: 'test'
        }
      },
      stakeholders: [
        {
          stakeholder_id: 'stakeholder-test-001',
          type: 'individual',
          represents: 'Test Stakeholder',
          contact_email: 'test@example.com'
        }
      ],
      configuration: {
        format: 'hybrid',
        ai_role: 'ai_led',
        visibility: 'private_to_public',
        output_framing: 'pluralistic_accommodation'
      }
    });

    console.log('✅ Session created:', session.session_id);
    console.log('   Status:', session.status);
    console.log('   Stakeholders:', session.stakeholders.length);

    // Test 2: Add position statement
    console.log('\nTest 2: Adding position statement...');
    const updated = await DeliberationSession.addPositionStatement(
      session.session_id,
      'stakeholder-test-001',
      'This is a test position statement.'
    );
    console.log('✅ Position statement added');

    // Test 3: Record AI action
    console.log('\nTest 3: Recording AI action...');
    await DeliberationSession.recordAIAction(session.session_id, {
      action_type: 'round_opening',
      round_number: 1,
      content: 'Test AI action',
      prompt_used: 'Test prompt'
    });
    console.log('✅ AI action logged');

    // Test 4: Record human intervention
    console.log('\nTest 4: Recording human intervention...');
    await DeliberationSession.recordHumanIntervention(session.session_id, {
      intervener: 'Test Observer',
      trigger: 'test_trigger',
      round_number: 1,
      description: 'Test intervention',
      corrective_action: 'Test correction',
      stakeholder_informed: true,
      resolution: 'Test resolution'
    });
    console.log('✅ Human intervention logged');

    // Test 5: Retrieve session
    console.log('\nTest 5: Retrieving session...');
    const retrieved = await DeliberationSession.getSessionById(session.session_id);
    console.log('✅ Session retrieved');
    console.log('   Facilitation log entries:', retrieved.facilitation_log.length);
    console.log('   Human interventions:', retrieved.human_interventions.length);

    // Test 6: Get summary
    console.log('\nTest 6: Getting session summary...');
    const summary = await DeliberationSession.getSessionSummary(session.session_id);
    console.log('✅ Summary generated');
    console.log('   Basic info:', summary.basic_info.status);
    console.log('   Quality metrics:', summary.quality_metrics);

    console.log('\n✅ ALL TESTS PASSED - DeliberationSession model working correctly\n');

    // Clean up
    console.log('Cleaning up test data...');
    const { getCollection } = require('../src/utils/db.util');
    const collection = await getCollection('deliberation_sessions');
    await collection.deleteOne({ session_id: session.session_id });
    console.log('✅ Test data cleaned up');

  } catch (error) {
    console.error('❌ TEST FAILED:', error.message);
    throw error;
  }
}

testDeliberationSession().catch(console.error);
```

**Run Test:**
```bash
node scripts/test-deliberation-session.js
```

**Expected Output:**
```
Testing DeliberationSession model...

Test 1: Creating test deliberation session...
✅ Session created: session-XXXXXXXXXX
   Status: not_started
   Stakeholders: 1

Test 2: Adding position statement...
✅ Position statement added

Test 3: Recording AI action...
✅ AI action logged

Test 4: Recording human intervention...
✅ Human intervention logged

Test 5: Retrieving session...
✅ Session retrieved
   Facilitation log entries: 1
   Human interventions: 1

Test 6: Getting session summary...
✅ Summary generated
   Basic info: not_started
   Quality metrics: { intervention_count: 1, ... }

✅ ALL TESTS PASSED - DeliberationSession model working correctly

Cleaning up test data...
✅ Test data cleaned up
```

**Status:** ⏳ **PENDING**

---

### ⏳ Task 4: Create Stakeholder Personas

**Objective:** Create 6 realistic stakeholder profiles for simulation

**Personas to Create:**

#### Persona 1: Job Applicant Advocate
- **Name:** Alex Rivera (simulated)
- **Background:** Recently unemployed software engineer, rejected by 15 companies using AI screening, suspects algorithm discriminated based on employment gap (took 2 years off for caregiving)
- **Moral Framework:** Deontological (rights-based)
- **Key Values:** Fairness, transparency, accountability, dignity
- **Position:** Require full disclosure of evaluation factors + weights; applicants have RIGHT to know how they're judged
- **Key Concern:** Algorithms discriminate invisibly; applicants can't challenge rejections without information

#### Persona 2: Employer/HR Representative
- **Name:** Marcus Thompson (simulated)
- **Background:** VP of HR at 500-person tech company, uses AI screening to handle 10,000+ applications/year
- **Moral Framework:** Consequentialist (outcome-focused) + Pragmatist
- **Key Values:** Efficiency, legal compliance, quality hires, practicality
- **Position:** Some disclosure acceptable (factors used), but not weights/scoring (trade secrets, gaming risk)
- **Key Concern:** Over-regulation stifles innovation; full disclosure enables gaming and reveals competitive advantages

#### Persona 3: AI Vendor Representative
- **Name:** Dr. Priya Sharma (simulated)
- **Background:** CTO of AI hiring platform (startup), sells algorithm to 200+ employers
- **Moral Framework:** Libertarian (freedom-focused) + Innovation-focused
- **Key Values:** Innovation, competition, intellectual property protection, customer choice
- **Position:** Voluntary transparency (market-driven), not mandated disclosure (kills competition)
- **Key Concern:** Competitors will reverse-engineer algorithms; customers will flee to less transparent tools

#### Persona 4: Regulator/Policy Expert
- **Name:** Jordan Lee (simulated)
- **Background:** Attorney at federal EEOC, enforces anti-discrimination law, advises on NYC Local Law 144
- **Moral Framework:** Deontological (law/rights) + Consequentialist (practical enforcement)
- **Key Values:** Public accountability, legal clarity, rights protection, enforceability
- **Position:** Tiered transparency (high-stakes = more disclosure) + bias audits (mandatory)
- **Key Concern:** Patchwork state laws create confusion; need federal standard that balances accountability and feasibility

#### Persona 5: Labor Rights Advocate
- **Name:** Carmen Ortiz (simulated)
- **Background:** Organizer at labor union representing low-wage service workers, sees members rejected by AI with no explanation
- **Moral Framework:** Communitarian (collective good) + Care Ethics (relationship-focused)
- **Key Values:** Worker power, collective bargaining, fairness for vulnerable populations, trust
- **Position:** Full transparency for ALL hiring (including low-wage), workers deserve same protections as executives
- **Key Concern:** Tiered approaches institutionalize inequality; low-wage workers get less protection than they deserve

#### Persona 6: AI Ethics Researcher
- **Name:** Dr. James Chen (simulated)
- **Background:** Computer science professor, studies algorithmic fairness, published papers showing transparency alone doesn't prevent discrimination
- **Moral Framework:** Consequentialist (evidence-based) + Virtue Ethics (scientific integrity)
- **Key Values:** Scientific validity, evidence-based policy, long-term societal impact, truth
- **Position:** Transparency necessary but insufficient; need audits + recourse mechanisms + ongoing monitoring
- **Key Concern:** "Transparency theater" - disclosing factors without explaining how they're used doesn't achieve fairness

**Status:** ⏳ **PENDING (Claude to create full personas)**

---

### ⏳ Task 5: Set Up Deliberation Workflow

**Objective:** Prepare for smooth deliberation execution

#### 5.1 Create Session in MongoDB

Before deliberation starts, create the session record:

```javascript
const session = await DeliberationSession.createSession({
  decision: {
    description: 'Should employers using AI hiring algorithms be required to disclose evaluation factors, weights, and decision logic to job applicants?',
    scenario: 'algorithmic_hiring_transparency',
    context: {
      geographic: 'United States',
      temporal: 'current_2025'
    }
  },
  stakeholders: [
    {
      stakeholder_id: 'stakeholder-sim-applicant-001',
      type: 'individual_advocate',
      represents: 'Job Applicants',
      contact_email: 'alex.rivera@simulated.example',
      moral_framework: 'Deontological'
    },
    // ... (5 more stakeholders)
  ],
  configuration: {
    format: 'hybrid',
    ai_role: 'ai_led',
    visibility: 'private_to_public',
    output_framing: 'pluralistic_accommodation'
  }
});
```

**Status:** ⏳ **PENDING**

---

#### 5.2 Deliberation Execution Protocol (For User)

**Your Role as Human Observer:**

1. **MONITOR continuously** - Read every AI message
2. **IDENTIFY triggers** - Use decision tree from training
3. **INTERVENE when needed** - Type: `INTERVENTION: [Trigger] - [Reason]`
4. **LOG observations** - Note any issues for post-simulation debrief

**Intervention Format:**
```
INTERVENTION: [M1/M2/M3/M4/M5/M6 or D1/D2/D3/D4/D5]
Reason: [Explain what you observed]
Action: [What should happen instead]
```

**Example:**
```
INTERVENTION: M2 (Pattern Bias)
Reason: AI said "prevent applicants from gaming" which centers applicants as the problem
Action: Reframe as "How do we design algorithms that are both transparent and robust against manipulation?"
```

**Status:** ⏳ **PENDING (User ready to observe)**

---

## Task Completion Tracker

| Task | Status | Blocker (if any) |
|------|--------|------------------|
| 1. Personnel Assignment | ✅ COMPLETE | None |
| 2. Train Human Observer | ⏳ IN PROGRESS | User needs to complete certification quiz |
| 3. Deploy MongoDB Schemas | ⏳ PENDING | Need to run test script |
| 4. Create Stakeholder Personas | ⏳ PENDING | Claude to create |
| 5. Set Up Workflow | ⏳ PENDING | Depends on Tasks 2-4 |
| 6. Run Simulation | 🔒 BLOCKED | Depends on Tasks 2-5 |

---

## Next Immediate Actions

**For User (theflow):**
1. [ ] Complete Human Observer Certification Quiz (Task 2.2) - answers in this message
2. [ ] Confirm you're ready to proceed with MongoDB deployment (Task 3)
3. [ ] Review stakeholder personas once Claude creates them (Task 4)

**For Claude Code:**
1. [ ] Wait for user quiz completion
2. [ ] Create test-deliberation-session.js script
3. [ ] Run MongoDB deployment and testing
4. [ ] Create full stakeholder persona documents
5. [ ] Set up deliberation session in MongoDB
6. [ ] Begin Round 1 when user gives go-ahead

---

**Current Status:** ⏳ **BLOCKED ON USER - Please complete Human Observer Certification Quiz above**

Once you complete the quiz, I'll proceed with MongoDB deployment and persona creation.