tractatus/docs/AUTOMATED_SYNC_SETUP.md

# Automated Public Documentation Sync - Setup Guide

**Status**: Implemented
**Last Updated**: 2025-10-09
**Components**: GitHub Actions workflow + security validation

---

## Overview

Automatically syncs approved documentation from the private `tractatus` repository to the public `tractatus-framework` repository with security validation.

**What Gets Synced**:
- `docs/case-studies/*.md` → Public repo
- `docs/research/*.md` → Public repo
- `README.md` → Public repo (if marked safe)

**Security**: All files are scanned for sensitive information before sync. Violations block the sync.

---

## Setup Steps

### 1. Create GitHub Personal Access Token

1. Go to GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic)
2. Click "Generate new token (classic)"
3. Name: `Tractatus Public Sync Token`
4. Expiration: Set appropriate expiration (90 days recommended)
5. Scopes: Check `repo` (Full control of private repositories)
6. Generate token and **copy it immediately** (you won't see it again)

### 2. Add Secret to Private Repository

1. Go to `AgenticGovernance/tractatus` (private repo)
2. Settings → Secrets and variables → Actions
3. Click "New repository secret"
4. Name: `PUBLIC_REPO_TOKEN`
5. Value: Paste the PAT from step 1
6. Click "Add secret"

### 3. (Optional) Set Up Manual Approval Environment

For sensitive changes, you can require manual approval:

1. Go to `AgenticGovernance/tractatus` → Settings → Environments
2. Click "New environment"
3. Name: `public-sync`
4. Check "Required reviewers"
5. Add yourself as a required reviewer
6. Save protection rules

**Note**: The workflow is configured to use this environment, which creates a manual gate.

### 4. Test the Workflow

**Automatic trigger** (on push to main):
```bash
# Make a change to a case study
echo "Test update" >> docs/case-studies/test.md
git add docs/case-studies/test.md
git commit -m "docs: test automated sync"
git push origin main
```

**Manual trigger**:
1. Go to Actions tab in GitHub
2. Select "Sync Documentation to Public Repository"
3. Click "Run workflow"
4. Choose branch: `main`
5. Skip validation: `false` (recommended)
6. Run workflow

### 5. Monitor Workflow

1. Go to Actions tab → Select the workflow run
2. Review validation results
3. If manual approval is configured, approve the deployment
4. Check public repo for synced files

---

## How It Works

### Workflow Trigger

**Automatic** - Triggers on push to main when these paths change:
- `docs/case-studies/**/*.md`
- `docs/research/**/*.md`
- `README.md`

**Manual** - Can be triggered via GitHub Actions UI with option to skip validation (use cautiously)

### Security Validation

**Script**: `scripts/validate-public-sync.js`

**Scans For**:
- Internal file paths (`/home/`, `/var/www/`)
- Database names (`tractatus_dev`, `tractatus_prod`)
- Port numbers (27017, 9000, etc.)
- IP addresses and server hostnames
- Email addresses (except public)
- SSH keys and credentials
- API keys and tokens
- Cross-project references
- Internal documentation markers

**Exit Codes**:
- `0` = PASS (safe to sync)
- `1` = FAIL (security issues found, sync blocked)
- `2` = ERROR (validation system failure)

**Severity Levels**:
- CRITICAL: Immediate block (credentials, SSH keys)
- HIGH: Block sync (paths, database names, IPs)
- MEDIUM: Block sync (ports, emails, service names)
- LOW: Block sync (process management commands)

### Sync Process

1. **Checkout both repos** (private and public)
2. **Install dependencies** in private repo
3. **Run security validation** (unless manually skipped)
4. **Copy approved files** from private to public staging
5. **Commit changes** with automated message
6. **Push to public repo** on main branch
7. **Create sync report** (uploaded as artifact)
8. **Notify on failure** (creates GitHub issue)

---

## README Sync Special Case

The private `README.md` will only sync if it contains this marker:

```markdown
<!-- PUBLIC_REPO_SAFE -->
```

Add this comment at the top of README.md when it's been sanitized for public consumption.

**Without this marker**: README is skipped (remains private-only)
**With this marker**: README is synced to public repo

---

## Troubleshooting

### Sync Blocked - Security Issues Found

**Symptom**: Workflow fails with "Security validation failed - sync blocked"

**Solution**:
1. Download the sync report artifact from the workflow run
2. Review the detected security issues
3. Fix the issues in the source files
4. Push the fixes to main (triggers new sync)

**Common issues**:
- Database names in examples → Use `[DATABASE_NAME]` placeholder
- Internal paths in commands → Use generic paths or `[PATH]`
- Port numbers in configs → Use `[PORT]` placeholder

### Manual Approval Not Triggering

**Symptom**: Workflow doesn't wait for approval

**Solution**:
1. Verify `public-sync` environment exists
2. Check required reviewers are configured
3. Ensure workflow references correct environment name

### Token Expired

**Symptom**: Push to public repo fails with authentication error

**Solution**:
1. Generate new PAT (same permissions)
2. Update `PUBLIC_REPO_TOKEN` secret in repository settings
3. Re-run failed workflow

### Files Not Syncing

**Check**:
1. Are files in the correct directories? (`docs/case-studies/`, `docs/research/`)
2. Are files `.md` format?
3. Does README have `<!-- PUBLIC_REPO_SAFE -->` marker?
4. Check workflow logs for specific errors

---

## Sync Report

After each run, a sync report is generated and uploaded as an artifact:

**Contents**:
- Validation status (pass/fail)
- Number of files synced
- List of changed files
- Any errors encountered

**Access**:
1. Go to workflow run page
2. Scroll to "Artifacts" section
3. Download `sync-report`

**Retention**: 30 days

---

## Security Best Practices

### Before Committing New Docs

**Run local validation**:
```bash
node scripts/validate-public-sync.js
```

This runs the same security checks that GitHub Actions will run.

**Fix issues locally** before pushing to avoid failed workflow runs.

### Marking README as Safe

Only add `<!-- PUBLIC_REPO_SAFE -->` to README.md after:

1. Running security validation
2. Reviewing all content for internal references
3. Confirming no infrastructure details exposed
4. Verifying no cross-project references

### Emergency: Disable Sync

If you need to temporarily disable automatic sync:

**Option 1**: Delete or rename the workflow file
```bash
git mv .github/workflows/sync-public-docs.yml .github/workflows/sync-public-docs.yml.disabled
git commit -m "chore: temporarily disable auto-sync"
git push
```

**Option 2**: Disable the workflow in GitHub
1. Go to Actions tab
2. Select "Sync Documentation to Public Repository"
3. Click "..." → "Disable workflow"

### Revoking Access

If token is compromised:

1. Go to GitHub → Settings → Developer settings → Personal access tokens
2. Find the token → Click "Delete"
3. Generate new token
4. Update `PUBLIC_REPO_TOKEN` secret
5. Old token is immediately invalidated

---

## Maintenance

### Updating Validation Patterns

Edit `scripts/validate-public-sync.js`:

```javascript
const SECURITY_PATTERNS = [
  {
    pattern: /your-new-pattern/gi,
    severity: 'HIGH',
    description: 'What this pattern detects',
    category: 'Pattern Category'
  },
  // ... existing patterns
];
```

Test changes locally:
```bash
node scripts/validate-public-sync.js
```

### Updating Workflow

Edit `.github/workflows/sync-public-docs.yml`

**Common changes**:
- Add new file paths to sync
- Modify trigger conditions
- Update Node.js version
- Adjust manual approval requirements

---

## Files

**GitHub Actions**:
- `.github/workflows/sync-public-docs.yml` - Main workflow

**Scripts**:
- `scripts/validate-public-sync.js` - Security validation

**Documentation**:
- `docs/AUTOMATED_SYNC_SETUP.md` - This file

**Generated**:
- `sync-report.md` - Created during each run (artifact only)

---

## Support

**Workflow failures**: Check Actions tab → View logs
**Security validation**: Review sync report artifact
**Token issues**: Regenerate PAT and update secret
**Questions**: See GitHub Actions documentation

---

**Last validated**: 2025-10-09
**Validation script version**: 1.0
**Workflow version**: 1.0