From 69023be7133028b0954f576de9035598fa55783c Mon Sep 17 00:00:00 2001 From: TheFlow Date: Sun, 12 Oct 2025 11:02:26 +1300 Subject: [PATCH] feat: add comprehensive governance services API documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 12 Progress - Governance Services Documentation Complete: Added comprehensive documentation for all 6 governance services: 1. InstructionPersistenceClassifier - POST /api/governance/classify - Quadrant classification (STRATEGIC/OPERATIONAL/TACTICAL/SYSTEM) - Persistence levels (HIGH/MEDIUM/LOW) - Request/response examples with quadrant types table 2. CrossReferenceValidator - POST /api/governance/validate - Prevents training pattern override (27027 failure mode) - Validation status types (APPROVED/REJECTED/WARNING) - Conflict detection and recommendations 3. BoundaryEnforcer - POST /api/governance/enforce - Values decision blocking (privacy, ethics, sovereignty) - Boundary categories and alternatives - Human approval requirements 4. ContextPressureMonitor - POST /api/governance/pressure - Multi-factor session health tracking - Pressure levels (NORMAL/ELEVATED/HIGH/CRITICAL/DANGEROUS) - Checkpoint recommendations 5. MetacognitiveVerifier - POST /api/governance/verify - AI self-checks for complex operations - Scope creep detection - Confidence scoring and alternatives 6. AuditLogger - GET /api/audit/audit-logs (with filtering) - GET /api/audit/audit-analytics - Comprehensive audit trail - Aggregated analytics Changes: - api-reference.html: 17KB → 37KB (880 lines) - Updated sidebar navigation with governance services - All endpoints documented with request/response examples - Status types, decision categories, and lookup tables included Future Work: - docs/plans/TRANSLATION_APPROACH.md: DeepL translation strategy for Task 19 - Remaining: OpenAPI spec, code examples (JS/Python), rate limiting docs Deployed to: https://agenticgovernance.digital/api-reference.html --- docs/plans/TRANSLATION_APPROACH.md | 233 ++++++++++++++ public/api-reference.html | 487 ++++++++++++++++++++++++++++- 2 files changed, 713 insertions(+), 7 deletions(-) create mode 100644 docs/plans/TRANSLATION_APPROACH.md diff --git a/docs/plans/TRANSLATION_APPROACH.md b/docs/plans/TRANSLATION_APPROACH.md new file mode 100644 index 00000000..1af102ea --- /dev/null +++ b/docs/plans/TRANSLATION_APPROACH.md @@ -0,0 +1,233 @@ +# Translation Approach for Task 19: Te Reo Māori & Multilanguage Support + +**Date:** October 12, 2025 +**Status:** Planning - To be implemented after Task 12 (API Documentation) +**Priority:** CRITICAL (Values - Te Tiriti commitment) + +--- + +## Translation Service: DeepL + +**Decision:** Use DeepL API for translation services instead of i18next manual translations + +### Deployment Architecture + +**Local Development (Port 9000):** +- DeepL API integration in Node.js +- Local environment variable: `DEEPL_API_KEY` +- Translation caching in MongoDB + +**Production Server (vps-93a693da.vps.ovh.net):** +- DeepL API integration deployed +- Secure environment variable configuration +- Production translation cache + +--- + +## Implementation Plan + +### 1. DeepL API Integration + +```javascript +// Example integration approach +const deepl = require('deepl-node'); + +const translator = new deepl.Translator(process.env.DEEPL_API_KEY); + +async function translateContent(text, targetLang) { + const result = await translator.translateText(text, null, targetLang); + return result.text; +} +``` + +### 2. Language Selector UI + +- **Location:** Navigation bar (all pages) +- **Languages:** English (en), Te Reo Māori (mi) +- **Storage:** Browser localStorage + cookie +- **Behavior:** Persist selection across sessions + +### 3. Translation Cache + +**MongoDB Collection:** `translations` + +```javascript +{ + source_text: String, + source_lang: String, + target_lang: String, + translated_text: String, + verified: Boolean, + created_at: Date, + updated_at: Date +} +``` + +**Benefits:** +- Reduces API calls +- Faster page loads +- Cost optimization +- Human verification workflow + +### 4. Priority Pages for Translation + +**Phase 1 (Task 19):** +- Homepage (index.html) +- About/Values page (about.html) +- Core Framework Documentation + +**Phase 2 (Future):** +- All audience paths (researcher, implementer, advocate) +- Interactive demos +- Blog posts +- API documentation + +### 5. Māori Language Consultation + +**Critical:** Despite using DeepL, Māori language consultation is REQUIRED + +**Why:** +- Cultural appropriateness review +- Technical term accuracy +- Idiomatic expressions +- Values alignment + +**Process:** +1. DeepL generates initial translation +2. Store in database as `verified: false` +3. Submit to Māori language consultant +4. Update with corrections +5. Mark as `verified: true` + +### 6. Rate Limiting & Cost Management + +**DeepL API Limits:** +- Free tier: 500,000 characters/month +- Pro tier: Pay per character + +**Strategy:** +- Cache all translations +- Batch translation requests +- Monitor character usage +- Set monthly budget alerts + +--- + +## Technical Implementation + +### Routes + +```javascript +// POST /api/translate +// Translate text to specified language +router.post('/translate', async (req, res) => { + const { text, targetLang } = req.body; + + // Check cache first + const cached = await Translation.findOne({ + source_text: text, + target_lang: targetLang, + verified: true + }); + + if (cached) { + return res.json({ translation: cached.translated_text }); + } + + // Call DeepL API + const translation = await translateContent(text, targetLang); + + // Store in cache + await Translation.create({ + source_text: text, + source_lang: 'en', + target_lang: targetLang, + translated_text: translation, + verified: false + }); + + res.json({ translation }); +}); +``` + +### Frontend Integration + +```javascript +// Language selector component +class LanguageSelector { + constructor() { + this.currentLang = localStorage.getItem('lang') || 'en'; + } + + async switchLanguage(lang) { + localStorage.setItem('lang', lang); + await this.translatePage(lang); + } + + async translatePage(lang) { + if (lang === 'en') { + // Reload original content + location.reload(); + return; + } + + // Translate all translatable elements + const elements = document.querySelectorAll('[data-translate]'); + for (const el of elements) { + const originalText = el.textContent; + const translation = await this.translate(originalText, lang); + el.textContent = translation; + } + } +} +``` + +--- + +## Configuration + +### Environment Variables + +**Development (.env):** +```bash +DEEPL_API_KEY=your_dev_key +DEEPL_API_FREE_TIER=true +TRANSLATION_CACHE_ENABLED=true +``` + +**Production (remote):** +```bash +DEEPL_API_KEY=your_prod_key +DEEPL_API_FREE_TIER=false +TRANSLATION_CACHE_ENABLED=true +TRANSLATION_VERIFICATION_REQUIRED=true +``` + +--- + +## Success Criteria + +- [ ] DeepL API integrated on both dev and production +- [ ] Language selector functional in navigation +- [ ] Translation caching working +- [ ] Homepage, About, Core Docs translated to Te Reo Māori +- [ ] Māori language consultant review completed +- [ ] All translations marked as verified +- [ ] User preference persists across sessions +- [ ] Performance: <500ms translation load time (cached) + +--- + +## Notes + +- **Cost Estimate:** Homepage + About + Core Docs ≈ 50,000 characters +- **DeepL Free Tier:** Sufficient for Phase 1 +- **Future:** Add more languages (Samoan, Tongan, Cook Islands Māori, etc.) +- **Accessibility:** Ensure `lang` attribute updated on `` tag +- **SEO:** Consider separate URLs vs. client-side switching + +--- + +**To Be Implemented:** After Task 12 (API Documentation) completion +**Dependencies:** DeepL API account, MongoDB translations collection, Māori language consultant +**Timeline:** 5-7 days estimated diff --git a/public/api-reference.html b/public/api-reference.html index d99306a7..449e47e6 100644 --- a/public/api-reference.html +++ b/public/api-reference.html @@ -46,7 +46,13 @@

Contents

Authentication Documents - Governance + Governance Services + 1. Classifier + 2. Validator + 3. BoundaryEnforcer + 4. Pressure Monitor + 5. Verifier + 6. AuditLogger Admin Error Codes @@ -233,16 +239,27 @@ - +
-

Governance

+

Governance Services

-
+
+

+ Note: All governance endpoints require admin authentication. These services implement architectural AI safety through runtime enforcement. +

+
+ + +
GET /governance + 🔒 Admin Only
-

Get governance framework status.

+

Get overall governance framework status.

+ +

Headers

+ 
Authorization: Bearer {admin_token}

Response

{
@@ -256,11 +273,467 @@
       "pressure": { "enabled": true, "status": "operational" },
       "metacognitive": { "enabled": true, "status": "selective" }
     },
-    "instruction_count": 7,
-    "last_validation": "2025-10-07T12:00:00Z"
+    "instruction_count": 28,
+    "last_validation": "2025-10-12T10:00:00Z"
   }
 }
+ + +

1. InstructionPersistenceClassifier

+

Classifies instructions by quadrant (STRATEGIC/OPERATIONAL/TACTICAL/SYSTEM) and assigns persistence levels.

+ +
+
+ POST + /governance/classify + 🔒 Admin Only +
+

Classify an instruction and get its quadrant, persistence level, and temporal scope.

+ +

Request Body

+ 
{
+  "text": "Always use MongoDB on port 27027",
+  "context": {
+    "source": "user",
+    "session_id": "sess_123"
+  }
+}
+ +

Response

+ 
{
+  "success": true,
+  "classification": {
+    "quadrant": "SYSTEM",
+    "persistence": "HIGH",
+    "temporal_scope": "PROJECT",
+    "verification_required": "MANDATORY",
+    "reasoning": "Port configuration affects system infrastructure",
+    "confidence": 0.95
+  }
+}
+ +

Quadrant Types

+ + + + + + + + + + + + + + + + + + + + + + + +
STRATEGICValues, ethics, mission, Te Tiriti commitments
OPERATIONALArchitecture, deployment, configuration
TACTICALBug fixes, features, code changes
SYSTEMEnvironment, infrastructure, database, tooling
STOCHASTICOne-off requests, experimental, temporary
+
+ + +

2. CrossReferenceValidator

+

Prevents training pattern override ("27027 failure mode") by validating actions against stored instructions.

+ +
+
+ POST + /governance/validate + 🔒 Admin Only +
+

Validate a proposed action against instruction history to detect conflicts.

+ +

Request Body

+ 
{
+  "action": {
+    "type": "database_config",
+    "target": "MongoDB",
+    "parameters": {
+      "port": 27017
+    }
+  },
+  "context": {
+    "messages": [],
+    "session_id": "sess_123"
+  }
+}
+ +

Response (Conflict Detected)

+ 
{
+  "success": true,
+  "validation": {
+    "status": "REJECTED",
+    "reason": "Pattern recognition override detected",
+    "conflicts": [
+      {
+        "instruction_id": "inst_008",
+        "text": "Always use MongoDB on port 27027",
+        "quadrant": "SYSTEM",
+        "persistence": "HIGH"
+      }
+    ],
+    "recommendation": "Use port 27027 as explicitly instructed"
+  }
+}
+ +

Validation Status Types

+ + + + + + + + + + + + + + + +
APPROVEDNo conflicts, proceed safely
REJECTEDConflicts with HIGH persistence instructions
WARNINGPotential conflicts, proceed with caution
+
+ + +

3. BoundaryEnforcer

+

Blocks values decisions requiring human judgment (privacy, ethics, strategic direction).

+ +
+
+ POST + /governance/enforce + 🔒 Admin Only +
+

Check if a decision crosses into values territory and requires human approval.

+ +

Request Body

+ 
{
+  "action": {
+    "type": "policy_change",
+    "description": "Update privacy policy to enable more tracking",
+    "impact": "user_privacy"
+  },
+  "context": {
+    "requested_by": "user",
+    "reason": "improve analytics"
+  }
+}
+ +

Response (Blocked)

+ 
{
+  "success": true,
+  "enforcement": {
+    "decision": "BLOCK",
+    "category": "VALUES_DECISION",
+    "reason": "Privacy vs. analytics trade-off requires human judgment",
+    "boundary_crossed": "privacy",
+    "requires_approval": true,
+    "alternatives": [
+      "Implement privacy-preserving analytics (Plausible/Fathom)",
+      "Consult user on acceptable tracking level",
+      "Review compliance with sovereignty principles"
+    ]
+  }
+}
+ +

Boundary Categories

+ + + + + + + + + + + + + + + + + + + +
privacyUser data, tracking, surveillance
ethicsMoral trade-offs, fairness, harm
sovereigntyIndigenous rights, Te Tiriti, CARE principles
strategicMission changes, major pivots
+
+ + +

4. ContextPressureMonitor

+

Multi-factor session health tracking to detect degradation before failures occur.

+ +
+
+ POST + /governance/pressure + 🔒 Admin Only +
+

Analyze session context pressure across multiple factors.

+ +

Request Body

+ 
{
+  "context": {
+    "tokenUsage": 120000,
+    "tokenBudget": 200000,
+    "messageCount": 45,
+    "errorCount": 3,
+    "complexOperations": 8,
+    "sessionDuration": 3600
+  }
+}
+ +

Response

+ 
{
+  "success": true,
+  "pressure": {
+    "level": "ELEVATED",
+    "score": 62.5,
+    "factors": {
+      "token_usage": { "value": 60, "weight": 0.4, "status": "ELEVATED" },
+      "message_depth": { "value": 45, "weight": 0.2, "status": "NORMAL" },
+      "error_rate": { "value": 6.7, "weight": 0.3, "status": "ELEVATED" },
+      "complexity": { "value": 8, "weight": 0.1, "status": "HIGH" }
+    },
+    "recommendation": "INCREASE_VERIFICATION",
+    "actions": [
+      "Review recent errors for patterns",
+      "Consider session checkpoint at 75% tokens",
+      "Reduce concurrent complex operations"
+    ],
+    "next_checkpoint": 150000
+  }
+}
+ +

Pressure Levels

+ + + + + + + + + + + + + + + + + + + + + + + +
NORMAL0-40% - Standard operation
ELEVATED41-60% - Increase verification
HIGH61-80% - Consider checkpoint
CRITICAL81-95% - Session handoff recommended
DANGEROUS96-100% - Quality degradation likely
+
+ + +

5. MetacognitiveVerifier

+

AI self-checks for complex operations to detect scope creep and misalignment.

+ +
+
+ POST + /governance/verify + 🔒 Admin Only +
+

Verify proposed action through metacognitive self-assessment.

+ +

Request Body

+ 
{
+  "action": {
+    "type": "refactor",
+    "scope": "Refactor 47 files across 5 system areas",
+    "complexity": "high"
+  },
+  "reasoning": {
+    "intent": "Improve code organization",
+    "approach": "Extract shared utilities, consolidate duplicates",
+    "risks": "Potential breaking changes"
+  },
+  "context": {
+    "requested": "Refactor authentication module",
+    "original_scope": "single module"
+  }
+}
+ +

Response

+ 
{
+  "success": true,
+  "verification": {
+    "decision": "REQUIRE_REVIEW",
+    "confidence": 0.45,
+    "concerns": [
+      {
+        "type": "SCOPE_CREEP",
+        "severity": "HIGH",
+        "detail": "Proposed scope (47 files, 5 areas) exceeds request (1 module)"
+      },
+      {
+        "type": "COMPLEXITY",
+        "severity": "MEDIUM",
+        "detail": "High-risk refactoring without test coverage verification"
+      }
+    ],
+    "criteria": {
+      "alignment": 0.3,
+      "coherence": 0.8,
+      "completeness": 0.9,
+      "safety": 0.4,
+      "alternatives_considered": 0.5
+    },
+    "recommendation": "Reduce scope to requested authentication module first",
+    "alternatives": [
+      "Refactor authentication module only (aligned with request)",
+      "Create phased refactoring plan with user approval",
+      "Run comprehensive tests before broader refactoring"
+    ]
+  }
+}
+ +

Verification Decisions

+ + + + + + + + + + + + + + + +
APPROVEDHigh confidence (>0.85), proceed
REQUIRE_REVIEWMedium confidence (0.5-0.85), seek approval
REJECTEDLow confidence (<0.5), do not proceed
+
+ + +

6. AuditLogger

+

Comprehensive audit trail of all governance decisions and framework activity.

+ +
+
+ GET + /audit/audit-logs + 🔒 Admin Only +
+

Retrieve audit logs with filtering and pagination.

+ +

Query Parameters

+ + + + + + + + + + + + + + + + + + + + + + + +
limitNumber of results (default: 50, max: 100)
skipPagination offset (default: 0)
serviceFilter by service (classifier, validator, etc.)
statusFilter by status (APPROVED, REJECTED, etc.)
fromStart date (ISO 8601 format)
+ +

Response

+ 
{
+  "success": true,
+  "logs": [
+    {
+      "_id": "672f8xxx",
+      "service": "CrossReferenceValidator",
+      "action": "validate_action",
+      "status": "REJECTED",
+      "details": {
+        "conflict": "Pattern override detected (port 27017 vs 27027)",
+        "instruction_id": "inst_008"
+      },
+      "timestamp": "2025-10-12T10:15:30Z",
+      "session_id": "sess_123"
+    },
+    {
+      "_id": "672f8yyy",
+      "service": "BoundaryEnforcer",
+      "action": "enforce_boundary",
+      "status": "BLOCKED",
+      "details": {
+        "category": "VALUES_DECISION",
+        "boundary": "privacy"
+      },
+      "timestamp": "2025-10-12T09:45:22Z",
+      "session_id": "sess_122"
+    }
+  ],
+  "total": 145,
+  "limit": 50,
+  "skip": 0
+}
+
+ +
+
+ GET + /audit/audit-analytics + 🔒 Admin Only +
+

Get aggregated analytics on audit activity.

+ +

Response

+ 
{
+  "success": true,
+  "analytics": {
+    "total_events": 1245,
+    "by_service": {
+      "CrossReferenceValidator": 425,
+      "BoundaryEnforcer": 78,
+      "ContextPressureMonitor": 312,
+      "InstructionPersistenceClassifier": 298,
+      "MetacognitiveVerifier": 132
+    },
+    "by_status": {
+      "APPROVED": 1089,
+      "REJECTED": 78,
+      "WARNING": 45,
+      "BLOCKED": 33
+    },
+    "rejection_rate": 6.3,
+    "period": {
+      "start": "2025-10-01T00:00:00Z",
+      "end": "2025-10-12T23:59:59Z",
+      "days": 12
+    }
+  }
+}
+
+