status: ACTIVE

CEO Plan: Minions as Universal Agent Orchestration Protocol

Generated by /plan-ceo-review on 2026-04-15 Branch: garrytan/minions-jobs | Mode: SCOPE EXPANSION Repo: garrytan/gbrain

Vision

10x Check

Instead of "GBrain has a queue, OpenClaw uses it," make Minions a universal agent orchestration protocol. Any platform (OpenClaw, Hermes, Claude Code, Codex, custom scripts) submits, monitors, steers, and composes agents through the same Postgres-native protocol. GBrain IS the agent control plane.

Platonic Ideal (aspirational North Star, NOT in v1 scope)

Open a terminal, type gbrain jobs dashboard. See every agent across every platform. Their progress, tool calls, token spend. Click any agent for full execution trace. Type a message to redirect a running agent mid-flight. See the governor's decisions visualized. Run A/B tests between agent configurations. The feeling: complete situational awareness of your AI workforce.

Note: The dashboard, A/B testing, and visual governor are future phases. This plan builds the primitives they would sit on top of: real-time events, structured progress, token accounting, inbox with ack, and session transcripts.

Scope Decisions

#	Proposal	Effort	Decision	Reasoning
1	pg LISTEN/NOTIFY real-time events	S	ACCEPTED	Sub-second event delivery vs 5s polling. Every platform benefits.
2	Structured progress protocol	S	ACCEPTED	Standard progress makes unified dashboard possible.
3	Job cost tracking (token accounting)	M	ACCEPTED	Token cost is #1 thing users want to know about agent work.
4	Job replay	S	ACCEPTED	Small surface area, high utility for debugging failures.
5	Job groups / waves	M	DEFERRED	Parent-child already provides grouping. Overlap concern.
6	Inbox acknowledgment (read receipts)	S	ACCEPTED	Without it, inbox is fire-and-forget — same problem we're fixing.
7	Universal agent protocol	S	ACCEPTED	Design framing, not extra code. Platform-agnostic naming/docs.
8	Session transcript capture	M	ACCEPTED	Full audit trail of every agent run.

Accepted Scope — Implementation Detail

0a. Pause/resume (from base plan)

Schema: Add 'paused' to MinionJobStatus (already in migration v6 constraint).

New methods:

MinionQueue.pauseJob(id): MinionJob | null Transitions waiting or active → paused. For active jobs, clears lock_token and lock_until (worker will detect lock loss and stop). Returns null if job not in pausable state.
MinionQueue.resumeJob(id): MinionJob | null Transitions paused → waiting. Resets for claiming. Returns null if not paused.

Worker integration: Worker's lock renewal loop checks isActive(). When a job is paused, the lock is cleared, so renewLock() returns false and the worker stops execution gracefully (same path as stall detection). The job's progress and state are preserved in the DB for when it resumes.

MCP operations: pause_job, resume_job (added in Step 3 of implementation plan).

PGLite compatibility: Full.

0b. Resource governor (from base plan)

New file: src/core/minions/governor.ts

interface GovernorConfig {
  maxConcurrency: number;       // ceiling
  minConcurrency: number;       // floor (default 1)
  checkIntervalMs: number;      // default 10000
  cpuThreshold: number;         // default 0.80 (80%)
  memoryThreshold: number;      // default 0.85 (85%)
  circuitBreakerMemory: number; // default 0.90 (90%)
}

class ResourceGovernor {
  getEffectiveConcurrency(): number;  // current allowed concurrency
  start(): void;                       // begin polling system metrics
  stop(): void;                        // stop polling
  onCircuitBreak(cb: (jobId) => void): void; // kill callback
}

System metrics: Reuse getSystemLoad() from src/core/backoff.ts (already implements CPU and memory checks). Add event loop lag measurement via perf_hooks.monitorEventLoopDelay().

Worker integration: MinionWorker.start() consults governor.getEffectiveConcurrency() before claiming new jobs. If current in-flight count >= effective concurrency, skip claim.

Circuit breaker: If memory > 90%, governor calls onCircuitBreak with the lowest-priority active job ID. Worker cancels that job via failJob() with UnrecoverableError("circuit breaker: memory pressure").

Prerequisite: Concurrent job processing must be implemented first (see Concurrency Note below).

PGLite compatibility: Full (governor is app-level, not DB-level).

1. pg LISTEN/NOTIFY (real-time events)

Schema: No new columns. Add NOTIFY triggers to state transitions.

SQL trigger:

CREATE OR REPLACE FUNCTION notify_minion_job_change() RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify('minion_jobs', json_build_object(
    'id', NEW.id, 'status', NEW.status, 'name', NEW.name,
    'queue', NEW.queue, 'prev_status', COALESCE(OLD.status, 'new')
  )::text);
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER minion_job_notify AFTER INSERT OR UPDATE OF status ON minion_jobs
  FOR EACH ROW EXECUTE FUNCTION notify_minion_job_change();

New method: MinionQueue.subscribe(callback: (event) => void): () => void Returns unsubscribe function. Requires direct Postgres connection (NOT pooled).

PGLite compatibility: PGLite does NOT support LISTEN/NOTIFY. Fallback: polling via getJob() at configurable interval (default 2s). The subscribe() method detects engine type and uses polling fallback automatically.

Supabase constraint: Requires direct connection (port 5432), not pgBouncer pooler (port 6543). Document in skill file and setup guide.

2. Structured progress protocol

TypeScript interface (convention, not enforced at DB level):

interface AgentProgress {
  step: number;           // current step (1-based)
  total: number;          // total expected steps (0 = unknown)
  message: string;        // human-readable status
  tokens_in: number;      // cumulative input tokens
  tokens_out: number;     // cumulative output tokens
  last_tool: string;      // name of last tool called
  started_at: string;     // ISO 8601 when this step started
}

Storage: Existing progress JSONB column. No schema change needed. Handlers use ctx.updateProgress(agentProgress). Non-agent jobs can use any JSONB shape (backward compatible).

Validation: updateProgress() accepts any JSONB. The AgentProgress interface is a convention enforced by the agent handler, not by the queue.

3. Job cost tracking (token accounting)

Schema changes (migration v6):

ALTER TABLE minion_jobs ADD COLUMN tokens_input INTEGER DEFAULT 0;
ALTER TABLE minion_jobs ADD COLUMN tokens_output INTEGER DEFAULT 0;
ALTER TABLE minion_jobs ADD COLUMN tokens_cache_read INTEGER DEFAULT 0;
ALTER TABLE minion_jobs ADD COLUMN cost_usd NUMERIC(10,6) DEFAULT 0;

New method: MinionQueue.updateTokens(id, lockToken, { input, output, cache_read, cost_usd }) Accumulates (adds to existing values, does not replace).

Parent rollup: When completeJob() is called, if parent_job_id is set, add this job's token counts to the parent's via:

UPDATE minion_jobs SET
  tokens_input = tokens_input + $child_input,
  tokens_output = tokens_output + $child_output,
  tokens_cache_read = tokens_cache_read + $child_cache,
  cost_usd = cost_usd + $child_cost
WHERE id = $parent_id;

PGLite compatibility: Full support (standard columns).

4. Job replay

New method: MinionQueue.replayJob(id, dataOverrides?: Record<string, unknown>): MinionJob

Implementation: Read the completed/failed/dead job. Create a NEW job with:

Same name, queue, priority, max_attempts, backoff_type, backoff_delay
data = deep merge of original data + overrides
Fresh attempts_made: 0, status: 'waiting'
parent_job_id = null (replay is a new top-level job, not a child)
Does NOT clone children (replay is a single job, not a DAG)

Constraint: Only works on terminal statuses (completed/failed/dead). Returns the new job record.

Idempotency: Each replay creates a distinct new job. No deduplication. If the original had side effects, the replay may repeat them. Document this in the skill file as a user responsibility.

5. Inbox (sidechannel messaging)

Schema changes (migration v6):

ALTER TABLE minion_jobs ADD COLUMN inbox JSONB DEFAULT '[]';

Inbox message format:

interface InboxMessage {
  id: string;          // UUIDv4
  sent_at: string;     // ISO 8601
  read_at: string | null;  // null until worker reads it
  sender: string;      // 'parent' | 'user' | job ID
  payload: unknown;    // arbitrary directive
}

New methods:

MinionQueue.sendMessage(jobId, payload, sender?): InboxMessage Appends message to inbox array via atomic JSONB append (inbox = inbox || $1::jsonb), not read-modify-write. Returns the message with id + sent_at.
MinionQueue.readInbox(jobId, lockToken): InboxMessage[] Returns unread messages (read_at = null). Marks them as read (sets read_at). Token-fenced: only the worker holding the lock can read.

Worker integration: Agent handler calls readInbox() on each iteration. If messages exist, injects them into the agent's context as system messages.

PGLite compatibility: Full support (standard JSONB column).

6. Inbox acknowledgment (read receipts)

Built into the inbox design above. The read_at field on each InboxMessage provides the receipt. sendMessage() returns the message ID; the sender can later check getJob(id) and inspect inbox to see which messages have been read.

No additional schema or methods needed beyond what's in #5.

7. Universal agent protocol (platform-agnostic framing)

This is a design decision, not code. It means:

The skill file (skills/minion-orchestrator/SKILL.md) is written for ANY agent platform, not just OpenClaw. Examples show MCP tool calls, not OpenClaw-specific commands.

The agent handler (agent-handler.ts) accepts a generic interface:

interface AgentJobData {
  prompt: string;
  tools?: string[];        // MCP tool names
  model?: string;          // e.g., 'claude-opus-4-6', 'gpt-4o'
  context?: string;        // additional context
  platform?: string;       // 'openclaw' | 'hermes' | 'claude-code' | 'custom'
  max_iterations?: number; // agent loop budget
}

The OpenClaw plugin is ONE consumer. Hermes, Claude Code extensions, or custom scripts can submit agent jobs through the same MCP operations.
NOT in v1 scope: Multi-tenant auth, cross-network connectivity, protocol versioning, API key isolation. These are Phase 2 concerns when actual multi-platform usage materializes. v1 is single-user, single-brain.

Agent Handler Architecture (critical design decision)

The agent handler does NOT live in GBrain. GBrain provides the queue infrastructure and a clean handler contract. The actual agent execution lives in the platform plugin.

GBrain (this repo):
  MinionQueue  — queue/claim/complete/inbox/tokens/NOTIFY
  MinionWorker — poll/lock/stall/governor framework
  Handler contract — AgentJobData interface + MinionJobContext

OpenClaw plugin (separate repo):
  Registers "agent" handler with MinionWorker
  Handler calls OpenClaw's PI agent core (the actual LLM loop)
  Each iteration: readInbox → inject as system message, updateProgress, updateTokens
  Completion: store result + session transcript in job.result + job.stacktrace

GBrain ships a test/echo handler for unit testing only.

Handler contract (GBrain side):

// The handler receives this context (already exists in worker.ts)
interface MinionJobContext {
  id: number;
  name: string;
  data: Record<string, unknown>;  // AgentJobData when name="agent"
  attempts_made: number;
  updateProgress(progress: unknown): Promise<void>;
  updateTokens(tokens: TokenUpdate): Promise<void>;  // NEW
  log(message: string | TranscriptEntry): Promise<void>;
  isActive(): Promise<boolean>;
  readInbox(): Promise<InboxMessage[]>;  // NEW
}

Why this is right: GBrain is orchestration, not execution. OpenClaw has the PI agent core. Hermes has AIAgent. Claude Code has its own loop. Each platform brings its own engine and registers a handler. GBrain manages lifecycle, progress, steering, cost tracking, and persistence around it.

8. Session transcript capture

Extends existing stacktrace mechanism. The stacktrace field (JSONB array of strings) already captures log messages. Session transcripts use the same field with structured entries:

type TranscriptEntry =
  | { type: 'log'; message: string; ts: string }
  | { type: 'tool_call'; tool: string; args_size: number; result_size: number; ts: string }
  | { type: 'llm_turn'; model: string; tokens_in: number; tokens_out: number; ts: string }
  | { type: 'error'; message: string; stack?: string; ts: string };

Storage: Existing stacktrace JSONB column. No schema change. The agent handler appends TranscriptEntry objects instead of plain strings. Backward compatible: non-agent jobs continue appending strings.

Size concern: Long agent runs could generate large transcripts. Add a max_transcript_entries option (default 1000) that rotates oldest entries when exceeded (FIFO). The full transcript for forensic analysis can be stored as a brain file via gbrain files upload-raw.

Schema Migration v6

All schema changes are additive (ALTER TABLE ADD COLUMN). No backfill needed. Existing jobs continue to work with default values.

-- Migration v6: Agent orchestration primitives
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS tokens_input INTEGER DEFAULT 0;
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS tokens_output INTEGER DEFAULT 0;
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS tokens_cache_read INTEGER DEFAULT 0;

-- Separate inbox table (not JSONB on job row)
CREATE TABLE IF NOT EXISTS minion_inbox (
  id SERIAL PRIMARY KEY,
  job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
  sender TEXT NOT NULL,
  payload JSONB NOT NULL,
  sent_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  read_at TIMESTAMPTZ
);
CREATE INDEX IF NOT EXISTS idx_minion_inbox_unread
  ON minion_inbox (job_id) WHERE read_at IS NULL;

-- Status constraint update: add 'paused'
ALTER TABLE minion_jobs DROP CONSTRAINT IF EXISTS minion_jobs_status_check;
ALTER TABLE minion_jobs ADD CONSTRAINT minion_jobs_status_check
  CHECK (status IN ('waiting','active','completed','failed','delayed','dead','cancelled','waiting-children','paused'));

-- NOTIFY trigger for real-time events (Postgres only, not PGLite)
CREATE OR REPLACE FUNCTION notify_minion_job_change() RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify('minion_jobs', json_build_object(
    'id', NEW.id, 'status', NEW.status, 'name', NEW.name,
    'queue', NEW.queue, 'prev_status', COALESCE(OLD.status, 'new')
  )::text);
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER minion_job_notify AFTER INSERT OR UPDATE OF status ON minion_jobs
  FOR EACH ROW EXECUTE FUNCTION notify_minion_job_change();

PGLite Compatibility Matrix

Feature	Postgres	PGLite	Fallback
Pause/resume	Full	Full	—
Inbox + ack	Full	Full	—
Token accounting	Full	Full	—
Job replay	Full	Full	—
LISTEN/NOTIFY	Full	NO	Polling (2s interval)
NOTIFY trigger	Full	NO	Skipped in PGLite schema
Structured progress	Full	Full	—
Session transcripts	Full	Full	—
Resource governor	Full	Full	—
Worker daemon	Full	NO (existing limitation)	—

Concurrency Note

The current MinionWorker.start() processes jobs sequentially (one at a time) despite concurrency being declared in MinionWorkerOpts. Implementing actual concurrent job processing (Promise pool) is a prerequisite for the resource governor to be meaningful. The governor adjusts effective concurrency, which requires actual concurrent processing to exist.

Action: Implement concurrent job processing in worker.ts before or as part of the governor step. Use a semaphore pattern: maintain up to N in-flight promises, claim new jobs as slots free up.

Outside Voice Decisions (from adversarial review)

AbortController for pause/resume — Handler contract gets signal: AbortSignal. Pause clears lock AND signals abort. Handler must check signal.aborted on each iteration. Without this, pausing active jobs creates duplicate execution.
Drop cost_usd column — Token counts (input/output/cache_read) are stable facts. USD pricing is volatile. Compute cost at display/read time from a pricing table, not at write time. Removes cost_usd NUMERIC(10,6) from migration v6.

Separate minion_inbox table — Instead of JSONB array on job row, use a dedicated table for inbox messages. Avoids row bloat from rewriting entire inbox on every send. Properly concurrent-safe with standard INSERT (no JSONB append concerns).

CREATE TABLE minion_inbox (
  id SERIAL PRIMARY KEY,
  job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
  sender TEXT NOT NULL,
  payload JSONB NOT NULL,
  sent_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  read_at TIMESTAMPTZ
);
CREATE INDEX idx_minion_inbox_unread ON minion_inbox (job_id) WHERE read_at IS NULL;

One release, not two — Ship all features in one migration (v6). User prefers cohesive release over incremental delivery for this feature set.
Selective column projection — Fix SELECT * queries in getJobs(), claim(), handleStalled() to exclude stacktrace column. Include stacktrace only in getJob() detail view. Prevents transcript bloat from affecting query performance.

Future Phases (accepted trajectory)

Phase 2: Dashboard CLI — gbrain jobs dashboard live TUI showing all agents. Enabled by: LISTEN/NOTIFY, structured progress, token accounting.
Phase 3: Multi-tenant auth — Runtime MCP access control, per-platform API keys. Enabled by: platform-agnostic framing, sender validation on inbox.
Phase 4: Agent composition patterns — Map-reduce, pipeline, approval gates as first-class primitives. Enabled by: parent-child DAGs, inbox sidechannel.

Deferred to TODOS.md

Job groups / waves (parent-child covers this; revisit if real grouping need emerges)
cost_usd column (compute from pricing table at read time when pricing API exists)

Key Premises Confirmed

GBrain is intentionally evolving from knowledge brain to agent infrastructure (user confirmed)
Coupling between OpenClaw and GBrain's Postgres is acceptable (OpenClaw already depends on GBrain)
Full Infrastructure approach (all 8+ steps) selected over Minimal Viable or Sidecar Tracking
Prior learning [agent-dx-instruction-layer] validates that the teaching layer (skill + evals) is mandatory

MINIONS AGENT ORCHESTRATION