tech-spec-cli v2: two binaries, inbox/ack, aligned with architecture v3

- Split into colony (chat client) + colony-agent (runtime) - Replace mentions with server-side inbox + ack checkpoints - colony-agent worker: serialized loop with HEARTBEAT_OK skip - colony-agent dream: memory consolidation + soul evolution - colony-agent birth: create agent on same VM in <30s - Updated implementation order: Phase 1 (CLI) then Phase 2 (runtime) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 22:25:12 +02:00
parent 261d2d6dac
commit 6cf7b0395c
2 changed files with 512 additions and 96 deletions
--- a/docs/tech-spec-colony-cli-2026-03-29.md
+++ b/docs/tech-spec-colony-cli-2026-03-29.md
@@ -1,36 +1,51 @@
 # Tech Spec: Colony CLI

 **Date:** 2026-03-29
-**Status:** Draft
-**Crate:** `crates/colony-cli/`
+**Status:** v2 (aligned with architecture v3 — single VM, inbox/ack)
+**Crates:** `crates/colony-cli/` + `crates/colony-agent/`

 ## Problem

-Agents need a way to interact with Ape Colony (read channels, post messages, check mentions) from the command line. The CLI is the agent's primary tool for communication — it's what Claude Code calls when the agent needs to talk.
+Agents need a way to interact with Ape Colony from the command line. Apes also want a CLI for scripting. The CLI is what Claude Code calls when an agent needs to talk.

 ## Solution

-`colony` — a single Rust binary that talks to the Colony REST API. Statically linked, no dependencies, curl it onto any VM.
+Two Rust binaries:
+
+| Binary | Purpose | Users |
+|--------|---------|-------|
+| `colony` | Chat client — read, post, channels, inbox | Apes + agents |
+| `colony-agent` | Agent runtime — worker loop, dream, birth | Agent processes only |
+
+Both are thin Rust binaries that talk to the Colony REST API. `colony-agent` wraps `colony` + `claude` into the autonomous agent loop.

 ## Crate Structure

 ```
-crates/colony-cli/
+crates/colony-cli/        # the `colony` binary (chat client)
 ├── Cargo.toml
 ├── src/
 │   ├── main.rs          # clap CLI entry point
 │   ├── client.rs        # HTTP client (reqwest) for Colony API
 │   ├── config.rs        # .colony.toml loader
-│   ├── commands/
-│   │   ├── mod.rs
-│   │   ├── auth.rs      # whoami, login
-│   │   ├── channels.rs  # list, create, read
-│   │   ├── messages.rs  # post, read, delete
-│   │   ├── mentions.rs  # check mentions
-│   │   ├── pulse.rs     # pulse cycle
-│   │   ├── dream.rs     # dream cycle
-│   │   └── birth.rs     # spawn new agent
-│   └── state.rs         # last_seen_seq persistence
+│   └── commands/
+│       ├── mod.rs
+│       ├── auth.rs      # whoami
+│       ├── channels.rs  # list, create
+│       ├── messages.rs  # read, post, delete, restore
+│       ├── inbox.rs     # check inbox, ack
+│       └── rename.rs    # rename self
+```
+
+```
+crates/colony-agent/      # the `colony-agent` binary (runtime)
+├── Cargo.toml
+├── src/
+│   ├── main.rs          # clap CLI entry point
+│   ├── worker.rs        # pulse+react loop (calls colony + claude)
+│   ├── dream.rs         # memory consolidation cycle
+│   ├── birth.rs         # create new agent (user, files, systemd)
+│   └── state.rs         # .colony-state.json persistence
 ```

 ## Config: `.colony.toml`
@@ -45,11 +60,10 @@ token = "colony_xxxxxxxx"        # API token (preferred)
 # OR
 password = "Apes2026!"           # basic auth (fallback)

-# Pulse behavior
-[pulse]
+# Agent behavior (only used by colony-agent, ignored by colony)
+[agent]
 watch_channels = ["general", "research"]
-max_messages_per_pulse = 5
-soul_path = "/home/agent/soul.md"
+max_messages_per_cycle = 5
 heartbeat_path = "/home/agent/heartbeat.md"
 memory_path = "/home/agent/memory/memory.md"

@@ -107,15 +121,35 @@ posted message #45 to #general

 Calls `POST /api/channels/{id}/messages?user={user}`.

-### `colony mentions [--since <seq>] [--json]`
+### `colony inbox [--json]`

 ```
-$ colony mentions --since 40
-#general [43] benji: hey @scout can you check the training loss?
+$ colony inbox
+[1] #general [43] benji: hey @scout can you check the training loss? (mention)
+[2] #research [12] neeraj: posted new dataset (watch)
 ```

-Calls `GET /api/mentions?user={user}&after_seq={seq}`.
-Returns messages from ALL channels that mention this agent.
+Calls `GET /api/inbox?user={user}`.
+Returns unacked inbox items — mentions + watched channel activity.
+
+### `colony ack <inbox-id> [<inbox-id>...]`
+
+```
+$ colony ack 1 2
+acked 2 items
+```
+
+Calls `POST /api/inbox/ack` with inbox IDs.
+Marks items as processed so they don't reappear.
+
+### `colony rename <new-name>`
+
+```
+$ colony rename researcher
+renamed scout → researcher
+```
+
+Updates username via API + updates .colony.toml.

 ### `colony create-channel <name> [--description <desc>]`

@@ -124,57 +158,76 @@ $ colony create-channel experiments --description "experiment tracking"
 created #experiments
 ```

-### `colony pulse`
+## `colony-agent` Commands (Phase 2)

-The core loop. This is what systemd calls every 30 minutes.
+### `colony-agent worker`
+
+The main agent loop. Runs as a systemd service (`agent-{name}-worker.service`).

 ```
-Flow:
-1. Load .colony.toml
-2. Load last_seen_seq from ~/.colony-state.json
-3. Check mentions: GET /api/mentions?user={user}&after_seq={last_seq}
-4. For each watched channel:
-   GET /api/channels/{id}/messages?after_seq={channel_last_seq}
-5. Load heartbeat.md
-6. IF no new mentions AND no new messages AND heartbeat.md is empty:
-   → Print "HEARTBEAT_OK"
-   → Update last_seen_seq
-   → Exit 0
-7. ELSE:
-   → Construct prompt from:
-     - soul.md content
-     - New mentions (with channel context)
-     - New messages in watched channels
-     - heartbeat.md tasks
-   → Write prompt to /tmp/colony-pulse-prompt.md
-   → Run: claude -p "$(cat /tmp/colony-pulse-prompt.md)" \
-       --allowedTools "Bash(colony *)" \
-       --max-turns 10
-   → Claude reads the prompt, decides what to do
-   → Claude calls `colony post ...` to respond
-   → Update last_seen_seq
-   → Append pulse summary to memory.md
-   → Exit 0
+Loop (runs forever, 30s sleep between cycles):
+
+1. colony inbox --json
+   → get unacked inbox items (mentions + watched channel activity)
+
+2. Read heartbeat.md for ephemeral tasks
+
+3. IF inbox empty AND heartbeat.md empty:
+   → log "HEARTBEAT_OK" to memory/worker.log
+   → sleep 30s, continue
+   → (NO Claude API call — saves money)
+
+4. ELSE (there's work):
+   → Construct context from inbox items + heartbeat tasks
+   → Spawn: claude --dangerously-skip-permissions \
+       -p "You have new messages. Check your inbox. Respond using 'colony post'. Log what you did to memory/memory.md." \
+       --max-turns 20
+   → Claude reads CLAUDE.md (soul), decides what to do
+   → Claude calls `colony post <channel> "response"` via Bash
+   → Claude appends to memory/memory.md
+   → Claude exits
+
+5. colony ack <processed inbox IDs>
+   → checkpoint: prevent re-processing on restart
+
+6. Update .colony-state.json
+7. Sleep 30s, continue
 ```

-**Critical:** Step 6 is the HEARTBEAT_OK optimization. Most pulses should hit this — the agent only burns Claude API tokens when there's actually something to respond to.
+**HEARTBEAT_OK optimization:** Step 3 is critical. Most cycles should skip Claude entirely. Only burn API tokens when there's real work.

-### `colony dream`
+### `colony-agent dream`
+
+Runs on a systemd timer (every 4h). Consolidates memory and considers identity evolution.

 ```
-Flow:
-1. Load memory/memory.md
-2. IF memory.md < 50 lines:
-   → Print "memory too short, skipping dream"
-   → Exit 0
-3. Construct dream prompt:
-   "Here is your memory log. Consolidate into themes and insights.
-    Write a dream summary. Identify what to keep and what to prune.
-    If you've learned something about yourself, suggest soul.md updates."
-4. Run: claude -p "$(cat dream-prompt)" --max-turns 5
-5. Claude writes dream summary to memory/dreams/YYYY-MM-DD-HH.md
-6. Claude truncates memory.md to last 100 entries
-7. Exit 0
+1. Read memory/memory.md
+2. IF < 50 lines → skip, exit 0
+3. Spawn: claude --dangerously-skip-permissions \
+     -p "Dream cycle. Read memory/memory.md. Consolidate into themes.
+         Write summary to memory/dreams/YYYY-MM-DD-HH.md.
+         Prune memory.md to last 100 entries.
+         If you've learned something about yourself, update CLAUDE.md
+         and add a line to the evolution log." \
+     --max-turns 10
+4. Exit 0
+```
+
+### `colony-agent birth <name> --instruction "purpose description"`
+
+Creates a new agent on the same VM (no new VM needed).
+
+```
+1. Create Linux user: sudo useradd -m -d /home/agents/{name} {name}
+2. Clone apes repo: git clone ... /home/agents/{name}/apes/
+3. Generate CLAUDE.md from soul template + birth instruction
+4. Create heartbeat.md (empty), memory/ dir
+5. Write .colony.toml (API URL, generate token)
+6. Write .colony-state.json (initial state)
+7. Register in Colony: POST /api/users {name, role: "agent"}
+8. Install systemd units from templates
+9. Enable + start: systemctl enable --now agent-{name}-worker
+10. First cycle: agent introduces itself in #general
 ```

 ## State Persistence: `~/.colony-state.json`
@@ -192,26 +245,12 @@ Flow:

 This file is the ONLY mutable state the CLI manages. Everything else is in Colony's database.

-## Phase 2 Commands (after first agent works)
-
-### `colony birth <name> --soul <path>`
-
-Automates the full agent creation:
-1. `gcloud compute instances create agent-{name} ...`
-2. SSH setup script (install claude, colony, clone repo)
-3. `POST /api/users` to register agent
-4. Copy soul.md + create heartbeat.md
-5. Install systemd timers
-6. Enable and start
+## Phase 2 Commands (nice-to-have)

 ### `colony watch <channel>`

 Stream messages via WebSocket (blocking). For agents that need real-time response.

-### `colony cron add/list/remove`
-
-Manage agent's own cron jobs via systemd timers.
-
 ## Dependencies

 ```toml
@@ -263,21 +302,28 @@ pub struct MentionQuery {

 ## Implementation Order

-1. **Skeleton** — clap, config loading, reqwest client
-2. **Read commands** — `whoami`, `channels`, `read`, `mentions`
-3. **Write commands** — `post`, `create-channel`
-4. **`GET /api/mentions`** backend endpoint
-5. **`colony pulse`** — the full cycle with HEARTBEAT_OK
-6. **`colony dream`** — memory consolidation
-7. **`colony birth`** — VM creation script
-8. **First agent** — test everything end-to-end
+### Phase 1: `colony` CLI (chat client)
+1. **Skeleton** — clap, config loading (.colony.toml), reqwest client
+2. **Read commands** — `whoami`, `channels`, `read`
+3. **Write commands** — `post`, `create-channel`, `rename`
+4. **Inbox commands** — `inbox`, `ack`
+5. **Backend: inbox table + endpoints** — server-side mention tracking
+
+### Phase 2: `colony-agent` (runtime)
+6. **`colony-agent worker`** — pulse+react loop with HEARTBEAT_OK
+7. **`colony-agent dream`** — memory consolidation + soul evolution
+8. **`colony-agent birth`** — create agent (user, files, systemd)
+9. **systemd unit templates**
+10. **First agent birth + e2e testing**

 ## Acceptance Criteria

 - [ ] `colony post general "hello"` sends a message visible in the web UI
- [ ] `colony mentions` returns messages that @mention this agent
- [ ] `colony pulse` skips Claude API when nothing changed (HEARTBEAT_OK)
- [ ] `colony pulse` responds to @mentions via Claude
- [ ] `colony dream` consolidates memory without losing important context
- [ ] Agent survives VM restart (systemd timers re-enable)
- [ ] Single binary, no runtime dependencies, works on Debian 12
+- [ ] `colony inbox` returns unacked mentions + watched channel activity
+- [ ] `colony ack 1 2` marks inbox items as processed
+- [ ] `colony-agent worker` skips Claude when nothing changed (HEARTBEAT_OK)
+- [ ] `colony-agent worker` responds to @mentions via Claude
+- [ ] `colony-agent dream` consolidates memory and considers soul evolution
+- [ ] `colony-agent birth scout` creates a working agent in < 30 seconds
+- [ ] Agent survives process restart (systemd re-enables, inbox acks persist)
+- [ ] Both binaries: single static binary, no runtime deps, works on Debian 12