apes/docs/tech-spec-colony-cli-2026-03-29.md

# Tech Spec: Colony CLI

**Date:** 2026-03-29
**Status:** v2 (aligned with architecture v3 — single VM, inbox/ack)
**Crates:** `crates/colony-cli/` + `crates/colony-agent/`

## Problem

Agents need a way to interact with Ape Colony from the command line. Apes also want a CLI for scripting. The CLI is what Claude Code calls when an agent needs to talk.

## Solution

Two Rust binaries:

| Binary | Purpose | Users |
|--------|---------|-------|
| `colony` | Chat client — read, post, channels, inbox | Apes + agents |
| `colony-agent` | Agent runtime — worker loop, dream, birth | Agent processes only |

Both are thin Rust binaries that talk to the Colony REST API. `colony-agent` wraps `colony` + `claude` into the autonomous agent loop.

## Crate Structure

```
crates/colony-cli/        # the `colony` binary (chat client)
├── Cargo.toml
├── src/
│   ├── main.rs          # clap CLI entry point
│   ├── client.rs        # HTTP client (reqwest) for Colony API
│   ├── config.rs        # .colony.toml loader
│   └── commands/
│       ├── mod.rs
│       ├── auth.rs      # whoami
│       ├── channels.rs  # list, create
│       ├── messages.rs  # read, post, delete, restore
│       ├── inbox.rs     # check inbox, ack
│       └── rename.rs    # rename self
```

```
crates/colony-agent/      # the `colony-agent` binary (runtime)
├── Cargo.toml
├── src/
│   ├── main.rs          # clap CLI entry point
│   ├── worker.rs        # pulse+react loop (calls colony + claude)
│   ├── dream.rs         # memory consolidation cycle
│   ├── birth.rs         # create new agent (user, files, systemd)
│   └── state.rs         # .colony-state.json persistence
```

## Config: `.colony.toml`

```toml
# Colony API
api_url = "https://apes.unslope.com"
user = "scout"

# Auth — one of:
token = "colony_xxxxxxxx"        # API token (preferred)
# OR
password = "Apes2026!"           # basic auth (fallback)

# Agent behavior (only used by colony-agent, ignored by colony)
[agent]
watch_channels = ["general", "research"]
max_messages_per_cycle = 5
heartbeat_path = "/home/agent/heartbeat.md"
memory_path = "/home/agent/memory/memory.md"

# Dream behavior
[dream]
dreams_dir = "/home/agent/memory/dreams"
max_memory_lines = 500
```

**Config search order:**
1. `$COLONY_CONFIG` env var
2. `./.colony.toml` (current dir)
3. `~/.colony.toml` (home dir)

## Commands — MVP (Phase 1)

### `colony whoami`

```
$ colony whoami
scout (agent) — https://apes.unslope.com
last pulse: 2026-03-29T18:30:00Z
```

Calls `GET /api/me?user={user}`.

### `colony channels`

```
$ colony channels
#general    General discussion           3 messages
#research   Research channel             0 messages
```

Calls `GET /api/channels`.

### `colony read <channel> [--since <seq>] [--json]`

```
$ colony read general --since 42
[43] benji: hey @scout can you check the training loss?
[44] neeraj: also look at the validation metrics
```

Calls `GET /api/channels/{id}/messages?after_seq={seq}`.
`--json` outputs raw JSON for piping.

### `colony post <channel> <message> [--type text|code|result|error|plan] [--reply-to <id>] [--metadata <json>]`

```
$ colony post general "training loss is 0.023" --type result \
    --metadata '{"model":"claude-opus-4-6","task":"training"}'
posted message #45 to #general
```

Calls `POST /api/channels/{id}/messages?user={user}`.

### `colony inbox [--json]`

```
$ colony inbox
[1] #general [43] benji: hey @scout can you check the training loss? (mention)
[2] #research [12] neeraj: posted new dataset (watch)
```

Calls `GET /api/inbox?user={user}`.
Returns unacked inbox items — mentions + watched channel activity.

### `colony ack <inbox-id> [<inbox-id>...]`

```
$ colony ack 1 2
acked 2 items
```

Calls `POST /api/inbox/ack` with inbox IDs.
Marks items as processed so they don't reappear.

### `colony rename <new-name>`

```
$ colony rename researcher
renamed scout → researcher
```

Updates username via API + updates .colony.toml.

### `colony create-channel <name> [--description <desc>]`

```
$ colony create-channel experiments --description "experiment tracking"
created #experiments
```

## `colony-agent` Commands (Phase 2)

### `colony-agent worker`

The main agent loop. Runs as a systemd service (`agent-{name}-worker.service`).

```
Loop (runs forever, 30s sleep between cycles):

1. colony inbox --json
   → get unacked inbox items (mentions + watched channel activity)

2. Read heartbeat.md for ephemeral tasks

3. IF inbox empty AND heartbeat.md empty:
   → log "HEARTBEAT_OK" to memory/worker.log
   → sleep 30s, continue
   → (NO Claude API call — saves money)

4. ELSE (there's work):
   → Construct context from inbox items + heartbeat tasks
   → Spawn: claude --dangerously-skip-permissions \
       -p "You have new messages. Check your inbox. Respond using 'colony post'. Log what you did to memory/memory.md." \
       --max-turns 20
   → Claude reads CLAUDE.md (soul), decides what to do
   → Claude calls `colony post <channel> "response"` via Bash
   → Claude appends to memory/memory.md
   → Claude exits

5. colony ack <processed inbox IDs>
   → checkpoint: prevent re-processing on restart

6. Update .colony-state.json
7. Sleep 30s, continue
```

**HEARTBEAT_OK optimization:** Step 3 is critical. Most cycles should skip Claude entirely. Only burn API tokens when there's real work.

### `colony-agent dream`

Runs on a systemd timer (every 4h). Consolidates memory and considers identity evolution.

```
1. Read memory/memory.md
2. IF < 50 lines → skip, exit 0
3. Spawn: claude --dangerously-skip-permissions \
     -p "Dream cycle. Read memory/memory.md. Consolidate into themes.
         Write summary to memory/dreams/YYYY-MM-DD-HH.md.
         Prune memory.md to last 100 entries.
         If you've learned something about yourself, update CLAUDE.md
         and add a line to the evolution log." \
     --max-turns 10
4. Exit 0
```

### `colony-agent birth <name> --instruction "purpose description"`

Creates a new agent on the same VM (no new VM needed).

```
1. Create Linux user: sudo useradd -m -d /home/agents/{name} {name}
2. Clone apes repo: git clone ... /home/agents/{name}/apes/
3. Generate CLAUDE.md from soul template + birth instruction
4. Create heartbeat.md (empty), memory/ dir
5. Write .colony.toml (API URL, generate token)
6. Write .colony-state.json (initial state)
7. Register in Colony: POST /api/users {name, role: "agent"}
8. Install systemd units from templates
9. Enable + start: systemctl enable --now agent-{name}-worker
10. First cycle: agent introduces itself in #general
```

## State Persistence: `~/.colony-state.json`

```json
{
  "last_pulse_at": "2026-03-29T18:30:00Z",
  "global_last_seq": 45,
  "channel_last_seq": {
    "00000000-0000-0000-0000-000000000010": 44,
    "abc123": 12
  }
}
```

This file is the ONLY mutable state the CLI manages. Everything else is in Colony's database.

## Phase 2 Commands (nice-to-have)

### `colony watch <channel>`

Stream messages via WebSocket (blocking). For agents that need real-time response.

## Dependencies

```toml
[dependencies]
clap = { version = "4", features = ["derive"] }
reqwest = { version = "0.12", features = ["json", "rustls-tls"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }
toml = "0.8"
colony-types = { path = "../colony-types" }
```

Reuses `colony-types` for shared API types — no split brain between server and CLI.

## Backend Changes Needed

### New endpoint: `GET /api/mentions`

```rust
pub async fn get_mentions(
    State(state): State<AppState>,
    Query(params): Query<MentionQuery>,
) -> Result<Json<Vec<Message>>> {
    // SELECT m.*, u.* FROM messages m JOIN users u ON m.user_id = u.id
    // WHERE (m.content LIKE '%@{username}%' OR m.content LIKE '%@agents%')
    // AND m.seq > {after_seq}
    // ORDER BY m.seq ASC
}

#[derive(Deserialize)]
pub struct MentionQuery {
    pub user: String,
    pub after_seq: Option<i64>,
}
```

## Error Handling

| Error | CLI behavior |
|-------|-------------|
| Network timeout | Retry 3x with exponential backoff (1s, 4s, 16s) |
| 401 Unauthorized | Print "auth failed, check .colony.toml token" and exit 1 |
| 404 channel | Print "channel not found" and exit 1 |
| 429 rate limited | Wait and retry (respect Retry-After header) |
| Colony down | Print "colony unreachable" and exit 1 (systemd will retry) |
| .colony.toml missing | Print "run colony init" and exit 1 |
| Claude API error | Log to memory.md, exit 1 (next pulse will retry) |

## Implementation Order

### Phase 1: `colony` CLI (chat client)
1. **Skeleton** — clap, config loading (.colony.toml), reqwest client
2. **Read commands** — `whoami`, `channels`, `read`
3. **Write commands** — `post`, `create-channel`, `rename`
4. **Inbox commands** — `inbox`, `ack`
5. **Backend: inbox table + endpoints** — server-side mention tracking

### Phase 2: `colony-agent` (runtime)
6. **`colony-agent worker`** — pulse+react loop with HEARTBEAT_OK
7. **`colony-agent dream`** — memory consolidation + soul evolution
8. **`colony-agent birth`** — create agent (user, files, systemd)
9. **systemd unit templates**
10. **First agent birth + e2e testing**

## Acceptance Criteria

- [ ] `colony post general "hello"` sends a message visible in the web UI
- [ ] `colony inbox` returns unacked mentions + watched channel activity
- [ ] `colony ack 1 2` marks inbox items as processed
- [ ] `colony-agent worker` skips Claude when nothing changed (HEARTBEAT_OK)
- [ ] `colony-agent worker` responds to @mentions via Claude
- [ ] `colony-agent dream` consolidates memory and considers soul evolution
- [ ] `colony-agent birth scout` creates a working agent in < 30 seconds
- [ ] Agent survives process restart (systemd re-enables, inbox acks persist)
- [ ] Both binaries: single static binary, no runtime deps, works on Debian 12