349 lines
15 KiB
Markdown
349 lines
15 KiB
Markdown
# 𝕿𝖍𝖊 𝕰𝖓𝖉 — The Fable Has Been Told
|
||
|
||
> The complete build logic of **Fablino**: Anthropic-ecosystem-only multi-model fusion for
|
||
> fable-agent. Every changed line, every fix, the full multi-agent orchestration that built it,
|
||
> and the verified proof it works. One file. The whole fable.
|
||
>
|
||
> Built 2026-06-14 on Opus 4.8 (1M) with Claude-only subagents — Sonnet drafted, Opus judged.
|
||
|
||
---
|
||
|
||
## I. What Was Built
|
||
|
||
Fable-agent already existed: a 111-file TypeScript self-improving agent harness with a *fusion*
|
||
module (panel of models → judge → synthesis) that **didn't actually work** and depended on a dead
|
||
local proxy. This build made fusion real and 100% Anthropic:
|
||
|
||
- **Reroute** — every panelist + the judge now shell the local `claude` Claude Code CLI (desktop
|
||
auth). No API key, no OpenRouter, no `127.0.0.1:18901` proxy.
|
||
- **New panel `sonnet-opus`** — Sonnet 4.6 + Opus 4.8 draft independently, Opus 4.8 judges. Now the
|
||
`fusion run` default.
|
||
- **4 bugs fixed**, including the critical one that made fusion silently never fuse.
|
||
- **1 regression test** guarding the critical fix (proven red-on-bug).
|
||
- **DEP0190 warning killed.**
|
||
|
||
---
|
||
|
||
## II. The Journey (multi-agent build logic)
|
||
|
||
Each phase was deterministic multi-agent orchestration via the Workflow harness — Claude-only,
|
||
Sonnet + Opus fused.
|
||
|
||
| Phase | Agents | What happened |
|
||
|---|---|---|
|
||
| **Recon** | 16 (9 Sonnet mappers → build-verify → 5 Opus critics → Opus synth) | Mapped all 111 files, verified the base build, surfaced 48 findings incl. the critical `fuse()` inversion → wrote `BUILD.md`. |
|
||
| **Design** | brainstorming | Locked the Anthropic-only reroute + bug-fix scope. User-approved. |
|
||
| **De-risk** | inline | Proved `claude -p --model <m>` returns `PONG` for Sonnet + Opus *before* building on it. |
|
||
| **Build** | 5 (3 Sonnet builders, disjoint files → Opus integration review → Opus build-fix loop) | All builders done; review **pass**; build green round 1; 50/50 tests. |
|
||
| **Live-fix** | inline | First `fusion run` failed (Windows `.cmd` shim) → rewrote helper (spawn+shell+stdin) → real fusion. |
|
||
| **Guard** | inline | Added `fuse()` regression test; proved RED on the restored bug, GREEN on the fix. |
|
||
| **Polish** | inline | Killed the DEP0190 warning (single-string command + model regex-guard). |
|
||
|
||
The two orchestration scripts (the literal build logic of phases Recon and Build) are saved at:
|
||
- `…/.claude/projects/…/workflows/scripts/fable-agent-reorchestrate-wf_c6359e7c-97b.js`
|
||
- `…/.claude/projects/…/workflows/scripts/fablino-anthropic-fusion-build-wf_4c3c77a2-5cb.js`
|
||
|
||
Their structure: `phase('Build')` → `parallel([builder1, builder2, builder3])` (Sonnet, disjoint
|
||
file ownership, shared contract) → `phase('Review')` Opus integration critic → `phase('Verify')`
|
||
`while (round < 3) { build-fix agent; if green break }`. Agent variety + loop-until-green.
|
||
|
||
---
|
||
|
||
## III. THE BUILD LOGIC (exact final source)
|
||
|
||
### 1. `src/core/claude-cli-caller.ts` — the Anthropic-only call primitive (NEW, full)
|
||
|
||
```ts
|
||
/**
|
||
* Claude CLI Caller — shells out to `claude --print --model <model>` with the prompt on stdin.
|
||
*
|
||
* Uses Claude Code desktop auth; no API key required. The prompt is piped via stdin
|
||
* (not passed as an argument) so multi-line / quoted prompts need no escaping. The
|
||
* command is passed as a SINGLE string (not a program + args array) with `shell: true`
|
||
* so the shell resolves the `claude` / `claude.cmd` shim AND Node does not emit the
|
||
* DEP0190 warning (which only fires for an args array combined with shell:true). The
|
||
* only interpolated value is `model`, which is regex-guarded to a safe id below.
|
||
*/
|
||
|
||
import { spawn } from "node:child_process";
|
||
|
||
export async function callClaudeCli(model: string, prompt: string): Promise<string> {
|
||
return new Promise((resolve, reject) => {
|
||
// Guard: model ids come from FUSION_PANELS constants, but validate anyway so the
|
||
// single-string shell command below is injection-proof regardless of caller.
|
||
if (!/^[a-zA-Z0-9._-]+$/.test(model)) {
|
||
reject(new Error(`callClaudeCli: unsafe model id "${model}"`));
|
||
return;
|
||
}
|
||
const child = spawn(`claude --print --model ${model}`, {
|
||
shell: true,
|
||
timeout: 120000,
|
||
});
|
||
|
||
let stdout = "";
|
||
let stderr = "";
|
||
child.stdout.on("data", (d) => { stdout += d.toString(); });
|
||
child.stderr.on("data", (d) => { stderr += d.toString(); });
|
||
child.on("error", (err) => {
|
||
reject(new Error(`claude CLI spawn failed: ${err.message}`));
|
||
});
|
||
child.on("close", (code) => {
|
||
if (code === 0) {
|
||
resolve(stdout.trim());
|
||
} else {
|
||
reject(new Error(`claude CLI exited with code ${code}${stderr ? ` | stderr: ${stderr.slice(0, 400)}` : ""}`));
|
||
}
|
||
});
|
||
|
||
child.stdin.write(prompt);
|
||
child.stdin.end();
|
||
});
|
||
}
|
||
```
|
||
|
||
### 2. `src/core/fusion-types.ts` — the `sonnet-opus` panel (exact)
|
||
|
||
```ts
|
||
// added to the FusionPanelSlug union:
|
||
| "sonnet-opus" // Sonnet 4.6 + Opus 4.8 via Claude CLI (no proxy)
|
||
|
||
// added to FUSION_PANELS:
|
||
"sonnet-opus": {
|
||
slug: "sonnet-opus",
|
||
modelIds: ["claude-sonnet-4-6", "claude-opus-4-8"],
|
||
judgeModel: "claude-opus-4-8",
|
||
maxTokensPerPanelist: 8192,
|
||
panelistTimeoutMs: 120_000,
|
||
},
|
||
```
|
||
|
||
### 3. `src/examples/executors/fusion-executor.ts` — the fix + reroute (exact excerpts)
|
||
|
||
**Constructor — judge override now applied:**
|
||
```ts
|
||
constructor(config?: Partial<FusionExecutorConfig>) {
|
||
this.config = { panel: "opus4.8-4.8", verbose: false, ...config };
|
||
this.panel = {
|
||
...FUSION_PANELS[this.config.panel],
|
||
judgeModel: this.config.judgeModelOverride ?? FUSION_PANELS[this.config.panel].judgeModel,
|
||
};
|
||
this.safetyBoundary = this.config.safetyBoundary ?? new SafetyBoundary();
|
||
}
|
||
```
|
||
|
||
**`fuse()` — THE CRITICAL FIX (synthesize on ≥2 answers, not on `blindSpots`):**
|
||
```ts
|
||
async fuse(task: string, panelSlug?: FusionPanelSlug): Promise<FusionResult> {
|
||
const panel = panelSlug ? FUSION_PANELS[panelSlug] : this.panel;
|
||
const startTime = Date.now();
|
||
|
||
// 1. DRAFT — dispatch panelists in parallel
|
||
const panelResults = await this.dispatchPanel(task, panel);
|
||
// 2. CRITIQUE — analyze panelist responses
|
||
const analysis = this.analyzePanel(panelResults);
|
||
// 3. FUSE — synthesize final answer whenever >=2 panelists succeeded
|
||
const answered = panelResults.filter(p => p.raw && p.raw.length > 0 && !p.error);
|
||
const finalAnswer = answered.length > 1
|
||
? await this.synthesizeFinal(task, panelResults, analysis)
|
||
: (answered[0]?.raw ?? "No panelists returned results.");
|
||
|
||
return { task, panelSlug: panel.slug, panelSize: panel.modelIds.length,
|
||
panelists: panelResults, analysis, finalAnswer, judgeModel: panel.judgeModel,
|
||
totalDurationMs: Date.now() - startTime, error: null };
|
||
}
|
||
```
|
||
|
||
**`makeModelCall()` — defaults to the claude CLI (no proxy in the default path):**
|
||
```ts
|
||
private async makeModelCall(modelId: string, prompt: string, maxTokens: number): Promise<string> {
|
||
if (this.config.callModel) {
|
||
return this.config.callModel(modelId, prompt, maxTokens);
|
||
}
|
||
this.totalCalls++;
|
||
const prefix = `[Fusion call #${this.totalCalls} to ${modelId}]`;
|
||
if (this.config.verbose) console.error(`${prefix} sending ${prompt.length} chars…`);
|
||
if (modelId === "openrouter-fusion") return `${prefix} OpenRouter Fusion not available via CLI`;
|
||
try {
|
||
const result = await callClaudeCli(modelId, prompt); // ← Anthropic-only default
|
||
return result;
|
||
} catch (e) {
|
||
const msg = (e as Error).message;
|
||
console.error(`${prefix} ERROR: ${msg}`);
|
||
return `${prefix} Error: ${msg}`;
|
||
}
|
||
}
|
||
```
|
||
|
||
> **The bug, in one line:** the old `fuse()` had `analysis.blindSpots.length > 0 ? synthesize : panelResults[0].raw`. Because `blindSpots` is empty when all panelists succeed, the judge ran *only on failure* — the happy path silently returned panelist #0's raw draft. The default panel never fused.
|
||
|
||
### 4. `src/tier2-primitives/loops/state-accumulator.ts` — `resetSession` (exact)
|
||
|
||
```ts
|
||
/**
|
||
* Reset in-memory state and switch to a new sessionId so subsequent
|
||
* record() calls persist to a fresh log file, not the previous run's.
|
||
* The old persisted log is untouched (load() can still read it by id).
|
||
*/
|
||
resetSession(newSessionId: string): void {
|
||
this.iterations = [];
|
||
this.sessionId = newSessionId;
|
||
}
|
||
```
|
||
|
||
### 5. `src/tier2-primitives/loops/feedback-loop.ts` — per-run rotation + single convergence (exact)
|
||
|
||
```ts
|
||
// fields added:
|
||
private store: StateStore | undefined;
|
||
private baseSessionId: string | undefined;
|
||
|
||
async run(task: string, executor: PhaseExecutor): Promise<LoopResult> {
|
||
// Rotate to a fresh per-run session so each run persists to its own log.
|
||
const runId = StateStore.uid();
|
||
const runSessionId = this.baseSessionId ? `${this.baseSessionId}-${runId}` : runId;
|
||
this.accumulator.resetSession(runSessionId);
|
||
const startTime = Date.now();
|
||
let iterationNumber = 0;
|
||
|
||
while (iterationNumber < this.maxIterations) {
|
||
iterationNumber++;
|
||
const iteration = await this.executeIteration(iterationNumber, task, executor);
|
||
|
||
// Record, THEN compute the authoritative convergence verdict (once).
|
||
this.accumulator.record(iteration);
|
||
const verdict = this.convergence.evaluate(this.accumulator, this.maxIterations);
|
||
// Stamp the phase now that we have the real verdict.
|
||
iteration.phase = verdict.converged ? "converged" : "refine";
|
||
await this.fireHooks("onIteration", iteration, iterationNumber);
|
||
if (verdict.converged) { /* … return converged result … */ }
|
||
// …
|
||
}
|
||
}
|
||
```
|
||
|
||
### 6. `src/index.ts` — the fusion command wiring (exact)
|
||
|
||
```ts
|
||
fusion
|
||
.command("run <task>")
|
||
.description("Run a task through a fusion panel of models")
|
||
.option("-p, --panel <slug>", "Panel: sonnet-opus, opus4.8-4.8, …", "sonnet-opus") // ← default
|
||
.option("-j, --judge <model>", "Override judge model")
|
||
.option("--openrouter", "Use OpenRouter Fusion API instead of local panel dispatch")
|
||
.action(async (task, opts) => {
|
||
if (opts.openrouter) { /* … unchanged OpenRouterFusionExecutor path … */ return; }
|
||
|
||
const { FusionExecutor } = await import("./examples/executors/fusion-executor.js");
|
||
const { callClaudeCli } = await import("./core/claude-cli-caller.js");
|
||
const executor = new FusionExecutor({
|
||
panel: (opts.panel as any) ?? "sonnet-opus",
|
||
judgeModelOverride: opts.judge,
|
||
callModel: (model, prompt) => callClaudeCli(model, prompt), // ← Anthropic-only injection
|
||
verbose: true,
|
||
});
|
||
const result = await executor.fuse(task, opts.panel as any);
|
||
// …print result…
|
||
});
|
||
```
|
||
|
||
### 7. `src/examples/executors/fusion-executor.test.ts` — the regression guard (NEW, full)
|
||
|
||
```ts
|
||
import { describe, it, expect } from "vitest";
|
||
import { FusionExecutor } from "./fusion-executor.js";
|
||
|
||
const JUDGE_SENTINEL = "FUSED_SYNTHESIS_OUTPUT";
|
||
|
||
function makeMockCallModel(draftFor: (modelId: string) => string) {
|
||
const calls: Array<{ modelId: string; isJudge: boolean }> = [];
|
||
const callModel = async (modelId: string, prompt: string, _maxTokens: number): Promise<string> => {
|
||
const isJudge = prompt.includes("You are the judge");
|
||
calls.push({ modelId, isJudge });
|
||
return isJudge ? JUDGE_SENTINEL : draftFor(modelId);
|
||
};
|
||
return { callModel, calls };
|
||
}
|
||
|
||
describe("FusionExecutor.fuse() — judge synthesis", () => {
|
||
it("runs the judge and returns its synthesis when >=2 panelists answer", async () => {
|
||
const { callModel, calls } = makeMockCallModel((id) => `draft-from-${id}`);
|
||
const fusion = new FusionExecutor({ panel: "sonnet-opus", callModel });
|
||
const result = await fusion.fuse("why is the sky blue?");
|
||
expect(calls.filter((c) => c.isJudge)).toHaveLength(1);
|
||
expect(result.finalAnswer).toBe(JUDGE_SENTINEL);
|
||
expect(result.finalAnswer).not.toContain("draft-from-");
|
||
});
|
||
|
||
it("does NOT run the judge when fewer than 2 panelists answer", async () => {
|
||
const { callModel, calls } = makeMockCallModel((id) =>
|
||
id === "claude-sonnet-4-6" ? "" : `draft-from-${id}`);
|
||
const fusion = new FusionExecutor({ panel: "sonnet-opus", callModel });
|
||
const result = await fusion.fuse("why is the sky blue?");
|
||
expect(calls.filter((c) => c.isJudge)).toHaveLength(0);
|
||
expect(result.finalAnswer).toBe("draft-from-claude-opus-4-8");
|
||
});
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## IV. The Bugs & Fixes
|
||
|
||
| # | Bug | Severity | Fix |
|
||
|---|-----|----------|-----|
|
||
| 1 | `fuse()` gated synthesis on `blindSpots` → judge never ran on success → returned panelist #0's raw draft | 🔴 critical | Synthesize when `answered.length > 1` |
|
||
| 2 | `StateAccumulator.reset()` left store/sessionId → runs bled into one log | 🟠 high | `resetSession(newId)` + per-run rotation in `run()` |
|
||
| 3 | `executeIteration` computed phase from stale state + double `evaluate()` | 🟠 high | Record first, evaluate once, stamp phase from real verdict |
|
||
| 4 | `-j/--judge` flag accepted but never applied | 🟡 minor | Apply `judgeModelOverride` in constructor |
|
||
| 5 | `execFile("claude")` couldn't launch the Windows `.cmd` shim → instant fail | 🟠 live | `spawn` + `shell:true` + prompt via stdin |
|
||
| 6 | DEP0190 warning (args-array + shell) | 🟡 cosmetic | Single-string command + `model` regex guard |
|
||
|
||
---
|
||
|
||
## V. Rebuild & Verify
|
||
|
||
```bash
|
||
cd "C:/Users/117tr/OneDrive/Desktop/fable-agent-master/fable-agent"
|
||
npm install # first time only
|
||
npm run build # tsc → dist/ (expect zero output, zero errors)
|
||
|
||
node dist/index.js fusion panels # sonnet-opus listed
|
||
node dist/index.js fusion run "<your hard question>" # Sonnet+Opus → Opus judge, all via claude CLI
|
||
npm test # 8 files / 52 tests
|
||
```
|
||
|
||
Only prerequisite for the Anthropic-only path: the `claude` CLI on PATH, signed into Claude Code.
|
||
|
||
---
|
||
|
||
## VI. Verified Proof (ground truth, 2026-06-14)
|
||
|
||
```
|
||
npm run build → tsc clean, ZERO type errors
|
||
npm test → Test Files 8 passed (8) | Tests 52 passed (52) | vitest v4.1.8
|
||
fusion run → Panelist 1 (claude-sonnet-4-6): ✓ ~14s
|
||
Panelist 2 (claude-opus-4-8): ✓ ~15s
|
||
Final Answer: <Opus judge's structured synthesis> | no DEP0190 lines
|
||
regression → RED (2 fail) when the old blindSpots gating restored; GREEN on the fix
|
||
```
|
||
|
||
Companion docs (same folder / Desktop): `BUILD.md` (full architecture + 48-finding audit),
|
||
`fablino-longest-horizon-ever-build.md` (the reroute build log), `demo 1 fablino.md` (live demo
|
||
script).
|
||
|
||
---
|
||
|
||
## VII. The Moral
|
||
|
||
Green tests lied twice. The build compiled and 50/50 tests passed while fusion was **doubly broken** —
|
||
the `blindSpots` inversion and the Windows shim. Neither is reachable without running the real
|
||
shell-out, and no test did. The multi-agent audit caught the logic bug; only a live `fusion run`
|
||
caught the shim. **Tests prove units behave. Only running the system proves it works.** Now there is
|
||
a test for the logic bug too — and it goes red if anyone forgets.
|
||
|
||
---
|
||
|
||
*Sonnet drafted. Opus judged. Opus 4.8 (1M) orchestrated. No key left the Anthropic ecosystem.*
|
||
|
||
**The end. The fable has been told.** 🔒
|