192 lines
12 KiB
Markdown
192 lines
12 KiB
Markdown
# Loop Engineering Meta-Prompt
|
|
|
|
Use this meta-prompt for every TAC Product Factory / Plan F3 plan that touches agents, deploy, RSI, or self-improvement.
|
|
|
|
## Required loop sections
|
|
|
|
Every plan must explicitly map the work to four loops:
|
|
|
|
1. **Agent loop** — what agent/tool loop runs until done.
|
|
2. **Verification loop** — what deterministic checks, reviewers, receipts, or rubrics gate success.
|
|
3. **Event-driven loop** — what webhook, cron, queue, file event, or dashboard trigger runs the work without manual prompting.
|
|
4. **Hill-climbing loop** — what trace/evidence can safely modify prompts, skills, tools, or config later.
|
|
|
|
## Required safety fields
|
|
|
|
Every plan/receipt must include:
|
|
|
|
- Canonical deploy route: `git-proxy:8099/deploy`.
|
|
- Legacy note: `deploy-webhook:8098` is internal/stale for guarded product deploy.
|
|
- `auto_patch_proven`: `false` unless a degraded-skill recovery receipt exists.
|
|
- `rsi_canary_recovery_evidence`: `true` only when linking the canary receipt.
|
|
- Evidence receipt paths for any RSI/self-improvement claim.
|
|
|
|
## Required validation
|
|
|
|
At minimum, plans must name the checks that prove:
|
|
|
|
- plan schema/sections validate;
|
|
- deploy route is signed-action-only;
|
|
- tests pass;
|
|
- claims do not exceed receipts.
|
|
|
|
## Taste rule
|
|
|
|
Prefer the boring loop that compounds over the clever prompt that works once. If the plan needs a new framework, first prove a markdown file, JSON receipt, and pytest assertion cannot hold the invariant.
|
|
|
|
## Agent harness note
|
|
|
|
A loop is not just repeated model calls. Every agent plan must name the harness around the model:
|
|
|
|
- context and working memory;
|
|
- durable semantic memory / RAG sources;
|
|
- episodic memory / traces from prior runs;
|
|
- tool allowlist and signed deploy boundaries;
|
|
- end-loop guardrails that define when to stop;
|
|
- tracing for retrievals, tool calls, latency, token use, and errors;
|
|
- evals/receipts that decide whether prompt or config changes can feed back into the next run.
|
|
|
|
LLMOps/hill-climbing only promotes a changed prompt, retrieval config, tool config, or model parameter when traces plus evals show the new loop is safer or better.
|
|
|
|
Reference: Sean's AI Stories, "You Can Learn AI Agent Harness & Loop Engineering In 19 Min" (YouTube `GrNbuWWJYiI`).
|
|
|
|
## Open-source model portability note
|
|
|
|
A strong harness should make the task portable across model tiers, including open-source or lower-cost models. Plans must spell out enough context, phases, and quality gates that a non-frontier model can follow the work without relying on hidden taste.
|
|
|
|
For outbound/productized work, require:
|
|
|
|
- explicit prospect/input target;
|
|
- crawler or source gathering step;
|
|
- report generation step;
|
|
- cover letter or executive summary when the output is meant to sell or persuade;
|
|
- design/report review phase;
|
|
- quality gate checklist that loops back to the builder when review fails;
|
|
- model/cost notes when choosing GLM/open-source/local models over frontier models.
|
|
|
|
Reference: Jordan Urbs, "GLM 5.2 Proves Open Source AI Can Match Fable 5 (AI Harness Engineering)" (YouTube `dJI2GRG1GEE`).
|
|
|
|
## Autonomous work agent note
|
|
|
|
For business adoption, distinguish two operating modes before designing the harness:
|
|
|
|
1. **Team manages agents** — an agent factory or admin team creates, updates, and governs shared agents.
|
|
2. **Agents assist people** — each human gets a personal work agent that learns their context, skills, preferences, and second-brain material.
|
|
|
|
Every autonomous work-agent plan must include:
|
|
|
|
- per-agent isolation boundary, preferably one container or equivalent sandbox per agent;
|
|
- credential separation: no raw secrets inside the agent environment;
|
|
- vault/proxy credential injection for only the requests the agent is allowed to make;
|
|
- explicit access policies for tools, files, APIs, and outbound network;
|
|
- skill/instruction iteration loop for tuning the agent's actual output;
|
|
- management surface for upgrading, maintaining, and revoking agents after deployment;
|
|
- second-brain/context source plan when personal-agent behavior depends on private knowledge.
|
|
|
|
Reference: Latent Space, "The Blueprint for Autonomous Work Agents" with Gavriel Cohen / NanoClaw (YouTube `hLUGXO5DSpo`).
|
|
|
|
## Product work with coding agents note
|
|
|
|
When implementation gets cheap, do not delete product discipline. Plans should use prototypes to explore, but still preserve the product loop:
|
|
|
|
- PRD/spec captures why, users, constraints, and success criteria;
|
|
- prototype makes options concrete and testable;
|
|
- every artifact declares its stage: exploration, prototype, beta, or ship candidate;
|
|
- production-looking prototypes must not imply production readiness without receipts;
|
|
- human taste and systems thinking review whether the result fits the whole product;
|
|
- scheduled/background agents may gather context, but promotion still needs receipts and review;
|
|
- role boundaries can flex, but ownership, decision rights, and phase state must stay explicit;
|
|
- model-timing assumptions are explicit: some artifacts are kept to retest when model capability changes;
|
|
- complexity budget is tracked because autonomous loops tend to add code; deletion/simplification is a first-class review criterion.
|
|
|
|
Reference: Lenny's Podcast, "OpenAI Codex lead on the new shape of product work" with Andrew Ambrosino (YouTube `P3KDebPTUrw`).
|
|
|
|
## Activity modes note
|
|
|
|
Treat AI-era role archetypes as activity modes in the delivery loop, not permanent job titles. A person or agent can move across modes, but each mode needs its own invariant:
|
|
|
|
- **Prototyper** — creates options and working artifacts; invariant: stage is marked and prototype output cannot skip PRD/spec or receipts.
|
|
- **Grower** — compounds a rough artifact into a durable product; invariant: every iteration preserves traceability, tests, and user/problem fit.
|
|
- **Sweeper** — deletes, simplifies, migrates, and pays down prototype debt; invariant: complexity budget and deletion/simplification review are explicit.
|
|
- **Reviewer/Taster** — applies human taste, systems thinking, security, and product judgment; invariant: production readiness requires review evidence, not vibes.
|
|
- **Architect/Primitive keeper** — maintains shared primitives, interfaces, and harness boundaries; invariant: tools, credentials, deploy actions, and context sources stay governed.
|
|
- **Operator/SRE** — keeps the loop running in production; invariant: observability, rollback, incident ownership, and revocation paths are named.
|
|
|
|
Do not claim role collapse means expertise is obsolete. The safer claim is that boundaries flex while specialties, ownership, and best practices remain necessary.
|
|
|
|
## Founder learning loop note
|
|
|
|
For product/factory bets, do not treat market stories, macro narratives, or AI demos as strategy by themselves. Convert them into small operating loops:
|
|
|
|
- **Niche/distribution thesis** — name the narrow buyer, geography/channel, hard-to-replace acquisition route, and revenue concentration risk.
|
|
- **Money-flow map** — name what trust, incentive, or regime shift moves budget toward or away from the product.
|
|
- **AI amplifier check** — name what AI improves, what system weakness it does not fix, and which verification/cost signals prove the improvement.
|
|
- **Seven-day learning protocol** — summarize sources, extract one work-relevant idea each, run three micro-experiments, define KPIs, compare to baseline, then kill one / keep one / improve one.
|
|
|
|
Treat these as hypothesis inputs until receipts, customer evidence, or measurable experiment output exist.
|
|
|
|
References: My First Million `OXgqzF382o8`, Anthony Pompliano `bJD-Ftig30M`, Dylan Beattie / NDC `xunx7VvlOgs`.
|
|
|
|
## Insider-threat model note
|
|
|
|
Treat coding agents and frontier-model work agents as potential insider threats once they can read files, call tools, write code, or touch deploy paths. Plans that give an agent business-impacting access must include:
|
|
|
|
- least-privilege access by default, with explicit allowlists for files, tools, APIs, and network;
|
|
- no raw secrets in prompts, logs, workspaces, or agent-visible config;
|
|
- sandboxed execution for generated code and dependency operations;
|
|
- supply-chain checks before running generated install/build/test commands;
|
|
- human approval before any action with business, security, financial, customer, or production risk;
|
|
- append-only logs/receipts for agent decisions, tool calls, approvals, and promoted outputs;
|
|
- attestation that a human reviewed output before promotion to production.
|
|
|
|
Do not trust an agent because it is helpful, local, or model-authored. Trust only the bounded action, its receipt, and the verifier evidence.
|
|
|
|
Reference: Security Fest 2026, Dan Tentler, "Claude is your insider threat now" (YouTube `yvJYw2gR0cU`).
|
|
|
|
## External research anchors
|
|
|
|
Ground agent plans against external production guidance, not just video notes:
|
|
|
|
- NVIDIA Secure Agent Workspace: always-on agents need a managed workspace with credential proxy, enterprise tool access, governance, and runtime controls.
|
|
- Infisical Agent Vault: secrets stay in the vault and requests receive credentials through a proxy; agents must not hold raw secrets.
|
|
- Anthropic agent evals: judge trajectories and environment outcomes, not just final answers.
|
|
- OWASP AI Agent Security Cheat Sheet: treat prompt injection, memory poisoning, tool abuse, sandboxing, and access control as first-class risks.
|
|
|
|
These anchors support TAC's local rule: deploy remains signed-action-only through `git-proxy:8099/deploy`; do not reintroduce raw command deploy paths.
|
|
|
|
References: NVIDIA Secure Agent Workspace (`docs.nvidia.com/enterprise-reference-architectures/secure-agent-workspace-reference-design`), Infisical Agent Vault (`infisical.com/blog/agent-vault-the-open-source-credential-proxy-and-vault-for-agents`), Anthropic "Demystifying evals for AI agents" (`anthropic.com/engineering/demystifying-evals-for-ai-agents`), OWASP AI Agent Security Cheat Sheet (`cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html`).
|
|
|
|
## Model Workspace Protocol note
|
|
|
|
For sequential workflows with human review between stages, prefer folder-structured orchestration before multi-agent framework code. The Model Workspace Protocol pattern treats numbered folders as stages, markdown files as role/context carriers, and local scripts as the boring mechanical layer.
|
|
|
|
Use this when it fits:
|
|
|
|
- `00-intake/` — raw request, sources, constraints.
|
|
- `10-plan/` — Plan F3 artifact and assumptions.
|
|
- `20-build/` — implementation notes and changed files.
|
|
- `30-verify/` — test output, verifier notes, receipts.
|
|
- `40-ship/` — signed deploy request, outcome, rollback notes.
|
|
|
|
Reference: arXiv 2603.16021, "Interpretable Context Methodology: Folder Structure as Agentic Architecture".
|
|
|
|
## Agent multiplexer note
|
|
|
|
Terminal multiplexers built for AI coding agents (cmux, herdr) prove a thesis TAC should learn from: one primary agent can stand up, drive, and tear down a fleet of peer agents and terminals entirely through CLI + socket, with no MCP server required. The agent treats panes/tabs as the unit of parallelism and multiplexer state (working/idle/blocked) as the feedback signal.
|
|
|
|
Use this pattern when it fits:
|
|
|
|
- **Primary agent = orchestrator** — it opens workspaces, sends prompts to peers, observes output, and tears down. Peers are real terminals, not framework objects.
|
|
- **Pane = isolation boundary** — each peer runs in its own real terminal on its own cwd/branch; the multiplexer keeps sessions alive over SSH and across laptop sleep.
|
|
- **State awareness** — working/idle/blocked lets the orchestrator gate handoffs on real progress, not on timeouts.
|
|
- **CLI + socket, not MCP** — orchestration is driven by shell commands and a Unix socket; this keeps the surface small and scriptable.
|
|
|
|
TAC invariants still hold when a multiplexer is the substrate:
|
|
|
|
- panes that touch deploy still route through `git-proxy:8099/deploy` signed actions — the multiplexer is not a deploy bypass;
|
|
- peer agents are insider-threat candidates (see Insider-threat model note): least privilege, no raw secrets, sandboxed execution, human approval for business-impacting actions;
|
|
- every peer spawn/teardown and tool call stays receipted;
|
|
- the primary agent must not gain a raw-command deploy path just because it can drive shells.
|
|
|
|
References: cmux fleet-driving prompts (`github.com/disler/learning-cmux-with-agents`), herdr agent multiplexer (`herdr.dev`), cmux + Claude Code + Pi demo (YouTube `WAFUMBLOjHo`).
|