Skip to content
1Claw Academy
Curriculum/The AI Agent Threat Model2 minBeginner · Lesson 14 of 14

Capstone: audit an agent fleet

Capstone

Apply the whole track to three real agents: find the exploitable one, rank the risk, and say what you would change first.

This pulls the track together. You are handed three agents that a growing team shipped over six months. Each was reviewed and approved at the time. Your job is the one nobody did: look at them together and decide what to fix first.

Tip

Work through it before reading the verdicts. The exercise is the point; the answers are worth much less if you arrive at them by scrolling.

Agent one. A documentation bot that answers questions in a public Slack channel.

Audit this configuration

The docs bot reads questions from a public channel, searches an internal wiki, and replies in the thread. The wiki includes engineering runbooks.

For each capability, mark every circle it contributes. Some contribute none.

  • read_slack_message

    Reads the question posted in the public channel.

  • search_wiki

    Full-text search across the internal engineering wiki, including runbooks.

  • post_slack_reply

    Posts the answer back into the public channel.

Agent two. A deployment agent with production credentials.

Audit this configuration

The deploy agent reads a merged pull request, calls the deploy API through a binding, and writes a status back to the PR. It runs only on merges to main, which requires review approval.

For each capability, mark every circle it contributes. Some contribute none.

  • read_merged_pr

    Reads the title, body and diff of a PR that has already been approved and merged.

  • execute:deploy

    Triggers a deploy through a binding. The credential is injected server-side.

  • post_pr_status

    Writes a status comment back onto the pull request.

Decide

Agent three is a finance assistant. It reads invoices from a shared mailbox, looks up vendor records, and drafts payment instructions that a human approves in the dashboard before anything moves. All three trifecta circles are present.

You have budget to fix one agent this quarter. Which, and why?

Two lessons worth carrying out of this. First, the agent that looks least dangerous was the most exploitable, because nobody scrutinises a documentation bot. Second, the trifecta measures exfiltration and nothing else. The deploy agent's real risk is an unintended action, and you need the excessive-agency lens for that, not this one.

Watch out

A model that answers one question well will quietly answer a different question badly if you let it. Use the trifecta for disclosure and the agency root causes for impact, and run both.

Check your understanding

3 questions
1

The deploy agent has only two circles. What does that tell you about its risk?

2

Why was the documentation bot the right answer despite the finance assistant handling money?

3

What made the documentation bot easy to overlook in review?