Getting the most out of Tawa Guard

This guide assumes a scan is already wired into your agent. It covers the decisions around the call: what each error costs, how a flag should change the agent, and how you check the numbers on your own traffic.

1. Price the error on each surface

An agent reads from several places, and a wrong decision costs something different at each one. Start from who can write to the surface and who pays when the detector is wrong.

The defaults follow that asymmetry. Where a person is waiting, an uncertain result blocks. Inside the agent loop it passes with decision.needs_review set, because an unattended refusal costs more than a flag you can read later. Each default is a Block or Allow setting per content type in the app's settings, and decision.action always arrives as allow or block. Tool calls are the exception to the table: they are monitored by default, so decision.action is allow and the real verdict is stored on the job until you set Tool calls to Enforce (beta) under Scanning and blocking in the app's settings. Blocking is opt-in because a text pattern cannot tell whether a tool call was something the user authorised.

2. Let a flag change what the agent can do

Withholding one tool result is the weakest response. The flagged text is gone, but the agent still holds every tool it had, and its next step may act on the same intent. A stronger design treats a flag as taint on the turn and removes the tools that matter.

from tawaguard import TawaGuard

client = TawaGuard(api_key="tawa_sk_...")
HIGH_IMPACT = {"send_email", "transfer_funds", "delete_record"}

class Turn:
    tainted = False

def run_tool(turn, name, args):
    if turn.tainted and name in HIGH_IMPACT:
        return "Unavailable: an earlier result in this turn was flagged."

    call = client.scan_tool_call(name, args)
    if call.action == "block":
        return "Tool call withheld."

    output = tools[name](**args)
    result = client.scan_tool_output(tool_name=name, output=output)

    if result.action == "block":
        turn.tainted = True
        return "Tool result withheld."
    if result.needs_review:
        turn.tainted = True
        queue_for_review(name, output, result)
    return output

Tool calls are monitored by default, so call.action stays allow and the first branch only runs once you set Tool calls to Enforce (beta). Tool output is allowed through with the flag by default, so the last branch is where most flags surface. Scanning the call as well as the result matters for the same reason: an injected instruction usually shows up as a call the user never asked for. Tawa Guard returns the decision, and the taint, the tool list and the failure behaviour are yours. Decide what happens when a scan call fails. An agent that can send, pay or delete should fail closed.

3. Size the review queue before you set the defaults

Every Allow setting turns uncertain results into a queue, and every Block setting turns them into refusals. Choose from capacity, not instinct.

1. Turn on shadow mode for the app. Requests are scanned and logged, action stays allow, and verdicts still appear on each job, tagged as shadow.
2. After a week, filter Jobs by verdict, choose Needs review, and count flags per surface.
3. Multiply. As an illustration, 50,000 tool results a day at a 2% flag rate is 1,000 flags a day, about eight hours of review at 30 seconds each. If you cannot read that many, Block is cheaper for that surface than a queue nobody clears.
4. Route flags to a second check before a person: a second model, or a rule on the tool. Mark outcomes correct or incorrect on the job, so the next sample starts from labelled data.

4. Pin your logic to the stable fields

action and needs_review are the contract. risk and score_breakdown are diagnostic, and their values can move between releases while the shape stays the same. If you cut on risk yourself, treat the cut as something you derive and re-derive.

Take your benign sample from step 3, read its risk values, and set the cut where the share of flagged benign requests equals the budget you can review. That is the matched-budget method behind the benchmark, applied to your traffic. Store risk in your own log next to the job id, handle null (it is absent for some inputs), and repeat the derivation after each release. On Starter and above, a policy rule can match on risk_gte together with provenance, which gives each surface its own cut inside Tawa Guard.

5. Keep deterministic constraints out of the detector

Who may call which tool is business logic, and a model should not decide it. Pass policy_overrides on a scan to add rules for that call. A rule has a when, a list of directives and an optional force.

{
  "id": "hold-irreversible-tools",
  "when": { "tool_name_in": ["transfer_funds", "delete_record"] },
  "force": "block",
  "directives": ["block"]
}

Add it only on turns where your own context calls for it, for example when the caller is a guest. A matching force: "block" rule short-circuits every other rule, and the disable_tools directive comes back suffixed with the tool name. The worker only signals directives, so your code enforces them. The split is clean: detection judges the text, and rules state what your application never allows.

6. Scan what the model reads, and scan it once

Scan the assembled prompt as well as each chunk, since the model reads the whole context. Cache the verdict for a result you have already scanned, keyed by content, instead of scanning it again at every step of a turn.

Tawa Units follow the characters you scan, and every scan counts as at least 100 of them, so a stream of 20-character scans costs five times what the text alone suggests. Scan the unit the model reads. Pass a language hint when you know it. If your traffic spans several languages, compare the Multilingual engine, where it is enabled for your workspace, with the default on a sample of your own.

Move slow decisions to ingestion. Knowledge buckets are in beta. Files you upload (PDF, DOCX, Markdown, text, HTML) are scanned before an agent can read them, a document held for a person appears under Needs attention, and a document in review is not searched. Agents read served documents through POST /v1/knowledge/search. Keep the query-time scan as well, for content that arrived another way.

7. Close the loop with your own labels

Log the job_id from every response. Pass a hashed cuid, and tenant_id on multi-tenant apps, so decisions are attributed to a user, and to a customer on multi-tenant apps. Never pass raw personal data as the identifier. When a result is wrong, mark it on the job. Allow or Block on an uncertain result records what you would have done, and a reviewer who later opens a near-identical prompt sees that your team already reported it.

Those jobs become a regression set. On the Business and Enterprise plans, Policy Studio can backtest a draft rule against recorded traffic before you change a rule or apply a profile. Rerun the set after each change, and after each release.

8. Watch it run

Spike alerts, in each app's settings, fire when the calls an app blocks or flags jump past its trailing 30-day average. A jump can mean an attack or a release that changed your prompts, and both deserve a look. On the Business and Enterprise plans, SIEM export sends each event to your collector in CEF or JSON lines with the job id, provenance, app and a needs-review flag, so you can alert on repeat flags from one app or API key. The free Developer plan includes 5 million Tawa Units a month with a hard cap. Paid plans add a usage budget and alerts you set, so volume stays within a limit you know in advance.

What benchmark v13 measured

Benchmark v13 scored the Tawa Guard default engine and four openly available detectors on the same nine datasets, 14,865 rows each, on one machine with the same thread budget and no network. Each detector's threshold comes from its own benign rows, set so that it flags at most 1% of them, and recall is what remains at that budget.

Recall is a macro average over the six datasets that hold both classes. Latency is the median per prompt, in-process, one prompt at a time, on the short-prompt datasets, and it is a detector figure and not an end-to-end one. False positives on benign prompts that contain injection trigger words are scored separately, on NotInject at a fixed 0.50 threshold, and the figures for all five detectors are in the report.

Every figure is a benchmark figure, and none is a production figure. The method is in Benchmark v13, and sections 3, 4 and 7 are how you repeat the measurement on your own traffic.