Running

Two ways an agent runs under Akhara: offline evals and live runtime enforcement.

There are two ways an agent “runs” under Akhara: offline evals (batch, graded by verifiers) and live runtime enforcement (the PEP gating a real session).

Offline eval runs

# Run a whole suite against an agent
akhara eval tasks.py --agent checkout-agent

# Run a single task
akhara eval run --task vu.checkout.hard.t131

Each run produces a trace under runs/<slug>/ with a summary.json (akhara.harbor.trace_export.v1) plus per-step screenshots and verifier results. The index (runs/index.json) records:

{
  "id": "…",
  "slug": "cvs_t131_…",
  "task_id": "vu.checkout.hard.t131",
  "env_id": "env_cvs_gym",
  "agent": "checkout-agent",
  "final_score": 1.0,
  "status": "PASS",
  "step_count": 19,
  "verdicts": { "VALID_UI": "PASS", "VU-CHECKOUT": "PASS" }
}

A step captures the action, what the agent saw, and how each verifier graded it:

{
  "turn": 4,
  "step_index": 4,
  "action_type": "click",
  "action_params": { "point_2d": [630, 578] },
  "screen_id": "SEARCH_RESULTS",
  "screenshot_path": "screenshots/step_04.png",
  "step_reward": 0.04,
  "verifier_results": [{ "id": "VALID_UI", "verdict": "PASS" }]
}

Latch steps are flagged in the trace, so you can see exactly where a policy would have stopped or escalated the agent during evaluation.

Live runtime enforcement

In production the PEP wraps the five stages around a real turn.

A full chat turn

// 1. Gate the incoming request
if (!(await pep.checkInput(userText)).mayContinue) return refuse();

// 2. Minimize sensitive data before context leaves for the LLM
const ctx = await pep.checkContextEgress(accountContextJson);
const modelInput = ctx.transformedContent ?? accountContextJson;   // WARN rewrites here

// 3. Gate the model's draft
const out = await pep.checkOutput(draft);
if (!out.mayContinue) return escalateOrDrop(out);

// 4. Final delivery gate
const delivered = await pep.checkDelivery(out.transformedContent ?? draft);
if (!delivered.mayContinue) return escalateOrDrop(delivered);
return show(delivered);

A consequential action

The permit pattern: the side-effecting service refuses to run without a fresh permit.

const permit = await pep.authorizeAction("refund_payment", { orderId, amount });

if (permit.verdict === "ALLOW") {
  await paymentService.submit(orderId, amount, permit.permitId!); // permit required
} else {
  // BLOCK    → tell the customer it can't be done here
  // ESCALATE → hand off to a supervisor queue
}

Everything is evidence

Every authorize call (allow, warn, block, escalate) is written to the enforcement feed as a policy_decision event:

{
  "ts": 0,
  "type": "policy_decision",
  "data": {
    "agentId": "support-ai",
    "surface": "Support Copilot",
    "verdict": "BLOCK",
    "stage": "output",
    "rule": "PCI DSS Requirement 3.3",
    "policyId": "pci-0",
    "policyMatch": "Unmasked PAN in output",
    "original": "…", "final": "…",
    "delivery": "Response or action withheld",
    "evidence": "Logged",
    "permitId": null,
    "attachedPolicyIds": ["latch-0","…","pci-0"]
  }
}

That feed is your audit trail: nothing an agent does leaves without a record. Decisions can be sealed and later checked with the verification-record endpoints. For the full model (audit trail, signed records, trajectories), see Audit and evidence.

SDK requirements

Runtimes, versions, network, and configuration the PEP needs.