Introduction to Akhara Evaluation

Domain-agnostic AI evaluation, rubrics, expert review, CI/CD quality gates, and a hosted dashboard. Works for any vertical; healthcare is one of many.

Why Akhara Evaluation?

You need to know whether your model or agent is getting better, before that change ships. Akhara Evaluation gives you versioned datasets, deterministic and LLM judges, expert rubric workflows, and CI quality gates that block regressions in pull requests.

The platform is domain-agnostic: the same objects (datasets, samples, rubrics, evaluations, reviewers) apply whether you score support agents, RAG systems, voice bots, or clinical workflows. Healthcare coverage (voice, clinical notes, imaging) is an optional vertical, not the product definition.

Rubric-based scoring

Deterministic checks, LLM judges, and credentialed expert rubrics, not generic vibe scores alone.

CI/CD quality gates

Run eval suites on every PR. Fail the pipeline when metrics or safety gates regress.

Expert review

Route hard cases to domain experts for grading, adjudication, and ground-truth generation.

Hosted dashboard

Projects, case review, rubrics, and run history in the eval app at app.akhara.ai.

Product stack

Layer	What it is	Surface
Eval dashboard	Hosted UI for projects, cases, rubrics, reviewers, and reports	app.akhara.ai
Akhara SDK & API	Hosted API for datasets, samples, evaluations, scores, and human review	API reference · pip install akhara

How it works

Your model or agent logs outputs as samples. Akhara scores them against rubrics, routes hard cases to expert review, and surfaces results in the dashboard, where a CI gate turns them into a merge or deploy decision.

1. Ingest

   Log model outputs, transcripts, tool trajectories, or batch datasets via the SDK

2. Evaluate

   Score with deterministic checks, LLM judges, adaptive rubrics, or expert workflows

3. Gate

   Enforce thresholds in CI, block merge or deploy when gates fail

4. Review & iterate

   Drill into failures in the dashboard; compare versions; tighten rubrics

Quick start

Log a sample and start an evaluation:

quickstart.py

from akhara import Akhara

client = Akhara(api_key="your-api-key")

client.samples.create(
    dataset="support-agent-golden",
    input={"query": "I need to cancel my order and get a refund."},
    output={
        "response": "I can help with that. I've started the cancellation and refund.",
        "actions": ["cancel_order", "issue_refund"],
    },
    expected={
        "must_include": ["cancellation", "refund"],
        "must_not": ["upsell"],
    },
)

evaluation = client.evaluations.create(
    name="support-agent-regression",
    dataset="support-agent-golden",
    evaluators=[
        {"type": "contains_all", "config": {"fields": ["must_include"]}},
        {"type": "llm_judge", "config": {"rubric": "task_completion_v1"}},
    ],
)

print(evaluation.id)

For CI quality gates with the same SDK, see the CI/CD tutorial.

Use cases

Use case	What you evaluate
Agent / tool use	Task completion, tool-path correctness, policy adherence
RAG / Q&A	Groundedness, citation quality, answer completeness
Voice & multimodal	Transcript quality, intent accuracy, escalation behavior
Safety & compliance	Red-flag / policy misses, refusal quality, audit evidence
CI regression	Metric floors, max regression vs production baseline
Healthcare (optional vertical)	Triage, clinical NLP, imaging: same platform, specialized rubrics

Next steps

Platform capabilities

Datasets, rubrics, expert review, and the hosted dashboard

CI/CD evaluations

Wire quality gates into GitHub Actions or GitLab CI

Quickstart

Install the SDK and run your first evaluation

API reference

Datasets, evaluations, scores, and auth