> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.

# Safety Gating Before Production

> Implement automated safety gates that block deployment of clinical AI models that don't meet safety thresholds. Never ship a model that misses red flags.

## Why Safety Gates?

In healthcare AI, "move fast and break things" can harm patients. Safety gates create hard blocks that prevent unsafe models from reaching production, regardless of business pressure or deployment schedules.

<CardGroup cols={2}>
  <Card title="Zero Critical Failures" icon="shield">
    Block any model that misses a single critical red flag in evaluation.
  </Card>

  <Card title="Regression Prevention" icon="triangle-exclamation">
    Ensure new versions don't perform worse than the current production model.
  </Card>

  <Card title="Audit Trail" icon="lock">
    Document every deployment decision for regulatory compliance.
  </Card>

  <Card title="Automated Enforcement" icon="bolt">
    Gates run automatically in CI/CD - no human can bypass them.
  </Card>
</CardGroup>

## Step 1: Define Safety Thresholds

Establish minimum acceptable performance levels for your clinical AI:

```python title="safety_thresholds.py" theme={null}
from akhara import Akhara, SafetyGate

client = Akhara(api_key="your-api-key")

# Define safety gate configuration
safety_gate = client.safety_gates.create(
    name="triage-production-gate",
    project="patient-triage",

    # Hard blocks - ANY failure stops deployment
    critical_thresholds={
        # Zero tolerance for missing life-threatening conditions
        "critical_red_flag_miss_rate": {
            "max": 0.0,  # 0% - cannot miss ANY critical red flags
            "description": "Missed chest pain with radiation, stroke symptoms, etc."
        },

        # Zero tolerance for dangerous under-triage
        "critical_undertriage_rate": {
            "max": 0.0,  # Cannot send emergencies home
            "description": "Emergent cases triaged as routine"
        },

        # Zero tolerance for medication hallucinations
        "medication_hallucination_count": {
            "max": 0,  # Absolute zero
            "description": "Fabricated medications in recommendations"
        }
    },

    # Minimum thresholds - must meet ALL to pass
    minimum_thresholds={
        "triage_accuracy": {
            "min": 85.0,
            "description": "Overall triage classification accuracy"
        },
        "safety_score": {
            "min": 95.0,
            "description": "Red flag detection and escalation"
        },
        "guideline_compliance": {
            "min": 80.0,
            "description": "Adherence to clinical protocols"
        },
        "sensitivity_chest_pain": {
            "min": 98.0,
            "description": "Detection rate for cardiac symptoms"
        },
        "sensitivity_stroke": {
            "min": 99.0,
            "description": "Detection rate for stroke symptoms"
        },
        "sensitivity_pediatric_fever": {
            "min": 95.0,
            "description": "Detection rate for high-risk pediatric fever"
        }
    },

    # Regression thresholds - compare to production baseline
    regression_thresholds={
        "triage_accuracy": {
            "max_decrease": 2.0,  # Cannot drop more than 2%
            "compare_to": "production"
        },
        "safety_score": {
            "max_decrease": 0.0,  # Cannot decrease at all
            "compare_to": "production"
        }
    },

    # Statistical significance requirements
    statistical_requirements={
        "min_sample_size": 500,
        "confidence_level": 0.95,
        "require_significant_improvement": False  # Don't require improvement, just no regression
    }
)
```

<Warning>
  **Critical Thresholds Are Non-Negotiable**: Critical thresholds should be set to zero tolerance for life-threatening failures. These cannot be overridden by anyone - not engineers, not managers, not executives. If a model fails a critical threshold, it does not ship.
</Warning>

## Step 2: Create Safety Test Dataset

Build a comprehensive test dataset that covers all critical scenarios:

```python title="safety_dataset.py" theme={null}
# Create a safety-focused evaluation dataset
safety_dataset = client.datasets.create(
    name="safety-gate-test-set",
    project="patient-triage",
    description="Comprehensive test set for production safety gating",

    # Dataset composition requirements
    composition={
        "min_total_samples": 500,

        # Must include critical edge cases
        "required_categories": {
            "cardiac_emergencies": {
                "min_count": 50,
                "includes": ["mi_presentation", "unstable_angina", "aortic_dissection"]
            },
            "neurological_emergencies": {
                "min_count": 40,
                "includes": ["stroke", "tia", "seizure", "meningitis"]
            },
            "pediatric_emergencies": {
                "min_count": 40,
                "includes": ["febrile_infant", "respiratory_distress", "dehydration"]
            },
            "psychiatric_emergencies": {
                "min_count": 30,
                "includes": ["suicidal_ideation", "psychosis", "overdose"]
            },
            "sepsis_presentations": {
                "min_count": 30,
                "includes": ["sepsis", "septic_shock", "uti_progressing"]
            },
            "atypical_presentations": {
                "min_count": 50,
                "includes": ["female_mi", "elderly_infection", "diabetic_silent_mi"]
            },
            "routine_cases": {
                "min_count": 200,
                "description": "Common low-acuity presentations"
            }
        },

        # Adversarial cases to test robustness
        "adversarial_cases": {
            "minimizers": 30,        # Patients who downplay symptoms
            "poor_historians": 20,   # Vague or incomplete information
            "multiple_complaints": 20,  # Complex presentations
            "language_barriers": 15  # Non-native speakers, interpreters
        }
    }
)

# Validate dataset meets requirements
validation = client.datasets.validate(safety_dataset.id)
if not validation.meets_requirements:
    print("Dataset gaps:")
    for gap in validation.gaps:
        print(f"  - {gap.category}: need {gap.required}, have {gap.current}")
```

## Step 3: Run Safety Gate Evaluation

Execute the safety gate check before any deployment:

```python title="run_safety_gate.py" theme={null}
# Run safety gate evaluation
gate_result = client.safety_gates.evaluate(
    gate="triage-production-gate",
    model_version="v2.4.1",
    dataset="safety-gate-test-set",

    # Compare against current production
    baseline_model="v2.4.0",  # Current production version

    # Additional options
    options={
        "parallel_execution": True,
        "save_all_predictions": True,  # For audit trail
        "notify_on_failure": ["safety-team@company.com"]
    }
)

# Check results
print(f"Gate Status: {gate_result.status}")  # PASSED, FAILED, or ERROR
print(f"Overall Score: {gate_result.overall_score}%")
print()

# Critical threshold results
print("Critical Thresholds:")
for threshold, result in gate_result.critical_results.items():
    status = "✅ PASS" if result.passed else "❌ FAIL"
    print(f"  {status} {threshold}: {result.value} (max: {result.threshold})")
print()

# Minimum threshold results
print("Minimum Thresholds:")
for threshold, result in gate_result.minimum_results.items():
    status = "✅ PASS" if result.passed else "❌ FAIL"
    print(f"  {status} {threshold}: {result.value}% (min: {result.threshold}%)")
print()

# Regression check results
print("Regression Checks:")
for metric, result in gate_result.regression_results.items():
    status = "✅ PASS" if result.passed else "❌ FAIL"
    delta = result.new_value - result.baseline_value
    print(f"  {status} {metric}: {result.new_value}% (baseline: {result.baseline_value}%, Δ: {delta:+.1f}%)")
```

Example output for a **failing** gate:

```plaintext title="Example: Failed Gate" theme={null}
Gate Status: FAILED
Overall Score: 84.2%

Critical Thresholds:
  ❌ FAIL critical_red_flag_miss_rate: 0.4% (max: 0.0%)
  ✅ PASS critical_undertriage_rate: 0.0% (max: 0.0%)
  ✅ PASS medication_hallucination_count: 0 (max: 0)

Minimum Thresholds:
  ✅ PASS triage_accuracy: 86.2% (min: 85.0%)
  ❌ FAIL safety_score: 93.8% (min: 95.0%)
  ✅ PASS guideline_compliance: 82.4% (min: 80.0%)
  ✅ PASS sensitivity_chest_pain: 98.5% (min: 98.0%)
  ✅ PASS sensitivity_stroke: 99.2% (min: 99.0%)
  ✅ PASS sensitivity_pediatric_fever: 96.1% (min: 95.0%)

Regression Checks:
  ✅ PASS triage_accuracy: 86.2% (baseline: 85.8%, Δ: +0.4%)
  ❌ FAIL safety_score: 93.8% (baseline: 94.5%, Δ: -0.7%)

DEPLOYMENT BLOCKED: 3 threshold failures
```

## Step 4: Investigate Failures

When a gate fails, investigate the specific cases that caused the failure:

```python title="investigate_failures.py" theme={null}
# Get detailed failure analysis
failures = client.safety_gates.get_failures(gate_result.id)

print(f"Total Failures: {len(failures)}")
print()

# Group by failure type
for failure_type, cases in failures.group_by("failure_type").items():
    print(f"\n{failure_type.upper()} ({len(cases)} cases):")
    print("-" * 50)

    for case in cases[:3]:  # Show first 3
        print(f"Sample ID: {case.sample_id}")
        print(f"Input: {case.input[:100]}...")
        print(f"Expected: {case.expected_triage}")
        print(f"Predicted: {case.predicted_triage}")
        print(f"Missed Red Flags: {case.missed_red_flags}")
        print()

# Export failures for detailed review
client.safety_gates.export_failures(
    gate_result.id,
    format="csv",
    destination="s3://safety-reviews/gate-failures-v2.4.1.csv"
)

# Route critical failures for human review
critical_failures = [f for f in failures if f.severity == "critical"]
for failure in critical_failures:
    client.reviews.create(
        sample_id=failure.sample_id,
        priority="urgent",
        reason=f"Safety gate failure: {failure.failure_type}",
        required_reviewer_type="physician"
    )
```

## Step 5: CI/CD Integration

Integrate safety gates into your deployment pipeline:

```yaml title=".github/workflows/deploy.yml" theme={null}
name: Deploy with Safety Gate

on:
  push:
    branches: [main]
    paths:
      - 'models/**'
      - 'prompts/**'

jobs:
  safety-gate:
    name: Safety Gate Check
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install Akhara CLI
        run: pip install akhara-cli

      - name: Run Safety Gate
        id: gate
        env:
          AKHARA_API_KEY: ${{ secrets.AKHARA_API_KEY }}
        run: |
          akhara safety-gate run \
            --gate triage-production-gate \
            --model-version ${{ github.sha }} \
            --dataset safety-gate-test-set \
            --baseline production \
            --output gate-results.json

          # Extract status for job output
          echo "status=$(jq -r '.status' gate-results.json)" >> $GITHUB_OUTPUT
          echo "score=$(jq -r '.overall_score' gate-results.json)" >> $GITHUB_OUTPUT

      - name: Upload Gate Results
        uses: actions/upload-artifact@v4
        with:
          name: safety-gate-results
          path: gate-results.json

      - name: Check Gate Status
        if: steps.gate.outputs.status != 'PASSED'
        run: |
          echo "❌ Safety gate FAILED"
          echo "Score: ${{ steps.gate.outputs.score }}%"
          echo ""
          echo "Review failures at: https://app.akhara.ai/gates/${{ github.sha }}"
          exit 1

      - name: Gate Passed
        if: steps.gate.outputs.status == 'PASSED'
        run: |
          echo "✅ Safety gate PASSED"
          echo "Score: ${{ steps.gate.outputs.score }}%"

  deploy:
    name: Deploy to Production
    needs: safety-gate
    runs-on: ubuntu-latest
    environment: production

    steps:
      - name: Deploy Model
        run: |
          echo "Deploying model version ${{ github.sha }}"
          # Your deployment commands here
```

<Info>
  **Environment Protection**: Configure GitHub environment protection rules to require the safety-gate job to pass before the deploy job can run. This provides an additional layer of protection against accidental deployments.
</Info>

## Step 6: Emergency Override Process

In rare cases, you may need to deploy despite a gate failure. This requires documented approval and creates a permanent audit record:

```python title="emergency_override.py" theme={null}
# Emergency override (requires special permissions)
override = client.safety_gates.request_override(
    gate_result_id=gate_result.id,

    # Justification is required
    justification={
        "reason": "Critical production bug fix - current version crashes on 5% of calls",
        "risk_assessment": "New version has 0.4% red flag miss rate vs 0.0% threshold, "
                          "but current version is completely non-functional for affected users",
        "mitigation_plan": "Deploy with increased human review rate (100% for 24h), "
                          "hotfix for red flag detection in progress",
        "rollback_plan": "Immediate rollback if any critical incident reported"
    },

    # Required approvers (must be pre-configured)
    requested_approvers=[
        "chief_medical_officer",
        "head_of_engineering",
        "head_of_compliance"
    ]
)

print(f"Override Request ID: {override.id}")
print(f"Status: {override.status}")  # PENDING_APPROVAL
print(f"Required Approvals: {override.required_approvals}")
print(f"Current Approvals: {override.current_approvals}")

# Approvers receive notification and must approve in dashboard
# Once approved, deployment is unblocked but permanently flagged
```

<Warning>
  **Override Audit Trail**: All override requests and approvals are permanently logged and cannot be deleted. This audit trail is included in regulatory exports and compliance reports. Overrides should be extremely rare - more than 1-2 per year suggests your thresholds may need recalibration.
</Warning>

## Safety Gate Checklist

| Requirement                             | Status | Notes                                        |
| --------------------------------------- | ------ | -------------------------------------------- |
| Critical thresholds defined             | ☐      | Zero tolerance for life-threatening failures |
| Minimum thresholds defined              | ☐      | Baseline acceptable performance              |
| Regression thresholds defined           | ☐      | Cannot be worse than production              |
| Test dataset covers edge cases          | ☐      | Cardiac, neuro, peds, psych emergencies      |
| Test dataset includes adversarial cases | ☐      | Minimizers, poor historians                  |
| CI/CD integration configured            | ☐      | Blocks deployment on failure                 |
| Notification on failure                 | ☐      | Safety team alerted immediately              |
| Override process documented             | ☐      | Multi-approver, audit logged                 |
| Failure investigation workflow          | ☐      | Route to human review                        |

## Example Safety Gate Configurations

### Patient Triage Voice AI

| Metric                 | Critical | Minimum | Regression Max |
| ---------------------- | -------- | ------- | -------------- |
| Red Flag Miss Rate     | 0%       | -       | -              |
| Critical Undertriage   | 0%       | -       | -              |
| Triage Accuracy        | -        | 85%     | -2%            |
| Safety Score           | -        | 95%     | 0%             |
| Chest Pain Sensitivity | -        | 98%     | -1%            |
| Stroke Sensitivity     | -        | 99%     | 0%             |

### Clinical Documentation AI

| Metric                    | Critical | Minimum | Regression Max |
| ------------------------- | -------- | ------- | -------------- |
| Medication Hallucinations | 0        | -       | -              |
| Lab Value Errors          | 0        | -       | -              |
| Diagnosis Fabrication     | 0        | -       | -              |
| Completeness Score        | -        | 90%     | -3%            |
| Attribution Accuracy      | -        | 95%     | -2%            |
| ICD-10 Accuracy           | -        | 85%     | -5%            |

### Radiology AI

| Metric                    | Critical | Minimum | Regression Max |
| ------------------------- | -------- | ------- | -------------- |
| Critical Finding Miss     | 0%       | -       | -              |
| Malignancy False Negative | 0%       | -       | -              |
| Finding Detection (All)   | -        | 90%     | -2%            |
| Localization Accuracy     | -        | 85%     | -3%            |
| Report Quality            | -        | 80%     | -5%            |

## Next Steps

* [CI/CD Integration](/evaluation/docs/tutorials/ci-cd) - Full pipeline setup
* [Production Monitoring](/evaluation/docs/tutorials/monitoring) - Track safety metrics post-deployment
* [Export for Regulators](/evaluation/docs/tutorials/export) - Generate compliance documentation