APScheduler · 12 background jobs · live

The bot that never misses a session.

AutoProc's bot system combines real-time health monitoring, multi-layered failure prevention, and AI-powered clinical extraction — engineered to run reliably at scale across 18 clinical verticals including NDIS.

99.5%
launch success rate
< 2 min
failure recovery
40+
concurrent bots
18
clinical verticals
Architecture

Engineered from the ground up for resilience

Twelve background jobs coordinate bot launch, health monitoring, AI processing, and recovery — all orchestrated by a single APScheduler daemon.

ORCHESTRATION SCHEDULER JOBS EXECUTION APScheduler Daemon init_scheduler(app) · 12 background jobs Bot Launcher every 60 seconds Health Monitor every 30 seconds · dual jobs AI Processor Whisper + Claude · 15–20s Recovery SMS / Email · 4-hour token Bot Subprocess asha · gemini · peter Daily.co · HMAC-signed MySQL Database appointments · audit logs heartbeats · incidents WebSocket Push note_extracted event tenant:{'{id}'} · mobile Patient Alert resume link · 4-hour token SMS + email
How It Works

From appointment to clinical note — automatically

Three phases coordinate to deliver a complete, AI-extracted clinical session with zero manual intervention.

PHASE 01

Bot Spawns

Every 60 seconds, the scheduler scans for confirmed appointments. A 4-layer safeguard prevents duplicates, caps concurrency, and routes to the right bot type.

check_and_launch_bots() · interval: 60s
4 guards: PID · lock · attempts · cap
Routes: asha · gemini · peter
PHASE 02

Session Monitored

Two parallel jobs watch every active bot. Heartbeat detection via Daily.co webhook events; direct process exit monitoring via OS signal checks — every 30 seconds.

_monitor_bot_heartbeats() · 30s
_check_bot_process_exits() · 30s
Timeout threshold: 40 seconds
PHASE 03

AI Extracts Notes

On session end, Whisper transcribes the audio. Claude Opus extracts structured clinical fields — summary, diagnosis, goals, progress — pushed in real-time to mobile.

Whisper STT · confidence 0.95
Claude Opus · 6 clinical fields
Saves 5–10 min manual entry / session
Bot Types

Three bots, one intelligent router

Appointments are automatically routed to the optimal bot based on your AI agent configuration, language needs, and session type.

Primary
Asha Bot

Full-featured clinical assistant powered by Claude with ElevenLabs TTS. Handles complex multi-turn clinical conversations across all 18 verticals.

Default — all appointments
Claude claude-haiku-4-5 / claude-opus-4-7
ElevenLabs TTS · natural voice
Whisper STT · 21 languages
Multilingual
Gemini Bot

Google Gemini Live API with native multilingual support. Preferred when gemini_enabled = 1 is set on the tenant.

Trigger: gemini_enabled = 1
Gemini Live API · streaming
Native multilingual support
18 supported languages
Lightweight
Peter Bot

Optimised for follow-up calls. Lower resource footprint, faster startup, designed for scripted or structured follow-up session types.

Trigger: script_type = follow_up
Reduced compute · faster start
Follow-up call structure
Structured scripts supported
Reliability

Four layers that never let a bot run twice — or disappear unnoticed

Every launch passes through four independent checks before a subprocess is spawned. If a bot fails mid-session, the system detects it within 30 seconds and notifies the patient automatically.

1

PID Liveness Check

Non-fatal OS signal verifies whether the existing process is still alive. If dead, records are cleaned and a fresh subprocess is spawned.

os.kill(pid, signal.0)
2

Launch Lock Cooldown

A 60-second lock stored in bot_launch_locks prevents re-launch storms after a service restart. No race conditions.

locked_until = NOW + 60s
3

Max Attempt Breaker

If a bot has failed to launch 3 or more times, the scheduler stops and creates an incident. No infinite retry loops.

attempts ≥ 3 → incident AP-1001
4

Concurrency Cap

Active bot count is checked per tenant before each launch. If over the configured limit (default: 40), the appointment queues for the next cycle.

active_bots ≥ limit → defer 60s
PERFORMANCE TARGETS
99.5%
Bot launch success rate
99.9%
Heartbeat health — active bots with fresh heartbeat
< 2 min
Mean time to recovery after failure
99.8%
Transcript delivery success rate
RECOVERY FLOW

When a bot fails mid-session, the patient receives an automated recovery notification within 2 minutes containing a secure 4-hour resume link — no manual intervention required.

Bot failure detected (30s poll)
Incident AP-1003 or AP-1004 logged
4-hour token generated
SMS + Email dispatched to patient
AI Pipeline

From audio to structured clinical note — in under 2 minutes

Whisper STT transcribes the session. Claude Opus extracts structured clinical data. Both push to your mobile dashboard in real time.

Session Complete

Bot constructs full transcript, SOAP note, and session metadata on exit.

bot_exit → transcript_ready

Webhook POST

HTTP POST to /tenant/daily/webhook. HMAC-SHA256 signature verified. Up to 5 retries on failure.

max_retries: 5 · timeout: 10s

Whisper STT

OpenAI Whisper transcribes the audio file. Confidence score 0.95. Supports 21 languages. Polled every 15 seconds.

model: whisper-1 · poll: 15s

Claude Extraction

Claude Opus extracts six structured clinical fields from the transcript. 0.92 confidence. Saves 5–10 minutes of manual data entry per session.

claude-opus-4-7 · poll: 20s

Real-Time Push

WebSocket emits note_extracted event. Clinician sees the note on their mobile dashboard before leaving the session room.

event: note_extracted · room: tenant:{'{id}'}

Extracted Clinical Fields

summarySession overview
diagnosisDifferential / confirmed
progressTreatment progress
goalsPatient goals
concernsFlagged concerns
recommendationsNext steps

Whisper STT

Modelwhisper-1
Languages21 supported
Max file50 MB
Confidence0.95 typical

Claude Opus Extraction

Modelclaude-opus-4-7
Fields6 clinical
Confidence0.92 typical
Time saved5–10 min / session
Verticals18 clinical
FAQ

Technical questions, answered

The details matter. Here's how the bot system handles the hard cases.

What happens if a bot crashes mid-session?
Two independent monitor jobs catch this. The heartbeat monitor detects no Daily.co event within 40 seconds; the process exit monitor detects the dead PID via OS signal check — both run every 30 seconds. On failure, the bot is killed (SIGTERM → SIGKILL), an incident is logged (AP-1003 or AP-1004), and an automated SMS + email is sent to the patient with a 4-hour secure resume link. The entire cycle from failure to patient notification takes under 2 minutes.
How does the system prevent launching the same bot twice?
Every launch passes through four independent safeguards in sequence: (1) a non-fatal OS signal checks whether the stored PID is still alive, (2) a bot_launch_locks database record enforces a 60-second cooldown per appointment — surviving service restarts, (3) if bot_launch_attempts ≥ 3, the scheduler stops and creates an incident rather than retrying forever, and (4) the active bot count per tenant is checked against the configured cap (default: 40) before any new launch is attempted. All four must pass.
How is the Daily.co webhook secured?
All webhook events from Daily.co to /tenant/daily/webhook are verified using HMAC-SHA256 signatures against a per-tenant webhook_secret stored in the database. Any request with an invalid or missing signature is immediately rejected with a 401. Heartbeat events (used to detect bot presence) arrive approximately every 5–10 seconds from Daily.co's participant event stream.
How many concurrent bots can the system run?
The default concurrency cap is 40 bots per tenant, configurable via platform_config. Appointments that exceed this cap are automatically deferred to the next 60-second scheduler cycle rather than dropped. All audit activity — launches, skips, locks — is recorded in the append-only bot_launch_audit table for full observability.
What clinical specialties does the AI extraction support?
The extraction pipeline supports 18 clinical verticals. Claude Opus extracts six structured fields per session — summary, diagnosis, progress, goals, concerns, and recommendations — adapting the extraction schema to the session type. Whisper STT supports 21 languages, making the pipeline suitable for multilingual clinical environments. The Gemini Bot provides additional native multilingual support when enabled.
Where can I monitor the bot system's health?
The admin dashboard at /admin/resilience/ provides a live RED/ORANGE/YELLOW/GREEN system health indicator, active issue counts, stuck bot counts, failed webhook counts, and Phase 1 metrics. The diagnostic panel lets you search any appointment, view the last 100 lines of bot logs, and trigger manual integrity checks. A mini test mode can run 10 bots over 2 minutes for validation.
Get Started

Ready to let the bot handle the notes?

Join clinics already running AutoProc bots across 18 specialties. Setup takes minutes.

No credit card required · autoproc.com.au