This pre-registered, pragmatic, three-arm (1:1:1) patient-level randomized controlled trial with mixed-effects analysis at the encounter level tests two questions in real high-risk multidisciplinary clinical encounters at the Waymark clinically integrated network across three U.S. states (Ohio, Washington, Virginia): (1) does adding ANCHOR - a clinical AI structural verification layer - to a Gemini 3.1 Pro-assisted supervising-physician workflow reduce the rate of clinically meaningful safety failures, compared with the same Gemini 3.1 Pro-assisted workflow without ANCHOR? (2) does the Gemini 3.1 Pro-assisted workflow itself reduce the same safety endpoint compared with unassisted standard care in which the supervising physician writes their own SOAP assessment/plan from a blank template?
ANCHOR is a single-call structural verification layer combining a Logical Neural Network (Riegel et al. 2020) certificate, six specialist agents, and concept-decomposed output with PMID citation provenance. ANCHOR is physician-facing only and is used by supervising physicians, not by the multidisciplinary clinical team they oversee.
The trial randomizes 240 patients 1:1:1 across the Waymark clinically integrated network over a 12-week active-enrolment window (80 per arm). Eligible patients are adults (age 18+) identified as high-risk by combined claims-based and clinical criteria. Eligible encounters span three integrated Waymark service modalities: high-risk primary care, specialty care coordination, and real-time telemedicine urgent care. The primary endpoint is a per-encounter binary composite: any of (a) failure to mention a do-not-miss diagnosis, (b) under-triage, (c) contraindicated medication recommendation, (d) failure to recommend escalation when clinically warranted; adjudicated by a blinded panel of 3 board-certified physicians with majority-of-three scoring. The primary contrast is Arm 3 (LLM+ANCHOR) versus Arm 2 (LLM with safety prompt), isolating ANCHOR's marginal contribution over a deployment-equivalent LLM safety stack. The pre-specified secondary contrast is Arm 2 versus Arm 1.
The trial is sized to the operational ceiling of the Waymark integrated-network workflow across the three states (240 enrollees over 12 weeks). At realistic effect sizes derived from the retrospective evaluation, the trial is underpowered for definitive efficacy declaration on either pairwise contrast and is reported as an initial deployment-feasibility validation cohort with effect estimates and 95 percent confidence intervals; full power calculations are pre-registered in the Statistical Analysis Plan.
Single-blind outcome adjudication: 3 adjudicators score only the supervising physician's final clinical decision, so all three arms produce adjudication packets in identical format and arm allocation is structurally invisible. Statisticians remain blinded until database lock. A full waiver of informed consent is requested per 45 CFR 46.116(f)(3) with a companion HIPAA waiver of authorization under 45 CFR 164.512(i)(2)(ii). The study is registered on the Open Science Framework prior to first enrollment and reported under CONSORT-AI 2020.
Age 18 years or older.
Attributed to a participating Waymark provider (academic medical center, community-hospital network, federally qualified health center, or independent physician practice in Ohio, Washington, or Virginia; full TIN-consolidated list deposited at the Open Science Framework).
Meets high-risk multidisciplinary criteria (combined claims-based and clinical: 2 or more emergency-department visits or 1 or more hospitalization in the prior 12 months, 5 or more active medications, 2 or more active specialist relationships, 2 or more chronic conditions, or claims-based equivalents).
Encounter occurs in one of the three Waymark service modalities: high-risk primary care, specialty care coordination, or real-time telemedicine urgent care.
Pediatric (age less than 18 years).
Hospice or palliative-care-exclusive care plan.
Active psychiatric crisis routed to crisis line.
Encounter is administrative only.
Parallel
Gemini 3.1 Pro with Safety Prompt
BehavioralGemini 3.1 Pro generates the care-management recommendation under a clinical-safety system prompt, content filters, and retrieval-augmented generation. Supervising physician reviews the LLM output directly without ANCHOR augmentation.
ANCHOR Clinical AI Verification Layer (with Gemini 3.1 Pro)
BehavioralSame Gemini 3.1 Pro generation as Arm 2, with ANCHOR additionally applied: a single-call structural verification layer combining a Logical Neural Network safety certificate over a 3,206-rule clinical logic library, six concurrent specialist agents (drug interaction, lab interpretation, guideline compliance, citation verification, safety net, differential-diagnosis breadth), and a concept-decomposition module with PMID-traceable provenance. Decision support only; clinician retains all clinical decision authority.
Per-encounter clinical safety failure (adjudicated binary composite)
Adjudicated binary composite of any of: (a) failure to mention a do-not-miss diagnosis appropriate for the presentation; (b) under-triage (routine/semi-urgent when emergent/urgent appropriate); (c) recommendation of a medication contraindicated by documented allergies/conditions/comorbidities; (d) failure to recommend escalation when clinically warranted. Adjudicated by a blinded panel of 3 board-certified physicians (Internal Medicine, Family Medicine, or Medicine-Pediatrics); final scoring by majority of three. Reported as proportion of encounters with composite safety failure.
Appropriate triage escalation rate
Proportion of encounters in which the assigned triage level was appropriate to the clinical presentation, adjudicated.
Time-to-physician-decision
Continuous duration in seconds from encounter start to supervising-physician final decision; log-transformed for analysis. Excludes API latency.
30-day emergency-department visit rate (exploratory)
Any emergency-department visit within 30 days post-encounter, captured through Waymark's integrated electronic-health-record + real-time admission/discharge/transfer feed + claims pipeline. EXPLORATORY: underpowered at n=240; reported as effect estimate with 95 percent confidence interval without significance claim.
30-day hospitalization rate (exploratory)
Any hospitalization within 30 days post-encounter, captured through the integrated EHR + ADT + claims pipeline. EXPLORATORY: underpowered at n=240; reported as effect estimate with 95 percent confidence interval without significance claim.
Inappropriate over-escalation rate (safety)
Adjudicated binary indicator of inappropriate over-escalation. Pre-specified safety-halt threshold check at midpoint (n=120 cumulative across arms) for greater than 5 percentage points absolute increase in over-escalation in Arm 3 relative to Arm 2.
ANCHOR diagnostic-test characterization (Arm 3 only)
ANCHOR certificate (operational state: hazard flagged / no hazard flagged / out of scope) characterized as a diagnostic test against adjudicated ground truth: sensitivity, specificity, positive predictive value, negative predictive value, area under the receiver-operating-characteristic curve, and calibration metrics (Expected Calibration Error, Brier score, reliability diagram).
Algorithmic fairness audit
Equalized odds gap, equal opportunity gap, predictive parity gap, within-subgroup calibration (Brier-difference tolerance 0.05) on the primary composite endpoint across race/ethnicity (OMB 1997 categories), sex assigned at birth, age band (18-44 / 45-64 / 65-74 / 75+), urban/rural per RUCA 2024, and primary language. Multiplicity correction: Benjamini-Yekutieli false discovery rate with q=0.05. Bootstrap CI: 1000 resamples, seed 20260425, percentile method. Subgroup metrics reported only when n is at least 30 per cell.
Hallucination certificate audit (Arm 3 only)
Per Kalai et al. 2025. Every ANCHOR LNN response is assigned a formal certificate with operational categorical state (hazard flagged / no hazard flagged / out of scope) and a structured rule-firing trace against the 3,206-rule clinical logic library. Reported descriptively per acuity stratum and demographic subgroup; not powered for inference at n=240.
