Balancing Healthcare AI Risks

Physicians Embrace AI but Data Quality Remains Critical

Eight in ten U.S. physicians now use AI at work, according to WSJ reporting. Most rely on it for administrative tasks—scheduling, billing, documentation—where efficiency gains are easiest to measure. However, experts warn that deploying AI without clean patient data creates serious risks. Inaccurate, incomplete, or misidentified information should halt rollout entirely.

1 source

Medicare's Prior Authorization Pilot Revealed System Unpreparedness

The Medicare WISeR program tested AI-assisted prior authorization across 13 medical services in six states: Arizona, New Jersey, Ohio, Oklahoma, Texas, and Washington. One contractor warned the Centers for Medicare & Medicaid Services (CMS) before launch that its system wasn't ready. CMS proceeded anyway, and the results were troubling: requests exceeded the promised 72-hour turnaround. One reportedly went unanswered for 83 days. Clinicians reported that procedures were postponed or canceled due to delays.

The pilot's core problem was timing. Adding AI before workflows, contractor systems, and testing were mature created severe bottlenecks rather than eliminating them. Contractors were required to have clinicians validate AI-assisted determinations, yet that safeguard couldn't compensate for inadequate infrastructure. Upcoming CMS interoperability rules may improve outcomes by enabling electronic exchange between providers and payers by January 1, 2027, allowing AI agents to coordinate requests and monitor approvals across systems.

Australian Health Tech Aims to Expand Access With Clinician Oversight

Everlab, an Australian health-tech company, is piloting a different model: clinician-supervised AI systems designed to gather patient records, manage insurance approvals, and match patients with clinical trials. Its flagship health assessment costs $2,700. An entry-level subscription runs $300 per year and includes basic blood panels, genetic testing, and access to clinical and allied-health teams. The theory is that AI coordination reduces wait times while preserving human judgment for complex decisions.

Success depends on embedding AI within a human-supported service rather than as an autonomous tool. When delays or denials fall disproportionately on patients unable to pursue appeals, AI can worsen inequality. Governance—escalation rules, oversight, testing standards, and controls for bad data—matters as much as model quality. Forbes highlights that enterprise AI requires comprehensive security and control mechanisms beyond single-point solutions.

1 source

Stronger Oversight and Cost Accountability Emerging as Standards

The United Kingdom's national commission has recommended staged authorizations, continuous real-world safety monitoring, public reporting of incidents, and stronger regulatory enforcement. Health systems should evaluate AI regularly against its intended purpose, maintain clinical accountability, and redirect clerical savings toward patient interaction and staff training. At a corporate AI summit, KKR's CIO Ruchir Swarup emphasized that scaling AI requires tracking token use and assigning costs to individual strategies and agent actions—treating deployment as an economic discipline, not just a technology decision.