Agentic AI in Healthcare: Benefits, Risks & Governance – HIMSS26 Insights

Las Vegas, NV – The promise and peril of agentic artificial intelligence dominated discussions at this year’s HIMSS Global Health Conference & Exhibition, held in Las Vegas from March 16-20, 2026. While the potential for AI to automate complex healthcare tasks – from prior authorizations to patient outreach – is generating considerable excitement, experts are urging caution, emphasizing the critical need for robust governance and, crucially, human oversight. The conversation at HIMSS26 wasn’t simply theoretical; it was grounded in real-world deployments and quantifiable results, with organizations reporting significant gains in efficiency and cost savings, alongside a growing awareness of the inherent risks.

Agentic AI, unlike traditional AI systems, operates with a degree of autonomy, capable of independently executing multi-step workflows. This capability offers the potential to alleviate administrative burdens on clinicians and staff, freeing them to focus on patient care. Yet, this very independence introduces novel challenges. The healthcare industry, heavily regulated and deeply sensitive to patient safety, requires a level of control and accountability that autonomous systems can potentially compromise. The core question being addressed at HIMSS26 was not *if* agentic AI will transform healthcare, but *how* to implement it responsibly, and effectively.

The conference highlighted a shift from pilot programs to production-scale deployments of agentic AI, with vendors showcasing tangible outcomes from partnerships with health systems and payers. Google Cloud, for example, presented a portfolio of enterprise partnerships demonstrating the breadth of Gemini-powered agentic AI across the healthcare value chain, including commitments from Humana, CVS Health, Highmark Health, Waystar, and Quest Diagnostics. These deployments span payer operations, consumer health, revenue cycle management, and diagnostic services, signaling a broad industry embrace of the technology. Waystar, in particular, reported $15 billion in prevented claim denials through the leverage of agentic AI, a figure that underscores the potential financial benefits of these systems.

The Rise of Autonomous Workflows in Healthcare

The scope of agentic AI applications discussed at HIMSS26 was remarkably diverse. Beyond revenue cycle management, where AI is being used to automate prior authorizations and reduce claim denials, the technology is also being deployed in areas such as patient scheduling, appointment reminders, and even triage. One example highlighted at the conference involved “midnight triage nurses” – AI systems capable of handling initial patient inquiries and directing them to the appropriate level of care during off-peak hours. Microsoft Copilot was cited as handling 50 million health questions daily, demonstrating the scale at which AI can augment existing healthcare resources.

Donna Fortson, senior vice president and chief revenue officer at WellSpan Health, shared her experience transforming a primary care call center using an agentic AI solution. Prior to implementation, the call center was plagued by long wait times and negative feedback from both clinicians and patients, leading to an increase in in-person visits simply to schedule appointments. The AI-powered solution dramatically improved efficiency and patient satisfaction, demonstrating the potential for agentic AI to address real-world operational challenges.

However, the implementation wasn’t without its hurdles. Several speakers emphasized the importance of addressing “tool sprawl” and potential workflow reorganization when integrating agentic AI. Successfully deploying these systems requires careful planning and a holistic approach to ensure they seamlessly integrate with existing infrastructure and processes.

Governance and Human Verification: The Cornerstones of Safe Implementation

A recurring theme throughout HIMSS26 was the paramount importance of governance and human verification. Experts repeatedly stressed that agentic AI should not be viewed as a replacement for human judgment, but rather as a tool to augment and enhance human capabilities. Dr. Amish Desai, Chief Medical Officer for Population Health at Northwestern Memorial HealthCare, and Dr. R. Ryan Sadeghian, Chief Medical Information Officer at the University of Toledo, both emphasized the need for a governance structure with appropriate guardrails to mitigate potential risks.

The right people must be involved in the selection and implementation process from the outset, ensuring that the AI systems align with organizational values and clinical best practices. Healthcare organizations should focus on areas of friction and low-value function when considering agentic AI use cases, starting with tasks that are well-defined and have clear objectives. This phased approach allows organizations to gain experience and build confidence before deploying AI in more complex or critical areas.

The ethical implications of agentic AI were also a significant topic of discussion. Concerns were raised about potential biases in algorithms, the need for transparency in decision-making, and the importance of protecting patient privacy. Establishing clear ethical guidelines and ensuring that AI systems are used responsibly are essential for building trust and fostering widespread adoption.

Strengthening AI Strategy with Clinical Insight

Beyond the technical aspects of implementation, HIMSS26 underscored the need to strengthen AI strategy with clinical insight. Simply deploying AI tools without a clear understanding of clinical workflows and patient needs is unlikely to yield positive results. Healthcare organizations must involve clinicians in the development and evaluation of AI solutions, ensuring that they are aligned with clinical goals and improve patient outcomes.

The conference also highlighted the importance of interoperability and data standardization. Agentic AI systems rely on access to high-quality, structured data to function effectively. Improving data exchange between different healthcare systems and adopting common data standards are crucial for unlocking the full potential of AI.

Keynote speakers at HIMSS26, including venture capital leader Jon McNeill and Dr. John Halamka, President of the Mayo Clinic Platform, emphasized the need for a strategic approach to AI adoption. Organizations should not simply chase the latest trends, but rather focus on identifying specific problems that AI can solve and developing a long-term vision for AI integration. Sumbul Ahmad Desai, vice president of health and fitness at Apple, and Dr. Mehmet Oz, administrator of the Centers for Medicare and Medicaid Services, also contributed to the discussion, offering insights from the technology and policy perspectives, respectively.

Navigating the Challenges of Agentic AI

Despite the enthusiasm surrounding agentic AI, several challenges remain. One key concern is the potential for unintended consequences. As these systems operate autonomously, it can be difficult to predict how they will behave in all situations. Rigorous testing and ongoing monitoring are essential for identifying and mitigating potential risks.

Another challenge is the need for skilled personnel. Implementing and maintaining agentic AI systems requires expertise in areas such as data science, machine learning, and AI governance. Healthcare organizations may need to invest in training and development to build the necessary internal capabilities.

Finally, the regulatory landscape surrounding AI in healthcare is still evolving. Clearer guidance from regulatory bodies, such as the Food and Drug Administration (FDA), is needed to provide clarity and ensure that AI systems are safe and effective. The FDA has been actively working on a regulatory framework for AI-enabled medical devices, but further development is needed to address the unique challenges posed by agentic AI. The FDA’s website provides updates on their AI/ML initiatives.

Key Takeaways

• Agentic AI is rapidly gaining traction in healthcare, with deployments expanding beyond pilot programs to production-scale implementations.
• Governance and human verification are critical for mitigating the risks associated with autonomous AI systems.
• Clinical insight is essential for ensuring that AI solutions align with clinical goals and improve patient outcomes.
• Interoperability and data standardization are crucial for unlocking the full potential of AI.
• The regulatory landscape for AI in healthcare is evolving, and clearer guidance is needed from regulatory bodies.

Looking ahead, the focus will likely shift towards refining AI governance frameworks, addressing ethical concerns, and fostering greater collaboration between healthcare providers, technology vendors, and regulatory agencies. The HIMSS26 conference served as a crucial platform for these discussions, paving the way for a more responsible and impactful integration of agentic AI into the healthcare ecosystem. The next major checkpoint will be the release of updated FDA guidance on AI-enabled medical devices, expected in late 2026.

What are your thoughts on the role of agentic AI in healthcare? Share your perspectives in the comments below, and don’t forget to share this article with your colleagues.