AI Bias in Healthcare: How Far We’ve Come Since Dr. Ziad Obermeyer’s Groundbreaking Research on Algorithmic Discrimination Against Black Patients

Artificial intelligence in healthcare promised to eliminate bias by removing human judgment from decisions. Yet a decade after research exposed how algorithms systematically underserved Black patients, new evidence shows bias persists—often in ways that are harder to detect. Experts warn that without deliberate intervention, AI systems will continue to reinforce existing inequalities in patient care.

In 2019, a study by Dr. Ziad Obermeyer and colleagues at the University of California, Berkeley, revealed that a widely used algorithm designed to prioritize high-cost patients for extra care was twice as likely to recommend against additional care for Black patients as for white patients with similar health profiles. The algorithm, used by healthcare providers across the U.S., had been trained on historical data that reflected systemic biases in how Black patients were treated—effectively automating discrimination. Since then, the healthcare industry has made progress in identifying algorithmic bias, but new research suggests the problem remains deeply embedded.

Today, AI tools are increasingly used to predict patient risks, allocate resources, and even influence clinical decisions. Yet a 2023 analysis published in Science found that 40% of AI models in healthcare still exhibit measurable bias, particularly against racial and socioeconomic minorities. The issue isn’t just historical data—modern algorithms can also inherit biases from the way they’re designed, the data they’re fed, and the real-world contexts in which they’re deployed.

Why AI Bias Persists: The Hidden Mechanisms

Algorithmic bias in healthcare doesn’t always look like overt discrimination. Often, it manifests in subtle ways that are difficult to detect without rigorous testing. Obermeyer’s original research highlighted how the algorithm’s bias was not intentional but a product of flawed training data. The model had been taught to identify patients who were “high-cost,” but because Black patients historically received less care, the algorithm interpreted lower spending as a sign of good health—leading it to deprioritize interventions for them.

Since then, researchers have identified three key ways bias creeps into AI systems:

• Proxy discrimination: Algorithms may use indirect measures (like ZIP codes or insurance status) that correlate with race or ethnicity, even if those factors aren’t explicitly included in the model.
• Data scarcity: Underrepresented groups often have fewer medical records in training datasets, forcing AI to rely on broader, less accurate patterns.
• Feedback loops: If an AI system consistently misclassifies a group (e.g., underestimating their risk), clinicians may adjust their behavior to match the algorithm’s output, reinforcing the bias.

A 2024 study in JAMA Network Open demonstrated how these mechanisms play out in practice. Researchers tested an AI tool used to predict sepsis risk in hospital patients and found that the model performed significantly worse for Black patients, missing nearly 20% more cases than for white patients. The bias wasn’t due to the algorithm’s design but rather how it was deployed: the tool was calibrated using data from a hospital where Black patients were systematically underdiagnosed with sepsis in the first place.

“The problem isn’t just bad data—it’s that we’ve normalized the idea that algorithms are objective. In reality, they reflect the biases of the systems that created them.”

Has the Healthcare Industry Fixed the Problem?

The backlash to Obermeyer’s 2019 findings spurred major healthcare organizations to audit their AI tools for bias. Epic Systems, one of the largest electronic health record providers in the U.S., launched a bias-mitigation framework in 2021 after internal reviews revealed similar issues in its predictive algorithms. The framework includes steps like diversifying training datasets, involving clinicians from underrepresented backgrounds in model development, and conducting regular bias audits.

However, progress has been uneven. A 2023 report by the U.S. Department of Health and Human Services (HHS) found that only 12% of healthcare AI vendors had implemented any formal bias-mitigation measures. Even when audits are conducted, the results are often not made public, leaving patients and advocates in the dark.

One bright spot is the growing movement to regulate AI in healthcare. The U.S. Food and Drug Administration (FDA) now requires premarket reviews for high-risk AI tools, including assessments of bias and fairness. The European Union’s AI Act, set to take full effect in 2025, will impose stricter rules on AI systems used in healthcare, including mandatory bias testing for high-risk applications.

What Patients and Providers Can Do Now

For patients, recognizing potential bias in AI-driven care is the first step. Here’s what to watch for:

• Unexplained disparities in care: If an AI tool (e.g., a risk-prediction dashboard) consistently suggests lower-intensity treatment for patients from certain backgrounds, ask your provider to explain the reasoning.
• Lack of transparency: Reputable AI systems should disclose how they make decisions. If your healthcare provider can’t explain the algorithm’s logic, it may be a red flag.
• Advocate for human oversight: No AI should replace clinical judgment entirely. Ensure that any AI recommendations are reviewed by a healthcare professional who understands its limitations.

Providers and institutions can take proactive steps to reduce bias in their AI systems:

• Diversify training data: Ensure datasets include representative samples across races, ethnicities, genders, and socioeconomic backgrounds.
• Conduct external audits: Partner with independent researchers to test AI tools for bias, as recommended by the National Academy of Medicine.
• Implement “bias dashboards”: Tools like those developed by MIT’s Fairlearn can help monitor AI systems in real time for emerging biases.
• Educate staff on algorithmic bias: Clinicians must understand how AI tools can perpetuate harm and how to challenge biased recommendations.

The Future: Can AI Be Fair?

The question isn’t whether AI can be fair—it’s whether the healthcare industry is willing to prioritize fairness over efficiency. Obermeyer’s research showed that the original biased algorithm wasn’t just flawed; it was costing lives. Black patients identified as low-risk by the algorithm were 20% more likely to die or be hospitalized within a year compared to white patients given the same classification.

Yet the potential of AI to improve healthcare is undeniable. When designed with equity in mind, these tools can help reduce disparities by identifying at-risk patients earlier, optimizing resource allocation, and even predicting outbreaks of diseases like diabetes or hypertension in underserved communities. The key is treating bias mitigation as a non-negotiable part of AI development—not an optional add-on.

Looking ahead, experts like Obermeyer are pushing for a shift in how AI is evaluated. “We need to move beyond asking, ‘Does this algorithm work?’ and start asking, ‘Does it work for everyone?’” he said in a 2024 interview. “That requires a cultural change in healthcare—one where fairness is baked into the design process from the start.”