Joe W. Bowers Jr.
California Black Media

Artificial intelligence (AI) is changing how Californians receive medical care – diagnosing diseases, predicting patient needs, streamlining treatments, and even generating medical notes for doctors.

While AI holds promise, it also poses risks, particularly for Black patients. It can provide faster diagnoses and expand access to care, but it may also misdiagnose conditions, delay treatment, or overlook patient’s critical needs. AI’s impact on Black patients depends on how biases in medical data and algorithms are addressed in its development.

“As we progress toward a society with increased use of AI technology, it is critical that the biases and stereotypes that Black Americans have faced are not perpetuated in our future innovations,” said Dr. Akilah Weber Pierson (D – San Diego), a physician and state senator spearheading legislative efforts to address AI bias in healthcare.

Why AI Matters for Black Californians

Black Californians experience some of the worst health outcomes in the state due to systemic inequities, limited healthcare access, and exclusion from medical research. 16.7% of Black adults report fair or poor health, versus 11.5% of Whites. Black adults have the highest death rates from prostate, breast, colorectal, and lung cancer. Statewide, diabetes affects 13.6% of Black adults versus 9.1% of Whites, and 27% of Black adults over 65 have heart disease, compared to 22% of Whites. Life expectancy for Black Californians is about five years shorter than the state average.

Trending

Opinion – How To Win A Judgeship: Call Yourself A Civil Rights Lawyer

Benefits and Risks of AI in Healthcare

AI processes vast amounts of medical data using computer algorithms designed to identify patient health patterns, helping doctors to diagnose diseases, recommend treatment, and increase patient care efficiency. By analyzing scans, lab results, and patient history, AI can detect diseases

earlier, giving it the potential to improve care for Black patients, who face higher risks of prostate cancer, diabetes, heart disease and hypertension.

Dr. Judy Gichoya, an Interventional radiologist at the Emory University Winship Cancer Institute and AI researcher at Emory’s Healthcare AI Innovation and Translational Informatics (HITI) Lab, sees AI as a tool with great potential but cautions that its effectiveness depends on the diversity of the data it is trained on. She says, “Without diverse datasets, AI could overlook critical signs of diseases, especially in underrepresented populations like Black patients.”

Dr. Timnit Gebru, a computer scientist and AI ethics expert, is the founder and Executive Director of DAIR (Distributed AI Research Institute) in Oakland. She has extensively studied bias in AI systems and their impact on marginalized groups.

Gebru acknowledges that AI has the potential to improve healthcare by enhancing efficiency and expanding access to medical resources. But, like Gichoya she strongly stresses that for AI to be effective and equitable it needs to be subject to rigorous oversight.

AI is already helping doctors personalize cancer treatment by identifying biomarkers and genetic mutations. UCSF and Stanford Health use AI to analyze tumor DNA to match patients with the most effective chemotherapy or immunotherapy.

In diabetes care, AI predicts blood sugar fluctuations, helping doctors adjust treatment. It helps radiologists in early disease detection and identifies sepsis sooner, reducing hospital deaths. In cardiology, AI detects early signs of heart disease, spotting plaque buildup or abnormal heart rhythms before symptoms appear. It also helps predict strokes by analyzing brain scans to determine risk and guide intervention.

Kaiser Permanente uses AI scribes to reduce paperwork and improve patient interactions. Covered California has partnered with Google

Cloud to use AI to streamline document verification and eligibility decisions.

Despite these advancements, AI systems trained on biased medical data can perpetuate inequities for Black patients.

Gebru explains, “If AI learns from historically discriminatory medical decisions—such as undertreating Black patients—it will scale those biases.”

A notable example is in dermatology, where AI frequently misdiagnoses conditions in Black patients because most training datasets are based on lighter-skinned individuals. “Melanoma looks very different on darker skin,” Gebru notes. “It’s not just darker—it often appears differently, like under toenails, a pattern AI trained mostly on lighter skin won’t detect.”

Another risk of AI in healthcare is automation bias, where healthcare providers over-rely on AI, even when it contradicts medical expertise. “Doctors who would have prescribed medications accurately without AI sometimes make mistakes while using automated tools because they over-trust these systems,” Gebru adds.

AI-driven health insurance claim denials are a growing concern. UnitedHealthcare faces a class-action lawsuit for allegedly using an unregulated AI algorithm to deny rehabilitation coverage to elderly and disabled patients.

Beyond bias, AI also poses an environmental threat. AI systems require enormous amounts of energy for computing and massive amounts of water to cool data centers, which exacerbates climate change, an issue that already disproportionately impacts Black communities.

Trump Administration and DEI Impact

The Trump administration’s efforts to dismantle Diversity, Equity, and Inclusion (DEI) threatens funding for AI bias research in healthcare.

Less federal support could stall progress in making AI systems fairer and more accurate, increasing discrimination risks for Black patients.

California’s Legislative and Regulatory Response

Recognizing AI’s risks in healthcare, California lawmakers and state officials are implementing regulations. Weber Pierson introduced Senate Bill (SB) 503 to ensure that AI algorithms used in healthcare are tested for racial bias before implementation.

“We’ve already seen how biased medical devices like pulse oximeters can fail Black patients,” Weber Pierson explains. “If algorithms used in patient care aren’t inclusive, they’re not going to accurately serve melanated individuals.”

At a press conference introducing SB 503, Weber Pierson stressed that AI must be held accountable. “This bill focuses on ensuring that software used as an accessory to healthcare staff delivers sound, nondiscriminatory decisions that promote equitable outcomes.”

Other legislative efforts include Senate Bill (SB) 1120, by Sen. Josh Becker (D-Menlo Park), which stops insurance companies from using AI alone to deny or delay care and Assembly Bill (AB) 3030, by Assemblymember Lisa Calderon (D-Whittier), which requires healthcare providers to inform patients when AI is used in their care.

Attorney General Rob Bonta has issued a legal advisory barring AI from unfairly denying healthcare claims, falsifying records, or restricting access to care based on medical history. Gov. Gavin Newsom’s 2023 executive order directs state agencies to assess AI’s impact and establish consumer protections, particularly in healthcare.

Actions Black Patients and Families Can Take

As AI becomes more common in healthcare, Black Californians can ensure fair treatment by asking if AI is used, seeking second opinions, and supporting groups addressing algorithmic bias.

They can:

• Ask their healthcare providers whether AI played a role in their diagnosis or treatment.
• Request second opinions if an AI-generated diagnosis seems questionable.
• Advocate for AI policies and legislation promoting fairness and accountability. · Engage with community health organizations like the California Black Health Network (CBHN) that is engaged in ensuring AI is developed in ways to improve health outcomes for Black patients.

Rhonda Smith, CBHN’s executive director, says bias in medical algorithms must be eliminated. “There should never be any race-based adjustment in delivering patient care,” she said.

CBHN supports inclusive research and legislation like SB 503 to ensure AI promotes equity.

Ensuring AI Benefits All Communities

As a legislator, Weber Pierson is pushing for stronger safeguards to ensure AI serves all patients equitably. She says, “Innovation and technology are good, but new challenges arise if we don’t move in a direction inclusive and thoughtful of all people who utilize the healthcare space.”

AI has the potential to revolutionize healthcare, but experts warn it must be developed and regulated with transparency, accountability, and fairness – ensuring it reduces rather than worsens, racial health disparities.

Oakland Post

Posts by Oakland Post