California is one of the first states to set new guardrails on artificial intelligence’s use in health care under two bills Gov. Gavin Newsom (D) signed Sunday.
One establishes parameters for AI’s use in insurance coverage decisions. The other requires health care providers to disclose generative AI’s use in patient communications.
Both could serve as templates for other states seeking to balance AI’s challenges and potential in the already heavily regulated health care industry.
“This shows that states are grappling with it,” said California Sen. Josh Becker (D), who introduced the insurance bill. “And hopefully it shows one path forward on one piece of it.”
State lawmakers are still in the early process of determining their role in setting standards that could at some point be determined at the federal level.
At least 40 state legislatures considered AI bills this year. Some of the legislation, including broad AI bills passed in Utah and Colorado, had specific implications for health care providers and patients.
So did executive orders and bills passed in a number of states that set up task forces or select legislative committees. Utah’s bill, for example, required health care providers to tell consumers when they are interacting with an AI-powered chatbot instead of a human.
But only a handful of bills focused specifically on AI in the health care sector. Those all addressed concerns that advances in so-called generative AI technology could introduce biases in treatment and insurance coverage decisions without patients’ knowledge or consent.
“We actually expected more laws like this to be introduced this year and think it’s likely we will see a few other states introduce similar bills next year,” said Randi Seigel, partner at Manatt, which tracks state legislation related to AI in health care.
“However, it may be another few years before we see mass adoption of these types of laws as many states will still be studying how they want to legislate AI through their task forces and councils that were created under this years’ laws.”
Becker said he was alarmed by reports over the past year of providers that rely on AI and algorithms wrongfully denying requests for care. Those reports have prompted concern on Capitol Hill and class action lawsuits against insurers, including UnitedHealthcare and Humana.
Under the new law, any denial or modification of care in a health plan prior authorization process must be overseen by a human physician. AI tools used to support physicians in utilization review processes must comply with federal and state standards to protect against discrimination. Use of AI, algorithm, or other software must be based upon a patient’s medical or other clinical history and individual clinical circumstances, not group datasets.
“This was a targeted bill in one targeted area. But it shows that we can get things done, and hopefully it strikes the right balance,” Becker said. “It doesn’t say, ‘There is no AI in health care.’ It is tackling discrete issues, and saying, ‘We don’t want algorithms used to deny care.’”
Seigel said the new law is consistent with guidance the Centers for Medicare & Medicaid Services published in February regarding Medicare Advantage plans’ use of AI in coverage decisions.
“This bill, similar to CMS’s guidance, appears to strike a fair balance between allowing a plan to use AI to improve efficiency in utilization review, while ensuring that a human reviews decisions (and agrees with decisions) that would have an adverse impact on the plan’s members,” Seigel said.
Supporters included the California Medical Association, which was listed as a sponsor of the bill. The legislature did not receive any written opposition, according to a bill analysis.
“This is a key piece of legislation to allow us to provide some important specific rail guards as AI develops in healthcare,” California Medical Association Trustee Anna Yap said at an April hearing. “As the leaders of the healthcare team, we bear the ultimate responsibility for the treatments that are administered. And it is important that physicians, not computers, can make sure that our patients get the care that they need.”
Under the other law, health care providers must tell patients when they use artificial intelligence tools to generate communications about clinical information, unless they are reviewed by a licensed or certified health care provider. They also must include clear instructions on how to talk with a human health care provider or health care facility employee.
“As health care providers deploy GenAI, patients should be aware of who they are communicating with to ensure confidence in the health care that they’re receiving,” Assemblymember Lisa Calderon (D), who introduced the bill, said at an April hearing.
This story, originally published Sept. 10, was updated on Sept. 30 to reflect that the governor signed both bills.