Litigation and regulation of AI in healthcare and clinical decision support

1. Litigation and liability exposure for AI-generated clinical content

1.1 UnitedHealth nH Predict litigation

1.1.1 Case identification and procedural status

The litigation formally captioned Estate of Gene B. Lokken et al. v. UnitedHealth Group Inc. et al. is a significant judicial development regarding liability exposure for health systems and insurers deploying AI-generated clinical guidance. The case is pending in the United States District Court for the District of Minnesota under docket number 23-cv-03514-JRT-SGE, assigned to United States District Judge John R. Tunheim and United States Magistrate Judge Shannon G. Elkins (March 9, 2026 order) (Georgetown litigation tracker). The complaint was filed on November 14, 2023 on behalf of the estates of two deceased Medicare Advantage beneficiaries, who alleged that UnitedHealth's subsidiary naviHealth (which does business as Home & Community) used the nH Predict algorithm to systematically deny medically necessary post-acute care coverage (complaint) (Healthcare Dive) (defendants' response, February 4, 2026).

The procedural history shows accelerating judicial engagement with AI system transparency. On February 13, 2025, Judge Tunheim issued a significant ruling on Defendants' motion to dismiss, dismissing claims for unjust enrichment, insurance bad faith, and various state law claims as preempted by the federal Medicare Act, but expressly allowing plaintiffs' claims for breach of contract and breach of the implied covenant of good faith and fair dealing to proceed (February 13, 2025 ruling). As the March 9, 2026 order summarizes that ruling, the surviving claims "effectively arise out of [Defendants'] evidence of coverage documents because the question [is] whether UHC complied with its statement that claim decisions would be made by 'clinical services staff' and 'physicians' when it allegedly used artificial intelligence" (March 9, 2026 order). On this theory, the court said, it would only be required to investigate whether UHC complied with its own written documents (February 13, 2025 ruling, p. 19).

Following the partial dismissal, UnitedHealth sought to bifurcate discovery into two stages, proposing that initial discovery be limited to whether nH Predict was actually used for the named plaintiffs' claims rather than physicians. Magistrate Judge Elkins denied this request in an order dated September 8, 2025 (September 8, 2025 order). As the March 9, 2026 order later described it, discovery would "fully proceed as to the putative class action, from the beginning, due to the enmeshed factual and legal issues comprising the Plaintiff's cause of action" (March 9, 2026 order).

1.1.2 Discovery order and the April 29, 2026 production deadline

The March 9, 2026 discovery order issued by Magistrate Judge Elkins granted in part and denied in part plaintiffs' motion to compel discovery into UnitedHealth's use of the nH Predict algorithm, with oral argument heard on February 11, 2026 (order). The order required UnitedHealth to produce all documents it required within 21 days of its issuance. On March 31, 2026 the court extended the deadline for complete compliance to April 29, 2026, and on May 4, 2026 it extended parts of the production to May 6 and May 20, 2026 (March 31, 2026 order) (May 4, 2026 order).

The motion covered seven categories of documents, and the court granted it at least in part in every category (order):

The court's specific findings on category (B) are particularly consequential: "Plaintiffs are entitled to discovery of documents regarding how nH Predict works, its development goals and anticipated benefit, and whether it was designed to supplant physician decision-making" (order) (Hunton).

The court also rejected UnitedHealth's attempt to limit discovery to records from July 1, 2019 onward, the date from which UnitedHealth said it used nH Predict, finding that "information from 2017-2019 is relevant and could constitute circumstantial evidence as to the breach of contract claims." In doing so it cited the U.S. Senate Permanent Subcommittee on Investigations' October 2024 report, Refusal of Recovery: How Medicare Advantage Insurers Have Denied Patients Access to Post-Acute Care, which in the court's words "confirmed that UHC's claim denial rate for post-acute claims more than doubled after they began using naviHealth and nH Predict in 2019" (order) (ArentFox Schiff). The report found that UnitedHealthcare's denial rate for post-acute care prior authorization requests rose from 8.7% in 2019 to 22.7% in 2022 (Senate report, p. 21).

In a declaration filed with its February 4, 2026 response to the motion, before the order, UnitedHealth stated that it had already produced 5,509 documents, approximately 54,934 pages plus numerous Excel spreadsheets and other documents in native form, and had spent approximately $156,000 on document review and production (defendants' response).

1.2 Abridge AI ambient documentation litigation

1.2.1 Saucedo v. Sharp HealthCare

The Saucedo v. Sharp HealthCare litigation, filed in San Diego Superior Court on November 26, 2025 (Case No. 25CU063632C) (San Diego Superior Court case index), is a legal challenge to ambient AI documentation technology. The case alleges that Sharp HealthCare's deployment of Abridge AI, Inc.'s ambient clinical documentation technology violated California's all-party consent recording statute and medical confidentiality laws by recording patient-clinician conversations and sending the audio to the vendor's servers without obtaining valid consent from patients (KPBS) (Fisher Phillips) (Medscape).

According to reporting on the complaint, it estimates that more than 100,000 Sharp patients may have been recorded during the rollout, and it alleges that Abridge automatically inserted incorrect statements into medical charts documenting that patients "were advised" the visit was being recorded and that they "consented," even when the patient said this never happened (KPBS) (Medscape).

The legal theories draw upon two California statutory frameworks:

1.2.2 Washington v. Sutter Health et al.

Washington v. Sutter Health et al. was filed in the U.S. District Court for the Northern District of California on April 7, 2026 (the complaint is file-stamped April 8), Case No. 4:26-cv-03012-HSG (docket) (complaint). As filed, it named Sutter Health, Memorial Health Services Inc., and MemorialCare Medical Foundation as defendants and pleaded claims under CIPA, the CMIA, the Unfair Competition Law, the federal Wiretap Act and common-law intrusion upon seclusion (complaint) (Alston & Bird) (Paubox). In August 2026 the claims against the Memorial defendants were split off: the second amended complaint names only Sutter Health (second amended complaint), and the claims against MemorialCare were refiled as Matulic v. Memorial Health Services, Inc., No. 3:26-cv-08206 (N.D. Cal.) (docket).

The federal Wiretap Act prohibits interception of wire, oral and electronic communications and carries criminal penalties (18 U.S.C. § 2511). Its civil remedy provision allows recovery of actual damages or statutory damages of whichever is the greater of $100 a day for each day of violation or $10,000, punitive damages in appropriate cases, and a reasonable attorney's fee (18 U.S.C. § 2520).

1.2.3 Parallel litigation

Proposed class actions have now been filed against more than one health system over the same ambient AI technology. The cases involve the same vendor's technology and overlapping theories of unauthorized recording and disclosure under CIPA and the CMIA, although they were brought by different law firms and the vendor is not named as a defendant (Alston & Bird) (KPBS) (complaint). Alston & Bird notes that damages may be assessed on a per-violation or per-encounter basis (Alston & Bird). The Saucedo and Washington complaints were filed about four and a half months apart, and the MemorialCare claims severed from Washington now proceed as a separate case (docket).

2. United States federal regulatory framework

2.1 HIPAA Privacy and Security Rule provisions for AI-generated clinical content

2.1.1 Applicable regulatory provisions

The Health Insurance Portability and Accountability Act of 1996 (HIPAA) and its implementing regulations at 45 C.F.R. Parts 160 and 164 establish the foundational federal regulatory framework for AI-generated clinical content delivery, though the technology-neutral structure of these rules creates both compliance flexibility and interpretive uncertainty for novel delivery architectures (QuickIntell) (edenlab.io). The Privacy Rule governs the use and disclosure of protected health information (PHI) by covered entities (health plans, healthcare clearinghouses, and healthcare providers who transmit health information in electronic form) and extends to business associates who create, receive, maintain, or transmit PHI on behalf of covered entities (QuickIntell) (edenlab.io) (45 C.F.R. Part 164).

For AI clinical decision support vendors operating through cloud infrastructure, the Business Associate Agreement (BAA) requirement is the critical compliance gateway. Under 45 C.F.R. § 164.504(e), any AI vendor that creates, receives, maintains, or transmits PHI on behalf of a covered entity must execute a BAA specifying permitted uses and disclosures, safeguard requirements, and breach notification obligations (QuickIntell) (edenlab.io). This requirement applies categorically to AI vendors processing clinical data, including ambient documentation systems, diagnostic algorithms, and patient-facing guidance platforms. The BAA framework creates a contractual accountability chain that extends to all subcontractors with access to PHI, including cloud infrastructure providers, creating compliance complexity that scales with the number of vendors in the AI delivery chain (edenlab.io) (Accountable).

The minimum necessary standard, 45 C.F.R. § 164.502(b), requires that uses and disclosures of PHI be limited to the minimum necessary to accomplish the intended purpose (QuickIntell) (edenlab.io). For AI-generated clinical content, this standard creates particular complexity: AI systems that generate comprehensive clinical guidance from broad patient data inputs may incorporate or reveal more PHI than is necessary for the specific clinical decision at hand.

The technology-neutral framework permits healthcare operations disclosures without patient consent, 45 C.F.R. § 164.506, which covers quality assessment, competency assurance, and other operational activities (QuickIntell) (edenlab.io). This provision enables current ambient AI and clinical decision support deployments but does not preempt more protective state laws, as the Abridge litigation demonstrates. The interplay between HIPAA permissiveness and state law restrictiveness creates a complex compliance landscape where federal compliance alone is insufficient to avoid liability (edenlab.io) (Accountable).

2.2 ONC Information Blocking Rule and HTI-1 Final Rule

2.2.1 HTI-1 Final Rule: algorithm transparency requirements

The Office of the National Coordinator for Health Information Technology (ONC) has promulgated the most specific federal regulatory requirements for AI transparency in healthcare through the Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule, published at 89 Fed. Reg. 1192 (January 9, 2024) and codified at 45 C.F.R. Parts 170 and 171 (Crowell & Moring LLP) (GovInfo). This rule introduces Decision Support Intervention (DSI) transparency requirements that apply from January 1, 2025, by which date certified health IT had to meet the new DSI certification criterion in place of the former clinical decision support criterion, establishing mandatory disclosure obligations for predictive algorithms integrated into certified health IT products (Crowell & Moring LLP) (Federal Register).

The DSI transparency provisions require developers of certified health IT to support "source attribute" information for predictive DSIs that includes: intended use and intended user descriptions; training data characteristics and limitations; model performance metrics, including validity and fairness; and known risks, limitations, and cautioned out-of-scope uses (45 C.F.R. § 170.315(b)(11), in the HTI-1 final rule) (akingump.com).

The HTI-1 rule narrows scope from the proposed rule to apply specifically to predictive DSIs supplied by the health IT developer as part of its Health IT Module, rather than extending accountability to third-party algorithms with which the module merely interfaces (Crowell & Moring LLP). As a result, developers must provide transparency for their own embedded algorithms, but the rule creates a regulatory boundary around externally-hosted or third-party AI services. The rule emphasizes enabling healthcare providers to assess algorithms for "fairness, appropriateness, validity, effectiveness, and safety" (ASTP).

Source for the table: HTI-1 final rule, Federal Register; Crowell & Moring LLP; ASTP.

2.2.2 Information blocking obligations and external platform transmission

The information blocking provisions at 45 C.F.R. Part 171, originally established under the 21st Century Cures Act and modified by HTI-1, directly govern how providers and health IT developers may restrict the access, exchange, or use of electronic health information (EHI), including AI-generated clinical content (Crowell & Moring LLP) (GovInfo). The core prohibition applies to any practice that is likely to interfere with, prevent, or materially discourage access, exchange, or use of EHI, with specific exceptions for practices such as preventing harm, protecting privacy, managing security, and addressing infeasibility (Crowell & Moring LLP) (akingump.com).

For providers transmitting clinical records and AI-generated care guidance through external platforms, the information blocking rules create complex compliance obligations. The prohibition on interfering with EHI access, exchange, or use means that providers cannot selectively block or delay transmission of AI-generated clinical content to patients or other authorized recipients. However, the privacy and security exceptions permit providers to implement technical safeguards and consent-based access controls that may limit the modalities of information exchange. The infeasibility exception, as modified by HTI-1, applies when compliance with an EHI request would be infeasible due to unforeseeable or unavoidable circumstances outside the actor's control, with the modification requiring direct causal relationship between the uncontrollable event and the inability to fulfill the request (PAMED) (Crowell & Moring LLP).

2.2.3 Enforcement posture and regulatory evolution

ONC's enforcement mechanism for information blocking violations operates through referral to the HHS Office of Inspector General (OIG), which may impose civil monetary penalties of up to $1 million per violation for health IT developers and health information networks/exchanges, and disincentives under applicable programs for healthcare providers (mapyourshow.com) (HHS.gov) (HIT Consultant). The HTI-5 proposed rule, announced on December 22, 2025 and published in the Federal Register on December 29, 2025, proposes to remove over 50% of the ONC Health IT Certification Program's certification criteria and to revise or remove certain information blocking terms, conditions and exceptions (ASTP) (Federal Register). As of May 2026, HTI-5 remained a proposed rule; its 60-day public comment period closed on February 27, 2026 (Federal Register).

Despite this regulatory flux, the core information blocking obligations remain enforceable. On September 3, 2025, HHS announced that Secretary Robert F. Kennedy, Jr. had directed increased resources toward curbing information blocking (HHS.gov). In the announcement, Deputy Secretary Jim O'Neill said that "unblocking the flow of health information is critical to unleashing health IT innovation," and Acting Inspector General Juliet T. Hodgkins said that "HHS-OIG will deploy all available authorities to investigate and hold violators accountable" (HIT Consultant) (HIPAA Journal). ASTP/ONC said it had already begun reviewing reports of information blocking against developers of certified health IT and was providing technical assistance to OIG for investigations (HIPAA Journal).

The HTI-1 Final Rule states that "as technology related to Predictive DSIs continues to evolve and as industry consensus matures, [ONC] expects that new information may need to be made available through source attributes for new models." It also provides for "a limited set of identified users to record, change, and access additional source attribute information not specified" in the baseline requirements (HTI-1 final rule, Federal Register).

2.3 21 CFR Part 11: electronic records and audit trail requirements

2.3.1 Record integrity and audit trail mandates for AI-generated content

The FDA's regulations at 21 C.F.R. Part 11 establish comprehensive requirements for electronic records and electronic signatures in FDA-regulated activities, with direct applicability to AI-generated clinical records, informed consent documents, and patient-facing outputs in pharmaceutical, clinical trial, and certain medical device contexts (IntuitionLabs). The regulation applies to records in electronic form that are "created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency regulations" (21 C.F.R. § 11.1(b)).

The audit trail provisions of 21 C.F.R. § 11.10(e) mandate that systems use "secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records." The regulation requires that audit trail documentation be retained for "a period at least as long as that required for the subject electronic records" and that it "shall be available for agency review and copying" (21 C.F.R. § 11.10(e)) (IntuitionLabs).

For AI-generated content, IntuitionLabs recommends treating AI events as controlled records and logging the model identifier and version, the user or operator identity, a timestamp, input prompts and output results, and human review or sign-off decisions (IntuitionLabs).

In the EU, GMP Annex 11 is under revision. IntuitionLabs reports that the proposed revision would state that audit trails must not be editable or deactivatable by operators, with any deletion requiring a controlled override; a separate draft, Annex 22, addresses artificial intelligence (IntuitionLabs). Both remained drafts as of October 2026 (European Commission, EudraLex Volume 4).

2.3.2 Industry practice for AI audit trails

Industry best practices for AI audit trail compliance, as documented in technical analyses of GxP requirements, emphasize "prompt-to-portal" traceability that captures the complete chain from user request through AI processing to final output delivery. For pharmaceutical sponsors implementing generative AI for clinical document generation, quality assurance protocols require that all AI outputs flow into controlled document management systems with logged prompt records, output records, and edit records, each stamped with user identity and timestamp (IntuitionLabs).

2.4 Federal information security frameworks and military medicine

2.4.1 Federal security frameworks

The Department of Veterans Affairs (VA) and Department of Defense (DoD), like other federal agencies, operate under overlapping federal information security frameworks: the Federal Information Security Management Act (FISMA), which mandates risk-based information security programs for federal agencies (44 U.S.C. § 3551 et seq.); NIST Special Publication 800-53 Revision 5, which provides the security and privacy control catalog for federal information systems (NIST); the FedRAMP program, which standardizes security assessment and authorization for cloud products and services used by federal agencies (44 U.S.C. § 3608); and the NIST AI Risk Management Framework (AI RMF), which provides voluntary guidance for managing risks in AI systems (NIST AI 100-1).

2.4.2 Military medicine

RIVANNA's Accuro 3S-MIL, a portable AI-powered ultrasound guidance system designed for military use cases, is being developed with a $3 million Technology/Therapeutic Development Award from the Congressionally Directed Medical Research Programs. It targets epidural steroid injections in field settings, where fluoroscopy systems are too large, costly and infrastructure-dependent (RIVANNA).

3. United Kingdom regulatory framework

3.1 NHS Digital and the Data Security and Protection Toolkit (DSPT)

3.1.1 Applicable standards for AI-generated clinical content

The National Health Service (NHS) in England operates under the Data Security and Protection Toolkit (DSPT), a mandatory online self-assessment framework for health and care providers and other organisations that process NHS health and care data (Evalian).

The DSPT measures organisations against the National Data Guardian's 10 data security standards (Evalian). Standard 4, for example, covers managing access to systems holding personal confidential information, including role-based access controls, logging and account removal (DSPT, Data Security Standard 4, 2023/24).

A December 2025 Clyde & Co analysis cites NHS England guidance of April 30, 2025 addressing AI use in clinical settings, which states that "the final decision about the care that people receive should be made in consultation with the patient or service user, using your professional judgment." This reinforces the principle that AI-generated clinical guidance must support, not replace, human clinical decision-making (Clyde & Co).

3.1.2 Ambient scribe guidance and international transfers

NHS England's guidance on the use of AI-enabled ambient scribing products in health and care settings, published on April 27, 2025, asks organisations to complete a data protection impact assessment, to comply with the Data Security and Protection Toolkit, and to be transparent with patients about how their information is used, for example by explaining how it will be used before the processing takes place and giving them the chance to object (NHS England).

Under the UK GDPR, every restricted transfer of personal data outside the UK must be covered by UK adequacy regulations, appropriate safeguards or an exception (ICO).

4. State-level and emerging regulatory requirements

4.1 California privacy law framework

4.1.1 California Invasion of Privacy Act (CIPA)

The California Invasion of Privacy Act (CIPA), Cal. Penal Code § 630 et seq., applies directly to ambient AI documentation systems (§ 630). Enacted in 1967, CIPA requires the consent of all parties to record a confidential communication, and a person injured by a violation may recover the greater of $5,000 per violation or three times actual damages, without needing to show actual damages (§ 637.2).

In Saucedo v. Sharp HealthCare and Washington v. Sutter Health, plaintiffs argue that ambient AI systems record confidential patient-clinician conversations without all-party consent and that sending the captured audio to the vendor's servers is an unauthorized disclosure (KPBS) (complaint).

4.1.2 California Confidentiality of Medical Information Act (CMIA)

The California Confidentiality of Medical Information Act (CMIA), Cal. Civ. Code § 56 et seq., complements CIPA by prohibiting providers from disclosing medical information without prior authorization, subject to statutory exceptions (§ 56.10). The litigation against Sharp HealthCare and Sutter Health alleges that the transmission of patient-clinician conversation data to Abridge for processing violated the CMIA's authorization requirements (KPBS) (complaint).

4.2 State AI statutes

4.2.1 Colorado: SB 24-205 and SB 26-189

Colorado enacted the Colorado Artificial Intelligence Act as Senate Bill 24-205, signed on May 17, 2024. It would have required deployers of high-risk AI systems to maintain risk management policies, complete impact assessments, notify consumers, and give them an opportunity to correct data and appeal adverse consequential decisions (SB 24-205). Its requirements were delayed to June 30, 2026 by SB 25B-004 (SB 25B-004) and never took effect: SB 26-189, signed on May 14, 2026, repealed and re-enacted the framework as a regime for automated decision-making technology, with duties applying from January 1, 2027 (SB 26-189).

Under SB 26-189, developers must give deployers technical documentation describing a covered system's intended uses, categories of training data, known limitations, and instructions for appropriate use and human review. Deployers must give consumers clear and conspicuous notice at the point of interaction and, within 30 days of an adverse decision, a plain-language description of the system's role; consumers may request correction of factually incorrect personal data and meaningful human review and reconsideration of an adverse consequential decision. The Attorney General enforces the Act through the Colorado Consumer Protection Act and, before January 1, 2030, must give a 60-day notice and opportunity to cure before bringing an enforcement action (SB 26-189).

4.2.2 Texas TRAIGA (HB 149)

The Texas Responsible Artificial Intelligence Governance Act, enacted as House Bill 149 and effective January 1, 2026, does not impose risk assessment, consequential-decision notice or human review duties on private-sector deployers. It requires a governmental agency that makes available an AI system intended to interact with consumers to disclose that fact, and it requires the provider of a health care service or treatment in which an AI system is used to give that disclosure to the patient or the patient's personal representative no later than the date the service or treatment is first provided, except in an emergency (Section 552.051). It also prohibits developing or deploying AI for specified purposes, including behavioral manipulation to incite self-harm, harm to others or criminal activity, and intentional unlawful discrimination. The Attorney General has exclusive enforcement authority, with civil penalties and a 60-day cure period (HB 149, enrolled text).

4.2.3 California AI legislation (AB 489)

California's Assembly Bill 489, signed on October 11, 2025 (Chapter 615, Statutes of 2025), prohibits the use, in the advertising or functionality of an AI or generative AI system, of terms, letters or phrases that indicate or imply that the care, advice, reports or assessments offered through the technology are being provided by a natural person holding the appropriate health care license or certificate. It makes the existing professional-title protections enforceable against entities that develop or deploy such technology (AB 489).

5. Summary of primary sources and status