Governing AI in Healthcare: The Emerging Regulatory Landscape for Artificial Intelligence in American Medicine

Executive summary

Artificial intelligence is transforming healthcare delivery faster than the law can keep up.

From diagnostic imaging tools and clinical decision support software to algorithmic insurance claim processing and AI-powered mental health chatbots, these technologies are reshaping the relationship between patients, providers, and the institutions that govern care. Yet the legal infrastructure tasked with overseeing these tools remains fragmented, contested, and in significant flux.

This white paper examines the emerging regulatory landscape for AI in American healthcare as of mid-2026. It surveys three intersecting domains of governance: the growing patchwork of state-level AI healthcare legislation, the evolving posture of the Food and Drug Administration toward AI-enabled medical devices and clinical decision support software, and the accelerating deployment of algorithmic tools in health insurance utilization review. Across each domain, a central tension emerges between the promise of AI to improve efficiency, access, and clinical outcomes and the risks of algorithmic bias, opacity, and the erosion of human oversight in consequential medical decisions.

For public agencies, municipal governments, healthcare providers, and policymakers navigating this landscape, the stakes are considerable. The decisions made in the coming months regarding AI governance in healthcare will shape patient safety, health equity, and the boundaries of algorithmic authority in American medicine for years to come.

I. Introduction: a critical inflection point

The integration of AI into healthcare has advanced from theoretical possibility to operational reality. Over one thousand AI and machine-learning-enabled medical devices have received FDA authorization, spanning applications from tumor detection on imaging scans to wearable cardiac monitors and digital therapeutic apps. Nearly half of U.S. healthcare organizations report implementing generative AI technologies in some capacity. The market is vast, the applications are multiplying, and the regulatory environment is struggling to keep pace.

What makes 2026 a particularly consequential moment is the convergence of several regulatory developments. Multiple state AI laws took effect on January 1, 2026, targeting healthcare applications specifically. The FDA issued revised guidance in January 2026 that significantly softened its oversight of certain AI-enabled health products. The Colorado AI Act, widely regarded as the most comprehensive state-level AI consumer protection law in the nation, is set for enforcement beginning June 30, 2026. And at the federal level, the current administration has signaled a strong preference for deregulation, issuing executive orders aimed at discouraging state-level AI regulation while declining to advance comprehensive federal legislation.

The result is a governance environment defined by fragmentation, tension between state and federal authority, and significant uncertainty for the organizations developing, deploying, and using AI in clinical and administrative settings. This white paper provides a structured analysis of these developments, with particular attention to the implications for public interest stakeholders, municipal health agencies, and the patients whose care increasingly depends on algorithmic decision-making.

II. The state-level regulatory patchwork

A. A surge of healthcare-specific AI legislation

In the absence of comprehensive federal AI legislation, states have moved aggressively to regulate AI in healthcare. Thirty-eight states passed AI-related legislation in the past legislative cycle, and several laws that took effect on January 1, 2026, directly target the use of AI in clinical settings, insurance processing, and patient-facing communications.

California has been particularly active. Building on prior legislation requiring healthcare providers to disclose AI use (AB 3030 and SB 1120, effective January 2025), the state enacted AB 489, effective January 1, 2026, which prohibits AI system developers and deployers from using terms, design elements, or branding that could mislead patients into believing they are interacting with a licensed healthcare professional. The state has also extended requirements for mental health chatbots, mandating protocols to prevent responses that could encourage self-harm and requiring referral notifications for users expressing suicidal ideation.

Texas enacted one of the broadest AI healthcare laws in the country. The Texas Responsible Artificial Intelligence Governance Act (TRAIGA), effective January 1, 2026, establishes disclosure requirements for any healthcare provider using AI in connection with services or treatments. Providers must furnish written notice to patients prior to or on the date of service, except in emergencies. Ohio and Pennsylvania have introduced legislation that would require written informed consent from patients regarding AI use, signaling an emerging legislative trend toward affirmative patient authorization rather than passive disclosure.

Illinois has taken a targeted approach to mental health applications. An amendment to the Managed Care Reform and Patient Rights Act, effective August 2025, prohibits the use of AI in therapy or psychotherapy to make independent therapeutic decisions, interact directly with clients in therapeutic communication, or generate treatment plans without review and approval by a licensed professional. This represents one of the most explicit statutory limits on autonomous AI clinical decision-making in any state.

B. The Colorado AI Act: a national test case

The Colorado AI Act (SB 24-205), with enforcement beginning June 30, 2026, represents the most comprehensive attempt by any U.S. state to regulate high-risk AI systems across multiple sectors, including healthcare. The Act requires deployers of high-risk AI systems to exercise reasonable care to protect consumers from algorithmic discrimination and mandates impact assessments, risk management programs, detailed documentation, and consumer disclosures.

Healthcare is squarely within scope. The Act defines "consequential decisions" to include those with a material legal or similarly significant effect on the provision, denial, cost, or terms of healthcare services. Deployers and developers face obligations to inventory AI systems, assess risk levels, establish governance frameworks aligned with recognized standards such as the NIST AI Risk Management Framework or ISO/IEC 42001, and maintain documentation suitable for regulatory review.

The Act does include notable exemptions for healthcare entities. HIPAA-covered entities providing AI-generated healthcare recommendations that require a provider to take action for implementation, and that are not otherwise classified as high-risk, are excluded. AI systems that have been authorized, approved, or certified by a federal agency such as the FDA are also exempt, as are systems in compliance with ONC standards established under the HTI-1 Final Rule.

Colorado's experience will be closely watched nationally. Several states have modeled draft legislation on the Colorado framework, and the Act's fate may influence whether other jurisdictions pursue similarly comprehensive approaches or opt for narrower, sector-specific regulation. The tension between Colorado's framework and the federal administration's deregulatory posture also raises significant preemption questions that may ultimately require judicial or congressional resolution.

C. Federal preemption and the state-federal tension

The current administration has issued executive orders signaling intent to preempt state AI laws and promote a "minimally burdensome national policy" for AI development. However, executive orders cannot invalidate state statutes; only Congress holds the constitutional authority to enact true preemption through legislation. The executive orders do signal that the federal government may actively oppose enforcement of certain state AI requirements, and the Office of Management and Budget has been directed to consider states' AI regulatory climates when making federal funding decisions.

For healthcare organizations operating across multiple states, this creates a compliance environment of considerable complexity. State laws remain on the books and enforceable, yet the federal posture introduces uncertainty about long-term durability. The prudent approach, as multiple legal analysts have recommended, is to prepare for compliance with the most stringent applicable state requirements while monitoring federal developments closely.

III. FDA oversight of AI-enabled medical devices

A. The January 2026 guidance revisions

On January 6, 2026, the FDA published revised final guidance documents on clinical decision support (CDS) software and general wellness products. Commissioner Martin Makary framed the revisions as efforts to reduce unnecessary regulation and promote AI innovation in healthcare. The changes represent a meaningful shift toward a more permissive regulatory posture for certain categories of AI-enabled health products.

The revised CDS guidance expands the categories of software that qualify for exclusion from medical device regulation under the 21st Century Cures Act. Previously, the FDA had narrowly interpreted these exclusions, treating products that delivered a single clinical recommendation as regulated devices. The updated guidance reverses this position, extending enforcement discretion to such products provided they meet other criteria for exemption, including that a healthcare professional can independently review the underlying basis for the recommendation. The guidance also removes prior language stating that software producing a risk score or probability does not qualify as non-device CDS.

The revised general wellness guidance similarly broadens the category of products that fall outside FDA oversight. It clarifies that certain products using non-invasive sensing to estimate physiological parameters such as blood pressure, oxygen saturation, or blood glucose may qualify as general wellness products rather than medical devices, provided they are intended solely for wellness purposes and do not prompt specific clinical action.

B. Implications and concerns

The regulatory implications of these revisions are significant. A substantial number of AI-enabled software tools and consumer wearables that influence health behavior but do not make clinically unreviewable decisions may now enter the market without FDA premarket review, registration, or compliance requirements. Commissioner Makary has indicated that the agency plans to develop a new risk-based AI framework and has previewed plans to eliminate at least half of existing software and digital health guidance documents.

Critics have raised concerns that the relaxed oversight may expose patients to unvalidated AI tools, particularly given that the vast majority of medical AI has never been reviewed by a federal regulator and likely not by any state regulator either. Harvard Law School's I. Glenn Cohen has noted that when AI handles medium-to-high-risk clinical functions, some form of regulation, whether internal self-governance or external governmental oversight, is essential. The challenge lies in calibrating oversight to the pace of innovation without creating bottlenecks that delay beneficial technologies or, conversely, permitting tools with significant safety implications to proliferate without adequate scrutiny.

The FDA also faces significant capacity constraints. Staffing levels as of late 2025 were down approximately fifteen percent from 2023 levels. While the agency has begun deploying its own internal generative AI tools to assist with scientific reviews, the question of how reduced oversight capacity intersects with a broadened market of AI-enabled health products remains unresolved.

IV. Algorithmic decision-making in health insurance

A. The rise of AI in utilization review

Health insurers have rapidly adopted AI tools for utilization review, the process through which they evaluate requests for coverage of medical procedures, drugs, and services. A 2025 survey by the National Association of Insurance Commissioners found that seventy-one percent of responding health insurers reported using AI for utilization management, encompassing prior authorization and concurrent authorization processes.

The prior authorization process has long been a source of friction in the American healthcare system. It is costly, time-consuming, and a significant contributor to provider burnout and care delays. Even before AI, studies documented high denial rates for authorization requests and correspondingly high reversal rates on appeal, including an eighty-two percent overturn rate documented in Medicare Advantage plans. The introduction of algorithmic processing has accelerated the volume and speed of claim decisions, but has also amplified concerns about accuracy, fairness, and the adequacy of human oversight.

B. Concerns about transparency, bias, and oversight

Research published in Health Affairs in January 2026 identifies several categories of concern with AI in utilization review. These include the opacity of algorithmic determinations, the potential for automation bias (in which human reviewers defer excessively to AI recommendations), organizational pressures that discourage departure from algorithmic outputs, and the absence of publicly available data to assess whether AI produces better or worse outcomes in insurance processes. Insurers have not shared the information that would validate their claims that AI benefits clients, and many lack robust governance processes for monitoring accuracy and potential biases.

The equity implications are particularly troubling. Research has documented that at-risk populations experience significantly higher denial rates. Patients with lower educational attainment and racial and ethnic minorities face disproportionate denial rates, raising concerns that AI systems trained on historically biased datasets may perpetuate and intensify existing healthcare disparities.

C. State responses and the emerging AI arms race

Several states have responded with legislation targeting AI in insurance decision-making. Arizona, Maryland, Nebraska, and Texas have enacted laws prohibiting insurers from using AI as the sole decision-maker in prior authorization or medical necessity denials. New York's pending Assembly Bill A9149 would require health insurers to conduct clinical peer review of AI-based decisions, disclose AI use publicly, and submit algorithms and datasets to state regulators for certification against discrimination. Illinois has amended its Managed Care Reform and Patient Rights Act to address AI in prior authorization, though it permits either healthcare professionals or accredited automated processes to certify medical necessity.

Simultaneously, a parallel development has emerged: patients and providers are deploying their own AI tools to contest insurance denials. Nonprofit organizations and commercial services now offer AI-powered applications that analyze denial letters, cross-reference policy language and clinical evidence, and generate customized appeal letters. This dynamic has been characterized as an "AI arms race" in health insurance, where algorithmic tools are deployed on both sides of coverage disputes, raising fundamental questions about the role of human judgment in consequential healthcare decisions.

V. Implications for public agencies and municipal governments

For public agencies and municipal governments, the current AI healthcare governance landscape presents both obligations and opportunities. Agencies that administer public health programs, operate municipal hospitals or clinics, or contract with healthcare providers and insurers must grapple with the compliance requirements imposed by emerging state laws and the governance expectations established by accreditation bodies.

The Joint Commission and the Coalition for Health AI released guidance in late 2025 recommending governance frameworks for healthcare facilities implementing AI. These recommendations place the compliance burden largely on individual facilities, which raises capacity and equity concerns for smaller or under-resourced public health systems. A voluntary AI certification program is expected later in 2026, and AI governance may increasingly factor into accreditation assessments.

Municipal agencies should consider several practical steps. First, conducting an inventory of all AI systems in use across healthcare operations, including vendor-provided tools embedded in administrative, clinical, and insurance-related workflows. Second, establishing governance policies that address risk management, bias mitigation, clinical oversight, and ongoing monitoring. Third, preparing disclosure and consent mechanisms that meet the requirements of applicable state laws. Fourth, engaging with the evolving federal landscape, including monitoring executive orders and potential congressional action on AI preemption, while maintaining compliance with current state obligations.

Beyond compliance, public agencies have an opportunity to lead by example in establishing governance practices that prioritize transparency, equity, and accountability. The public sector's distinct mandate to serve all community members, including the most vulnerable, positions municipal governments to advocate for AI governance frameworks that center patient welfare rather than deferring to industry-driven standards of acceptable risk.

VI. Conclusion and recommendations

The governance of AI in American healthcare is at an inflection point. The convergence of state legislative action, shifting federal regulatory postures, and the rapid expansion of algorithmic tools across clinical and administrative settings creates a landscape that demands proactive, informed engagement from every stakeholder, particularly those in the public sector whose mandate is to serve community welfare.

We offer the following recommendations for public agencies, policymakers, and healthcare organizations navigating this environment:

Prioritize compliance with the most protective state requirements. Given the fragmented regulatory landscape and the uncertain trajectory of federal preemption efforts, organizations should build governance programs around the strictest applicable standards. The Colorado AI Act, California's healthcare AI disclosure requirements, and Texas's TRAIGA represent current high-water marks that may serve as models for other jurisdictions.

Advocate for transparency and human oversight in algorithmic insurance processes. The deployment of AI in utilization review without adequate transparency, bias monitoring, or meaningful human review poses significant risks to patient welfare and health equity. Public agencies should support legislative and regulatory efforts that require disclosure of AI use in coverage decisions, mandate human review of denials, and establish accountability mechanisms for algorithmic errors.

Monitor FDA guidance developments closely. The agency's shift toward a more permissive posture for AI-enabled health products creates both opportunities and risks. Public health agencies should track the forthcoming risk-based AI framework and engage in public comment processes to ensure that patient safety considerations remain central to regulatory design.

Invest in institutional AI governance capacity. Establishing internal governance frameworks, conducting impact assessments, training staff, and building documentation practices are essential preparations regardless of which specific regulations ultimately take effect. Organizations that invest in governance infrastructure now will be better positioned to adapt to whatever regulatory requirements emerge.

Center health equity in AI governance. The evidence of disparate impacts in algorithmic healthcare decision-making demands that equity considerations be embedded in governance frameworks from the outset, not treated as an afterthought. Public agencies should require bias audits, monitor outcomes across demographic groups, and ensure that AI deployment does not exacerbate existing disparities in access or quality of care.

The technologies are here. The regulatory frameworks are still being written. The choices made in 2026 will shape whether AI serves as a tool for more equitable, effective healthcare or becomes another vector through which systemic inequities are encoded and amplified. For those entrusted with the public interest, the time to engage is now.