Health care services: artificial intelligence

The law bars AI from replacing licensed professionals in functions the law reserves for them and subjects qualifying health chatbots to confidentiality and misuse penalties under California medical-information law.

What the law does ​

• Requires facilities, clinics, and physician and group-practice offices to take reasonable steps to preserve a licensed provider’s independent professional judgment when AI clinical decision support informs patient care.
• Prohibits AI tools from independently performing licensed clinical functions or directing unlicensed personnel to perform them.
• Applies California medical-information confidentiality standards and penalties to businesses offering qualifying health care chatbots.
• Allows licensing boards to seek injunctions or restraining orders when violations amount to unlicensed practice.
• Exempts automated documentation and communications that do not involve professional judgment, such as record-update notices, reminders, and information-finding assistance.

Who it affects ​

• Health facilities, clinics, physician’s offices, and group-practice offices using AI in clinical care.
• Licensed health care professionals and unlicensed personnel involved in clinical functions.
• Businesses offering consumer-facing, generative-AI health care chatbots that use health-related information to support health services.
• Patients whose care or health information is handled through these AI systems.

Breakdown ​

Health Care Chatbot Privacy ​

The bill treats businesses that offer health care chatbots for consumers to manage health information or to diagnose, treat, or manage a medical condition as health care providers under California medical privacy law. This means those businesses must follow CMIA limits on using or sharing medical information. Violations that cause economic loss or personal injury can carry existing misdemeanor penalties.

Key takeaways

• Businesses offering qualifying health care chatbots are newly covered by the Confidentiality of Medical Information Act.
• Covered chatbots include those offered to help consumers manage their information or diagnose, treat, or manage a medical condition.
• These businesses must follow restrictions on sharing, selling, marketing with, or otherwise using medical information outside of providing health care services, unless an exception applies.
• Because the bill expands conduct subject to existing criminal penalties, it creates a state-mandated local program.

Clinical AI Must Preserve Professional Judgment ​

The bill requires health care facilities and medical offices to take reasonable steps to ensure licensed professionals can still use their own independent judgment when clinical decision-support systems inform patient care. It also bars AI tools from independently carrying out clinical tasks that state law requires a licensed person to perform. The rule does not cover automated documentation or routine communications that do not require professional judgment.

Key takeaways

• Health care facilities, clinics, and medical offices must preserve a licensed professional’s ability to make independent decisions when using clinical decision-support systems.
• AI tools cannot independently perform clinical functions that the law reserves for licensed health care professionals.
• Physicians who violate these requirements may be subject to oversight by the Medical Board of California or the Osteopathic Medical Board of California.
• Professional licensing boards may seek court orders to stop violations that amount to practicing a health profession without a license.
• The restrictions do not apply to automated documentation or patient communications that do not involve professional judgment, such as messages about updates to health records.

No State Reimbursement ​

This part says the state does not have to reimburse local agencies or school districts for costs caused by this law. The bill relies on a specified legal reason for that conclusion under California’s state-mandate rules.

Key takeaways

• Local agencies and school districts will not receive state reimbursement for costs imposed by this act.
• The bill expressly addresses California’s constitutional and statutory rules on reimbursing state-mandated local costs.
• The digest does not identify the specified reason for denying reimbursement in this excerpt.