The Stakes Are Different With Protected Health Information
Healthcare organizations want the same AI capabilities as everyone else—intelligent search over clinical notes, automated triage, documentation assistance. But they're working with protected health information (PHI), which means HIPAA compliance isn't optional. A data breach or unauthorized disclosure doesn't just cost money; it erodes patient trust and invites regulatory scrutiny.
The challenge is that most LLM APIs are designed for convenience, not compliance. Sending PHI to a third-party API—even with encryption in transit—may violate your Business Associate Agreement (BAA) unless the vendor has signed one and implemented appropriate safeguards. Even then, you're trusting that vendor's security posture. For many healthcare organizations, that's an unacceptable risk.
Start With Private Deployments
The safest approach: keep PHI on your infrastructure. Deploy models in your VPC, on-premises, or in a private cloud with strict network boundaries. No data leaves your environment. You control access, logging, and retention. This is more expensive and complex than using a public API, but it's the only way to guarantee that PHI never touches a third party.
If you must use a third-party LLM provider, verify they've signed a BAA, they support private endpoints, and they don't use your data for training. Document everything. Your compliance team will ask for proof.
Access Control and Redaction
Not everyone should see all PHI. Implement role-based access controls at the application layer. A scheduling assistant doesn't need access to lab results. A billing agent doesn't need clinical notes. Scope permissions tightly. Log every access.
For some use cases, you can redact or anonymize PHI before it reaches the LLM. If you're building a triage tool that routes patients based on symptoms, you might not need their name or date of birth. Strip identifiers, keep the clinical context. This reduces risk without sacrificing utility.
Be careful with re-identification. Even if you remove names, a combination of quasi-identifiers (age, zip code, diagnosis) can sometimes uniquely identify a patient. Test your redaction logic with a privacy expert before deploying.
Audit Trails and Retention
HIPAA requires that you log access to PHI and maintain those logs for at least six years. Your LLM system should capture: who accessed what data, when, for what purpose, and what the system returned. Store these logs securely, separate from the PHI itself, with tamper-proof timestamps.
Retention policies matter, too. If you're storing prompts and responses, you need a retention schedule. How long do you keep them? Who can access them? When are they deleted? Document the policy, implement it in code, and audit compliance regularly.
Governance and Accountability
Technology alone isn't enough. You need governance: who approves new AI use cases? Who monitors for drift or errors? Who owns the risk if something goes wrong? Establish clear roles and responsibilities. Involve your privacy officer, compliance team, and legal counsel early.
Run a risk assessment for each use case. What's the worst-case failure mode? If the LLM hallucinates a drug dosage or misroutes a critical alert, what happens? Implement safeguards: human review for high-stakes outputs, confidence thresholds, and fallback to manual processes when the system is uncertain.
Training and Change Management
Your staff needs to understand how the AI works and what its limitations are. Train clinicians, administrators, and IT teams on appropriate use. Make it clear that the LLM is an assistant, not a decision-maker. Final clinical judgments always rest with licensed professionals.
Communicate with patients, too. If you're using AI to help with scheduling, triage, or documentation, let patients know. Transparency builds trust. HIPAA doesn't require explicit patient consent for all AI uses, but it's often the right thing to do.
Moving Forward Safely
HIPAA-aligned LLM delivery is absolutely possible, but it requires intentional design. Start small: pilot with a low-risk use case, document your controls, measure outcomes, and iterate. Don't rush to scale until you're confident your safeguards work.
The goal isn't to avoid risk entirely—that's impossible with any system. The goal is to manage risk responsibly, comply with regulations, and deliver value to patients and providers. Privacy first, then innovation.