Here's the uncomfortable truth: no LLM ships "HIPAA certified." Compliance isn't a feature you buy. It's how you deploy, govern, and monitor the model over time. OpenAI, Google, and Anthropic each publish HIPAA eligibility pages listing which specific products, tiers, and endpoints qualify — and none of it works without a signed Business Associate Agreement (BAA).
This guide covers what HIPAA actually requires, which tools qualify (and which don't), the four deployment architectures available to you, and the operational work that determines whether compliance holds up after launch.
Key Takeaways
- No LLM is inherently HIPAA compliant — it depends on signed BAAs, architecture choices, and Security Rule controls
- Free-tier ChatGPT, Gemini, and Claude are never compliant; only enterprise/API tiers qualify, and only with a BAA
- Four viable architectures exist: cloud endpoints under a BAA, direct vendor API, self-hosted open-weights models, and on-prem GPUs
- Workforce training and governance decide whether compliance holds once real staff start using the tools
Does HIPAA Apply to AI and LLMs?
Yes, and the trigger is lower than most people assume. HIPAA governs Covered Entities and Business Associates handling protected health information. The moment someone types a patient's name alongside any health detail into an LLM prompt, that prompt is PHI — full stop.
HHS applies the same rules to that AI workflow that it applies to a fax machine or an EHR:
- Privacy Rule: Protects individually identifiable health information held or transmitted by covered entities or business associates
- Security Rule: Requires administrative, physical, and technical safeguards for ePHI
- Breach Notification Rule: Requires notice when unsecured PHI is compromised
In its 2025 Security Rule proposal, OCR stated that ePHI used in AI training data, prediction models, and algorithm outputs remains protected, and organizations using AI must include those uses in their risk analysis.
Those obligations are easy to misread in practice.
Common misconception: "A human reviews every output, so we're covered."
Wrong. A clinician reviewing AI-generated notes does not exempt the underlying prompt-and-response flow from HIPAA. If PHI touched the model, HIPAA obligations exist regardless of who checks the output afterward.
Are Any LLMs HIPAA Compliant? (ChatGPT, Gemini, Claude, and Others)
Only specific enterprise tiers can be HIPAA-eligible, and only after you sign a BAA and stay inside the covered feature set. Brand name alone does not make a model compliant.
Free and consumer versions are never compliant. ChatGPT Free and consumer Gemini offer no BAA, and prompts may be logged or used for model training. Don't use them for anything involving a patient.
Enterprise tiers become eligible only with a signed BAA:
| Vendor | BAA-Eligible Products | Key Caveat |
|---|---|---|
| OpenAI | ChatGPT Enterprise, ChatGPT for Healthcare, API with Modified Retention | Excludes some connectors, internet access, and memory features; unlisted features aren't automatically covered |
| Microsoft Azure | In-scope Azure services under default BAA | Azure OpenAI isn't explicitly named on Microsoft's HIPAA page — get product-level written confirmation |
| Google Cloud/Gemini | Vertex AI Workbench, Gemini Enterprise, Gemini in BigQuery | Customer remains responsible for evaluating its own compliance |
| Anthropic Claude | First-party API and Enterprise plans (BAA must be activated) | Batch API, Files API, Computer Use, and Web Fetch are excluded from coverage |
BAA coverage is almost always narrower than the full product. A vendor may cover the core chat endpoint but exclude plugins, memory features, or specific API calls. Enable the wrong feature, and you can break compliance silently — until an audit catches it.
Compliance is shared. The vendor's BAA covers their infrastructure and contractual obligations. Risk analysis, access controls, and staff training remain entirely your responsibility, regardless of how complete the vendor's paperwork appears.
The Four Architectures for a HIPAA-Compliant LLM Deployment
Every viable deployment falls into one of four patterns. Each trades control for operational effort differently.
Architecture 1: Cloud Model Endpoint Under Existing Cloud BAA
Using Bedrock, Azure OpenAI, or Google Vertex through your existing cloud BAA. This is usually the lowest lift if you're already running infrastructure with that provider.
- Requires region pinning and IAM scoping to keep data in-boundary
- Access to frontier-quality models without standing up new infrastructure
- Still requires verifying the specific service is on the provider's covered list
Architecture 2: Direct Vendor API With a Dedicated BAA
A new business associate relationship directly with OpenAI, Anthropic, or similar.
- Narrower covered feature set than the full consumer product
- Requires ongoing vendor management discipline — someone has to track feature changes
- More control over the contract surface than a shared cloud BAA, with added vendor-oversight effort
Architecture 3: Self-Hosted Open-Weights Models
Running Llama, Mistral, or Qwen in a private VPC.
- No model vendor ever touches PHI — removes that BAA requirement entirely
- Your organization owns the entire Security Rule burden: encryption, logging, patching, access control
Architecture 4: On-Premises GPU Infrastructure
The only truly "no BAA required" option, since no third party touches PHI at all.
- Favored by large health systems with existing data-center operations
- Highest capital cost, but zero external data-sharing exposure
Decision Guidance
Smaller practices and mid-size organizations typically get the best cost-to-compliance ratio from Architecture 1 or 2. Architecture 3 or 4 fits organizations with strict internal policies against any external data sharing — or those that already run substantial infrastructure.
Architecture choice isn't academic. IBM's 2024 Cost of a Data Breach study puts the average healthcare breach cost at $10.93 million — the highest of any industry tracked. Getting the architecture wrong isn't a compliance footnote; it's a balance-sheet risk.
The Compliance Checklist Every Deployment Needs
Before any PHI touches an LLM, confirm these five items are in place:
- Signed BAAs with every vendor in the PHI path — model provider, cloud host, and any middleware in between
- Written no-training and data retention guarantees — not just default settings buried in a dashboard
- Immutable audit logging — every prompt and response, retained per policy and queryable if OCR ever comes calling
- Role-based access control and encryption — minimum necessary access, with data encrypted in transit and at rest
- Output validation and human review — catch hallucinations before they reach a clinical workflow
Miss any one of these, and a technically "compliant" vendor relationship still leaves you exposed.
Why Technically Compliant Deployments Still Fail (The Organizational Gap)
Here's what the case files show: most failed healthcare AI projects don't fail on architecture. They fail because staff weren't trained on what they can and can't type into a tool, governance policies existed on paper but weren't enforced, or configuration quietly drifted over months.
Verizon's 2025 Data Breach Investigations Report found human involvement in roughly 60% of breaches. Two recent OCR settlements make this concrete: the agency's actions against Deer Oaks and BST & Co. both cited inadequate risk analysis and missing workforce training, not model failures or infrastructure gaps. Process failures.
A BAA covers the vendor's contractual obligations. It does not cover:
- Whether your front-desk staff know PHI shouldn't go into a personal ChatGPT account
- Whether someone re-enables an excluded feature six months after go-live
- Whether your risk analysis gets updated when you add a new integration
This is where BestResults.AI's Proven Paths™ methodology fits the gap. It pairs the right architecture decision with the operational layer a signed BAA never touches:
- Structured assessment and governance planning
- Hands-on staff training and deployment support
- Ongoing monitoring so configuration does not drift
The goal is adoption that sticks six months later, without adding headcount to manage it.
Treat compliance architecture as a foundation you build on day one, not something you retrofit after an incident. Retrofitting always costs more in time, remediation, and occasionally a six-figure OCR settlement.
Frequently Asked Questions
Are any LLMs like ChatGPT or the OpenAI API HIPAA compliant?
No LLM is compliant by default. ChatGPT Enterprise and the OpenAI API become HIPAA-eligible only with a signed BAA and configuration limited to approved endpoints. Free or consumer ChatGPT is never compliant.
Does HIPAA apply to AI and LLMs?
Yes. HIPAA applies whenever a Covered Entity or Business Associate processes PHI through AI, including a single prompt containing patient identifiers. This triggers Privacy, Security, and Breach Notification Rule obligations.
Can I use an open-source LLM for HIPAA workloads?
Yes, self-hosting open-weights models removes the need for a vendor BAA. However, it shifts the entire Security Rule burden onto your organization: encryption, access control, logging, and patching.
What happens if my organization sends PHI to a non-compliant LLM by mistake?
This is an impermissible disclosure. It requires a breach assessment and may require patient notification. Put staff training and technical safeguards in place before deployment, not after.
Is a signed BAA enough to make an LLM deployment HIPAA compliant?
No. A BAA is necessary but not sufficient. Your organization still owns risk analysis, audit logging, access controls, and workforce training on your side of the relationship.
How long does it take to deploy a HIPAA-compliant LLM solution?
With the right architecture and a structured methodology, organizations can move from assessment to secure deployment in a few months. Skipping the planning stage typically leads to costly rework later.
