Regulation (EU) 2024/1689 · Artificial Intelligence Act ● Annex III: — days

AI in healthcare under the EU AI Act: clinical decision support and high-risk duties

Clinical AI now sits in the EU AI Act's high-risk tier. This guide covers Annex III point 5(a), the MDR interplay, hospital deployer duties and health-tech vendor obligations.

Renomira Team · 11 min read
A radiology control room with brain CT scans on monitors beside the scanner
Photo: Goleisureintl / CC BY 4.0 via Wikimedia Commons

Clinical decision support is no longer a quiet procurement decision for European hospitals: under the EU AI Act, a system that helps clinicians triage, diagnose or predict deterioration is high-risk — the health ai high risk classification eu in a nutshell — with defined duties for the vendor and the hospital alike. This guide covers AI act clinical decision support obligations practically: the Annex III point 5(a) catch, the MDR interplay, hospital deployer duties, and 2026 vendor due diligence.

Why clinical AI lands in the high-risk tier (Annex III point 5(a))

The AI Act’s high-risk category is a defined legal list with two entry routes: being a safety component of a regulated product, or appearing in Annex III’s enumeration of sensitive use cases.

Healthcare sits squarely in Annex III. Point 5(a) captures AI systems intended for use in medical contexts — clinical assessment, diagnosis, treatment support, triage and monitoring. Once caught, the full regime attaches: risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy and robustness testing, plus conformity assessment and EU database registration before the product reaches the market.

Two clarifications matter. Intended purpose governs, not marketing copy: a radiology tool sold as “workflow efficiency” software still qualifies if its output guides a clinical decision. And penalties reach both sides — high-risk non-compliance sits in the €15 million or 3% of global turnover fine tier, and it can land on the deployer as readily as the provider.

One device, two legal bases: the AI Act and the Medical Device Regulation

Clinical software that reaches a patient usually already has a legal home under the Medical Device Regulation (EU) 2017/745 — so the ai act medical device software relationship is an interplay question, not a replacement. Software intended for diagnosis, prevention, monitoring or treatment is software as a medical device under MDR, typically classified under Annex VIII Rule 11, and needs a CE mark and a maintained technical file.

The AI Act adds a second legal basis rather than swapping one: a triage tool can be a Class IIa device under MDR and a high-risk AI system under Annex III point 5(a) at once. The duties differ in kind — MDR examines the device (safety, performance, clinical evaluation) while the AI Act examines the system as AI (training-data governance, operation logging, human oversight, model-specific risk management such as distribution shift). An MDR technical file covers none of that.

For a mid-market hospital group such as the fictional Lumenfeld Klinikverbund — three sites, roughly 900 staff — the practical rule is to ask vendors for both artefacts: the MDR CE certificate and the AI Act declaration of conformity with a technical documentation summary. Treat “we are CE-marked, therefore compliant” as an incomplete answer.

What hospitals owe as deployers (healthcare provider ai act deployer duties)

A hospital does not become a manufacturer by buying a clinical AI tool, but it becomes a deployer, and deployers of high-risk systems carry binding duties. That is the core of the healthcare provider ai act deployer duties — and where most European hospitals are least prepared, because no MDR obligation ever fell on the buying trust.

Under Article 26, a hospital must:

  • Use it as instructed — within the stated intended purpose, input requirements and performance limits.
  • Assign competent human oversight — named clinicians with the training and authority to question the output, per ward rather than an anonymous “clinical team”.
  • Check input data — whether data fed at the point of use is representative of the local population; a model validated in one national health system may not transfer to another.
  • Retain operation logs so a review can reconstruct what the model saw and recommended.
  • Tell people when AI is in the decision — patients subject to AI-assisted decisions, and staff affected by them, must be informed.
  • Feed the DPIA — the deployment belongs in the hospital’s Article 35 impact assessment; public-body deployers also face the fundamental rights impact assessment.

None of this transfers to the vendor by contract: a clause making “the supplier responsible for all AI Act compliance” does not discharge deployer duties. At Lumenfeld the honest owner is the medical director’s office, working with the DPO.

Four hospital use cases classified at a glance

The table applies the ai act diagnostic software obligations and the wider Annex III logic to systems a typical European hospital runs today:

Use case High-risk? Why First duty
Radiology triage (prioritise suspected findings on a worklist) Yes Output guides clinical diagnosis; Annex III point 5(a) medical AI Provider: conformity assessment, Annex IV documentation, EU database registration
Sepsis alert at the bedside Yes Direct input into urgent treatment decisions on individual patients Deployer: assign trained human oversight, document the escalation protocol
No-show predictor for outpatient clinics Probably not Scheduling optimisation; no clinical decision on a patient’s health Still assess GDPR profiling duties and patient transparency
Admin scheduling / bed forecasting No Operational resource planning; not a clinical purpose Log it in the AI register; review if its purpose drifts

The dividing line is clinical influence, not data sensitivity. A no-show model reads personal data yet sits outside Annex III because its output never touches care; a tool touching no identifiable data can still be high-risk if it drives clinical decisions. Classify by intended purpose first, then layer GDPR on top — the questions are independent.

A system can also change tier without changing vendor: if Lumenfeld’s scheduling tool starts suggesting clinical priority ordering, its purpose has moved into Annex III point 5(a). Put a reclassification clause in the contract.

When the health-tech vendor becomes the provider

For a SaaS company selling clinical AI into hospitals, the decisive question is not whether you host software, but whether you place a high-risk system on the EU market under your own name. Vendors that develop the model, set the intended purpose and brand the output are providers, and the medical ai eu regulation compliance burden lands on them in full:

  • Technical documentation (Annex IV). Design, development process, data governance and validation results — written for an assessor, not a sales deck. It is the first artefact regulators request.
  • Risk management and data governance. Documented handling of model risks (drift, subgroup performance, robustness) and curated training, validation and testing data, including how bias across age, sex and language was searched for.
  • Conformity assessment and registration. Complete the assessment before market placement, engage a notified body where the route requires it, issue an EU declaration of conformity, and register the system in the EU database. An unregistered system ends the conversation.
  • Post-market monitoring and incident reporting. Collect performance data continuously and report serious incidents to market-surveillance authorities within the required window — the AI Act’s 15-day incident reporting where it applies.
  • A lifecycle quality management system, plus named compliance responsibility at management level.

One carve-out matters commercially: an organisation that modifies a third-party high-risk system for its own use — fine-tuning a general radiology model on local data for one trust — can become a provider itself. That trap catches in-house innovation teams who assume the original vendor’s CE mark follows the modified model. It does not.

The GDPR health-data interface

Health data is special-category data under GDPR Article 9, so every clinical AI deployment is an AI Act question and a data protection question at once. Lawful basis: the AI Act supplies none. DPIA: systematic health-data evaluation almost always triggers one, with Article 26 oversight and logging as controls. Article 22: where a triage score materially contributes to significant decisions about patients, human review and contestability are required — rubber-stamping does not satisfy this. Role maps: hospitals are deployers and controllers, vendors providers and often processors; each authority asks different things of the same contract.

Vendor due diligence: what to ask in 2026

The Act’s general applicability — transparency duties, enforcement powers and the penalty regime — runs from 2 August 2026, while the Annex III high-risk obligations are not expected before December 2027 after the Digital Omnibus adjustment. That gap is due-diligence time. Against every clinical AI supplier, the DPO and Head of Legal should be able to answer:

  • Registration: will you be in the EU database, and what is your intended-purpose statement?
  • Annex IV readiness: ask for the technical documentation index — refusal to name the sections signals trouble.
  • Notified body route: which conformity assessment route applies, and who is your notified body?
  • Logging: can operation logs be exported to our systems for incident review?
  • Human oversight: do you supply what trained overseers need to interpret confidence and failure modes?
  • Incidents and change control: who notifies whom after a serious incident, and how will you flag model updates that change performance or purpose?
  • GDPR: processor terms, sub-processors, and whether hospital data feeds model training.

Make sign-off conditional: no evidence, no deployment.

How to check your own position in three minutes

The free exposure scan asks twelve questions about your AI usage and governance and returns an exposure score, your ranked exposures with the exact article that fired, and your first three moves mapped to the dates that apply. Confirm your email and the full report unlocks, including the classification reasoning behind each flagged system.

Frequently asked questions

Does the EU AI Act apply to my hospital’s AI tools today?

The Act’s core application date has passed in 2026, but the high-risk duties that affect clinical software under Annex III — conformity assessment, registration and deployer duties — are not expected before December 2027 following the Digital Omnibus adjustment. For ai act for european hospitals, treat that as a build deadline rather than a start date: classification, register entries, oversight assignments and vendor evidence should be ready well beforehand, because contract renegotiations take longer than clinical teams expect.

If my product already has a CE mark under the Medical Device Regulation, is that enough?

No. The MDR CE mark addresses safety and performance of the device; it does not cover AI Act duties such as training-data governance, logging architecture or the Annex IV technical documentation. The two coexist — one device, two legal bases. Vendors presenting an MDR certificate as their complete AI Act answer are either early in their own preparation or avoiding the question.

Are we a provider or a deployer if we fine-tune a vendor’s model ourselves?

If you take a third-party model and redefine its intended purpose — fine-tuning it on local clinical data for your own patients and putting your own name to the output — you can become the provider of a modified high-risk system, bringing Annex IV documentation, conformity assessment and registration duties onto your team. Classify the activity with the Head of Legal before training starts, not after the model reaches a ward.

Does GDPR already cover the risks, so the AI Act adds little?

GDPR governs personal data; the AI Act governs AI-system risk. A DPIA produces no technical documentation, logging capability or conformity assessment, and an AI Act file says nothing about lawful basis or data subject rights. Health deployments need both, and the evidence overlaps enough that one governed programme can serve both regimes — but only if the DPO and the AI governance lead work from the same register.

What happens if a sepsis alert model misses a case?

Clinical liability still sits with the care team and institution under national law, but the AI Act adds a separate question: did the deployer fulfil its Article 26 duties — competent oversight, instructions followed, logs retained? Serious incidents must be reported by the provider to market-surveillance authorities within the required window, and retained operation logs are what make reconstructing the model’s behaviour possible. Absent logs, the hospital defends itself with nothing.

Next step

Put this guide to work on your own estate.

Run the free exposure scan to see which of these articles actually apply to your company — 12 questions, four minutes, no card. Confirm your email and the full report unlocks.

  • Free forever plan
  • No credit card
  • EU-hosted · GDPR-native
Run a free scan Sign up free

Build a live AI register in minutes — upgrade only when you need the audit pack.

ai act clinical decision support obligationshealth ai high risk classification euai act medical device software relationshipai in hospitals compliance requirementsai act diagnostic software obligationshealthcare provider ai act deployer dutiesmedical ai eu regulation complianceai act for european hospitals

Keep reading