What do FDA records show about updates to AI medical devices?
A peer-reviewed audit found that 56 of 163 sampled AI-enabled devices had at least one recorded algorithm update. Public files often omitted participant details, sample sizes or evidence of clinical benefit, limiting what clinicians and patients can learn after a device changes.
The short version
- What happened
- Researchers traced 352 FDA 510(k) entries for 163 AI or machine-learning devices and identified 56 devices with at least one recorded algorithm update, represented by 106 entries.
- Who it affects
- Clinicians need version-specific evidence when a device's inputs, features or model change; an original clearance summary may no longer describe the system in use.
- What remains unproven
- The cohort was historical and narrower than the FDA's full current list of AI-enabled devices.
Keep in mind
The cohort was historical and narrower than the FDA's full current list of AI-enabled devices.
The analysis used public 510(k) records and could not observe confidential evidence or changes that did not generate a separate modification request.
One reviewer performed the database search before two reviewers classified candidate algorithm updates, leaving some risk of missed records or misclassification.
The study describes reporting and update frequency; it does not measure diagnostic accuracy, patient harm, safety events or clinical benefit.
Most devices analysed medical images, which limits generalisation to other specialties and forms of clinical AI.
Source analysis · 2 linked sources. Primary analysed source date: 5 November 2026. Earliest linked supporting source: 4 September 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The finding: algorithm changes were common, but their evidence trail was uneven
A peer-reviewed JAMA Network Open research letter published on 5 November followed a defined historical cohort of 163 medical devices that explicitly used artificial intelligence or machine learning and had entered the US Food and Drug Administration's 510(k) pathway. The researchers retrieved 352 public database entries linked to those devices, including original clearances and later modification requests. They found 56 devices—34.4% of the cohort—with at least one recorded algorithm update, represented by 106 entries and ranging from one to thirteen updates per device.
The median time to a first identified update was 1.8 years, with an interquartile range from 0.9 to 2.5 years. Among the 56 updated devices, 42 had at least one change that altered how the previous version was used, nine had updates described only as improving algorithm performance, and five had changes whose nature was unclear in the public record. The study therefore documents a lifecycle problem: a cleared AI device can change materially after its first authorization, while the public explanation of that change may remain incomplete.
This does not mean a third of all current AI medical devices were secretly changed, nor that the identified updates harmed patients. It is a document audit of a historical cohort, not a clinical outcomes study. The denominator includes devices initially cleared from May 2015 through December 2021, and the search followed their related entries only through 31 March 2025. Its value is in showing what could and could not be reconstructed from the regulator's public files.[1]
How the audit was done—and what counted as an update
The authors began with a previously identified cohort of devices explicitly reporting AI or machine learning and cleared through 510(k) between 9 January 2019 and 29 December 2021; some devices in that cohort had earlier initial clearance dates. One reviewer searched the FDA database and extracted every matching entry. Two reviewers then judged whether each modification request involved an algorithm update and extracted its characteristics. The study followed the STROBE reporting guideline and used descriptive analysis in R.
Updates were separated into changes to previous usage and changes described only as performance improvements. Across 81 entries, the researchers identified 95 usage changes because one entry could contain more than one change. Twenty-three added features, such as an alert capability, while 72 changed algorithm input, such as adding blood-pressure data to a system that already used electrocardiogram signals. Twenty-one entries described performance-only improvement: ten added data, eight added neural networks and three changed architecture.
Most of the cohort—89%—analysed medical images. That concentration matters because reporting patterns in radiology and imaging software may not transfer to every clinical AI product. The paper also relies on what manufacturers and the FDA put into public summaries. A modification described in vague language can be misclassified, while an internal change that never generated a relevant submission cannot appear in the audit at all.[1]
The distinction that matters
What the FDA record audit could reconstruct
The denominators narrow at each step; a recorded update is not the same as a measured clinical benefit.
163 devices
Trace 352 linked 510(k) entries
Historical study cohort
56 devices
Identify 106 algorithm-update entries
At least one recorded update
42 devices
Classify changes to prior use
Use changed at least once
8 devices
Find clinical or organizational benefit
Benefit reported in public records
The missing information clinicians would need to judge a changed system
For each updated device, the researchers assessed whether public records described the data and people used for performance evaluation, the sample size, metrics and results, and any clinical or organizational benefit. Participant information was absent from 41 of 106 update entries. At the device level, 24 of 56 devices—42.9%—had no participant information in any update entry. Sample size was missing from 38 entries, leaving 15 devices with no sample-size information across their update records.
Metrics and results were more often present, but not universal. Fourteen entries omitted both, and eight devices had no entry reporting them. Evidence closest to patient care was rarest: 86 of the 106 entries did not report an organizational or clinical benefit, and 48 of 56 updated devices lacked that information across all associated update entries. A technical score can establish that a revised model performs on a test set; it does not show that the change improves decisions, workflow or health outcomes.
These are reporting gaps, not proof that evaluations were never conducted. Some information may exist in non-public material reviewed by the FDA or in manufacturer files. But public evidence has a distinct purpose: it lets clinicians, hospitals, patients and independent researchers understand what changed and whether evidence still matches the population and setting where the device is used. Missing denominators or participant descriptions make that scrutiny harder.[1]
Why the 510(k) route and change-control plans complicate the picture
The 510(k) pathway generally asks a manufacturer to establish substantial equivalence to a legally marketed predicate device. The paper examines updates that were visible through modification requests in that system. It does not cover every route by which a medical device can reach the US market, every AI feature inside a broader product or every modification made after deployment. The FDA's current public AI-enabled device list is broader and continuously updated, so it should not be treated as the study denominator.
The authors highlight a newer source of undercounting: a predetermined change control plan can set boundaries for future modifications, allowing specified changes without a separate new marketing submission. That can make planned learning and maintenance more manageable, but it also means a search for modification requests will not necessarily reveal every later version. The audit cannot determine whether devices without an identified update stayed unchanged or were modified through another permitted route.
A practical lifecycle record would connect a released software version to its intended use, inputs, evaluation population, sample size, comparator, metrics, known limitations and monitoring plan. Hospitals also need to know when the version they evaluated has changed. Procurement and governance teams should ask for version-specific evidence and change logs rather than assume the original clearance summary permanently describes the product in service.[1][2]
What would make the evidence more useful
The study supports standardized public reporting, but it does not test which template or enforcement mechanism works best. A useful minimum would state the old and new version, reason for the change, affected use, training and evaluation data, patient and site characteristics, sample sizes, uncertainty, subgroup results and whether clinical workflow or outcomes were assessed. Records should distinguish a new input, a new feature and a model retraining rather than collapsing them into a generic software modification.
Independent follow-up should link regulatory records to deployed versions, recalls, adverse-event reports and prospective performance monitoring. It should examine whether reporting quality differs by specialty, manufacturer, risk class and use of a change control plan. Patient-facing communication also matters: a hospital may know that software was updated while a patient affected by its output has no practical way to see what changed or how the new version was evaluated.
The assessment would improve if future cohorts captured all relevant devices and update routes, reviewers reported agreement on update classification, and outcome studies tested whether transparent records change procurement, clinical oversight or safety. It would weaken if missing public details consistently corresponded to strong confidential evidence with effective monitoring, although that would still leave a public accountability gap. For now, the finding is about traceability—not a verdict that every updated device is unsafe.[1][2]
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Continue the story
Read next
Continue exploring clinical evidence: compare the finding and its evidence limits.
What should Europe require before hospital AI becomes routine?
A same-day report from Europe’s science and medical academies argues that health AI needs evidence suited to changing systems, interoperable data, accountable regulation and a workforce able to challenge the technology—not simply more pilots.
Read the next reportExplore the full source and evidence record
What this means for people
- Clinicians need version-specific evidence when a device's inputs, features or model change; an original clearance summary may no longer describe the system in use.
- Patients and hospital governance teams cannot assess representativeness when evaluation populations or sample sizes are absent from public update records.
- Developers face a legitimate need to maintain software, but clearer change records can reduce uncertainty and make post-market monitoring more targeted.
Global context
The audit concerns the United States FDA pathway, but lifecycle transparency is a global problem for software that changes after approval or procurement. Regulatory routes, device definitions and public databases differ across regions. Hospitals using the same product in another country should check local authorization, version control and post-market duties rather than assuming a US 510(k) record settles those questions.
What the evidence does not yet show
- The cohort was historical and narrower than the FDA's full current list of AI-enabled devices.
- The analysis used public 510(k) records and could not observe confidential evidence or changes that did not generate a separate modification request.
- One reviewer performed the database search before two reviewers classified candidate algorithm updates, leaving some risk of missed records or misclassification.
- The study describes reporting and update frequency; it does not measure diagnostic accuracy, patient harm, safety events or clinical benefit.
- Most devices analysed medical images, which limits generalisation to other specialties and forms of clinical AI.
What to watch next
- Whether FDA public records adopt more consistent fields for algorithm version, evaluation population, sample size and clinical benefit.
- Studies linking updates to deployed versions, adverse events, recalls and real-world performance.
- How predetermined change control plans affect the visibility and oversight of later modifications.
- Whether hospitals disclose material model updates to clinicians and affected patients.
Living evidence record
Impact record IAI-1NRSSD3
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
5 November 2026
Source trail
2 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 5 November 2026
Research question, topics and regions
How often a historical cohort of FDA-cleared AI and machine-learning devices received recorded algorithm updates and how completely those changes were reported
Topics: Medical devices · FDA · Post-market monitoring · Clinical evidence · Algorithm updates
Regions: United States · France · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.