New research into the validation processes for medical devices using artificial intelligence has led some experts to believe that their use could “potentially pose risks to patient care.”

The study, the result of a collaboration between a team of researchers from the UNC School of Medicine, Duke University, Ally Bank, Oxford University, Colombia University and the University of Miami, found that many of the AI-powered medical devices approved by the U.S. Food and Drug Administration have not been validated on real patients. The team warned that using such tools in medical settings without clinical validation could put patients at risk.

“Although AI device manufacturers boast about the credibility of their technology and FDA approval, approval does not mean that the devices have been properly tested for clinical effectiveness using real patient data,” said the study’s lead author, Sammy Chouffani El Fassi, a medical student at the UNC School of Medicine and research fellow at the Duke Heart Center, in a press release about the findings.

The team published its analysis in Natural medicineIt provides a comprehensive overview of more than 500 FDA-cleared AI algorithms (each tailored for use in the medical field) and details on product validation.

They found that about 28% of the algorithms had been analyzed retrospectively and another 28% had been validated prospectively. Only 22 of the 521 algorithms had been validated using randomized controlled trials – the gold standard for clinical validation. In addition, 43% had no publicly available clinical validation data, and some devices had not been validated using human data at all, but rather using “phantom images.”