Studies Without Randomization are Prone to Various Types of Bias

Medical devices are essential for safe and effective prevention, diagnosis, treatment and rehabilitation of illness and disease. Fewer than half were randomized, blinded, or controlled among clinical studies used to support FDA approval of high-risk medical device modifications, according to a study published by JAMA. High-risk medical devices often undergo modifications, which are approved by the U.S. Food and Drug Administration (FDA) through various kinds of premarket approval (PMA) supplements. There have been multiple high-profile recalls of devices approved as PMA supplements. Rita F. Redberg, M.D., M.Sc., of the University of California, San Francisco, and colleagues examined the strength of evidence of clinical studies used in FDA-approved panel-track supplements (a type of PMA supplement pathway that is used for significant changes in a device or indication for use and always requires clinical data) between 2006 and 2015.

To determine methodological quality, studies were examined for the use of randomization, blinding, type of controls, clinical vs surrogate primary end points, use of post hoc analyses, and reporting of age and sex. Surrogate was defined as "a laboratory measurement or a physical sign used as a substitute for a clinically meaningful endpoint that measures directly how a patient feels, functions or survives."

The approval of 78 panel-track supplements were supported by 83 clinical studies, with 71 supplements (91 percent) supported by a single study. Of the 83 studies, 45 percent were randomized clinical trials and 30 percent were blinded. The median number of patients per study was 185 and the median follow-up duration was 180 days.

Of the primary end points, 121 of 150 (81 percent) were surrogate end points, and 38 percent were compared with controls. Age was not reported in 40 percent of the studies, and 30 percent did not report sex for all enrolled patients.

The authors write that "studies without randomization are prone to various types of bias, making it difficult to ascertain whether these modified devices are safer or more effective than previous iterations, conventional treatments, or no procedure." And that "for surrogate end points to be useful to patients and clinicians, they must be shown to predict meaningful clinical outcomes, which rarely happens. Therefore, use of surrogate measures can lead to uncertainty about clinical outcomes."

The study notes some limitations, including that because the focus was premarket clinical data, the analyses did not include preclinical data supporting panel-track supplements or postmarket studies initiated without FDA requirements.

Advertisement

"These findings suggest that the quality of studies and data evaluated to support approval by the FDA of modifications of high-risk devices should be improved," the researchers write.

Source-Eurekalert