Limitations of Death Prediction Models

A recent review published in JAMA questions about the accuracy and clinical utility of prognostic indices that are used to predict risk of death in older adults. "Failure to consider prognosis in the context of clinical decision making can lead to poor care. Hospice is underutilized for patients with nonmalignant yet life-threatening diseases. Healthy older patients with good prognosis have low rates of cancer screening. Older adults with advanced dementia or metastatic cancer are screened for slow-growing cancers that are unlikely to ever cause them symptoms but may lead to distress from false-positive results, invasive workups, and treatments," according to background information in the article. "In recognition of these phenomena, guidelines increasingly incorporate life expectancy as a central factor in weighing the benefits and the burdens of tests and treatments. Prognostic indices offer a potential role for moving beyond arbitrary age-based cutoffs in clinical decision making for older adults. However, little is known about the quality of prognostic indices for older adults, limiting their clinical use."

Lindsey C. Yourman, M.D., of the University of California, San Francisco, and colleagues conducted a systematic review to examine the quality and limitations of validated non-disease-specific prognostic indices that predict absolute risk of all-cause mortality in patients whose average age was 60 years or older. A review of the medical literature identified 16 indices that predict risk of mortality from 6 months to 5 years for older adults in a variety of clinical settings: the community (6 indices), nursing home (2 indices), and hospital (8 indices; 5 were intended for use in the emergency department or on hospital admission and 3 after hospital discharge). Studies were abstracted for information about index quality, including potential for bias, generalizability, and accuracy.

All indices were developed using secondary analysis of existing data sets of participants from the United States and western Europe. The most common final predictors of mortality included functional status and co-existing illnesses. The researchers found that at least 1 measure of transportability (the index is accurate in more than 1 population) was tested for all but 3 indices. By the measures used, no study was free from potential bias. Only 2 indices were independently validated by investigators who were not involved in the development of the index. No index had been prospectively tested and found to be accurate in a large diverse sample. "These factors limit a clinician's ability to assess the accuracy of these indices across patient groups that differ according to severity of illness, methodology of data collection, geographic location, and time," the authors write.

The researchers add that even if quality barriers are overcome, important limitations remain, such as indices requiring collection of information that may not be routinely assessed in elderly patients, like activities of daily living. They add that many of these indices rely on clinical information from administrative data sets, and the accuracy of certain codes has been called into question.

"Ultimately, an index will be judged not only on its accuracy across diverse settings, but also on its clinical effect. Studies that demonstrate effect on prognostic estimates, clinician behavior, and patient outcomes have a higher level of evidence for use in clinical decision making. We are aware of only 2 small studies that tested the effect of these indices on clinical outcomes. The highest level of evidence, however, would come from large prospective trials that randomize clinicians to using the index or not, evaluating the effect of the index on prognostic estimates, clinical decision making, and patient outcomes. Such large randomized trials have not been performed," the authors write.

"While neither a clinician nor an index can predict with absolute certainty how long an older adult will live, validated prognostic indices might improve the accuracy of the prognostic assumptions that influence clinical decisions. However, further research is needed before general prognostic indices for elderly individuals can be recommended for routine use. Future research should focus on prospectively testing the validity of these indices across diverse clinical settings and analyzing their effect on clinical decision making and patient outcomes."

Advertisement

Source-Eurekalert