Paul Raeburn

I’m not going to try to match this: Paul Raeburn, chief media critic at the Knight Science Journalism Program at MIT, has evaluated recent news coverage of a purported Alzheimer’s test. He found virtually all the coverage wanting, awarding the title of “worst” to CNN.

In Raeburn’s opinion, most stories were merely “awful.”

The principal problems with the coverage included:

• inadequate or incomplete description of the “accuracy” of the lipid-based test:  failing to describe sensitivity, specificity and positive predictive value;
• over-optimistic prediction that the test will be available in two years; and
• general breathlessness, in some cases suggesting this is the first test alleged to predict Alzheimer’s disease and in others, suggesting that the problem of Alzheimer’s had been “cracked,” that the test is the “Holy Grail” of Alzheimer’s.

“We pause here to reflect on the use of superlatives,” Raeburn wrote. “If you use them too soon, you run out of things to say. By which I mean: If a test for middle-age adults is the holy grail, what would we call an Alzheimer’s cure? A really really holy grail?

Raeburn noted the “nicely understated lede” by Dr. Danielle Krol at ABC NEWS: “A blood test for early Alzheimer’s disease may be on the horizon, according to a new small study that links substances found in blood to mental decline three years later.”

“We pause here to reflect on the use of superlatives. If you use them too soon, you run out of things to say. By which I mean: If a test for middle-age adults is the holy grail, what would we call an Alzheimer’s cure? A really really holy grail?” – Paul Raeburn

Raeburn reserved his highest praise for the redoubtable John Gever, deputy managing editor of MedPageToday. His lede read: “Blood biomarkers in cognitively normal seniors were associated with their 3-year risk of developing mild cognitive impairment or Alzheimer’s disease, researchers said, although the accuracy fell short of what would normally be acceptable for a screening test.” (emphasis is Raeburn’s and mine)

Apart from the news that the “accuracy” of the test was far from optimal, Gever’s story also included the information that:

• none of the 10 plasma phospholipid molecules in the test are conventional markers for Alzheimer’s disease (although he did note that the use of markers unrelated to the APOE gene or beta-amyloid and tau proteins may represent a “welcome” new direction for predicting Alzheimer’s disease risk) and
• his calculation put the test’s positive predictive value at just 35%, calling into question the principal investigator’s statement in a news release that it represents “a major step toward the commercialization of a preclinical disease biomarker test that could be useful for large-scale screening.” (Gever also noted the paper in Nature Medicine was rather more understated.)

“We might not all learn to calculate positive predictive values, but we can learn to ask about them,” Raeburn commented.

“We might not all learn to calculate positive predictive values, but we can learn to ask about them.” – Paul Raeburn

Raeburn didn’t address Canadian coverage but would have found similar shortcomings as he did in most of the U.S. coverage. The Globe and Mail and CBC Radio’s The Current glossed over the findings and moved on to the ethical implications of finding out you’re at risk for a devastating condition for which there’s no treatment.

Dr. Sandra Black on Canada AM (not voted out of the Masterchef Kitchen)

National Post ran the Daily Telegraph’s story; I wasn’t able to find a story on the Toronto Star’s website.

CTV News Network consulted a researcher who didn’t seem to have read the study, but Canada AM (while the interviewer was over-earnest) had the wisdom to consult Dr. Sandra Black, director of the Brain Sciences Program of the Sunnybrook Research Institute.~TM