“Do doctors understand test results?”

by

The short answer in many cases is “no”:

In one session, almost half the group of 160 gynaecologists responded that the woman’s chance of having cancer was nine in 10. Only 21% said that the figure was one in 10 – which is the correct answer. That’s a worse result than if the doctors had been answering at random.
 
The fact that 90% of women with breast cancer get a positive result from a mammogram doesn’t mean that 90% of women with positive results have breast cancer. The high false alarm rate, combined with the disease’s prevalence of 1%, means that roughly nine out of 10 women with a worrying mammogram don’t actually have breast cancer.
 
It’s a maths puzzle many of us would struggle with. That’s because, Gigerenzer says, setting probabilities out as percentages, although standard practice, is confusing. He campaigns for risks to be expressed using numbers of people instead, and if possible diagrams.
 
Graphic showing “false positives” in mammogram tests
Even so, Gigerenzer says, it’s surprising how few specialists understand the risk a woman with a positive mammogram result is facing – and worrying too. “We can only imagine how much anxiety those innumerate doctors instil in women,” he says. Research suggests that months after a mammogram false alarm, up to a quarter of women are still affected by the process on a daily basis.
 
Survival rates are another source of confusion for doctors, not to mention journalists, politicians and patients. These are not, as you might assume, simply the opposite of mortality rates – the proportion of the general population who die from a disease. They describe the health outcomes of people who have been diagnosed with a disease, over a period of time – often five years from the point of diagnosis. They don’t tell us about whether patients die from the disease afterwards.

The linked article is worth reading despite its implicit pro-NHS boosterism. See also this. The poor education in statistical analysis of doctors, lawyers, journalists and members of other influential groups in our society is a significant problem.

(Via Mangan RT by heartiste on Twitter.)

UPDATE: Gerd Gigerenzer’s Books

7 thoughts on ““Do doctors understand test results?””

  1. I spent a year at Dartmouth learning, among other things, Bayesian algebra. One branch of this study is the plotting of Receiver Operating Characteristic curves which began as an analysis of signal strength but is also very useful in analyzing diagnostic testing.

    Your point about doctors’ understanding of test results is an excellent one. Mammography is a simple example. The problem is that this method will be used by bureaucrats to reduce access to care. One of the first examples is the PSA test for prostate cancer.

  2. I can certainly understand why a discussion of statistics and their uses would end up on the topic of Climate Science .

    Bjorn Lomborg’s calculation that if the climateers’ disaster scenarios are correct, then Germany’s investment of $100 billion in solar power schemes “can only reduce the onset of Global Warming by a matter of about 37 hours by the year 2100.”

  3. Gigerenzer’s book on risk is stunningly good. Every schoolboy should read it. And every doctor.

  5. “The fact that 90% of women with breast cancer get a positive result from a mammogram doesn’t mean that 90% of women with positive results have breast cancer.”

    Actually, mammography is not quite that good. My own experience was that the true positive rate was about 75%. The worst, most poorly differentiated and those with the least host resistance, often have negative mammograms. I think MRI has been better in recent years. I have seen palpable mammogram negative cancers and malignant cysts, usually a benign condition.

    I just did a quick and dirty search and found another interesting item.

    “The harm-benefit ratio for film mammography is more favorable than for digital mammography because film has a lower false-positive rate.”

    Digital mammography has increased false positive rate.

  6. @Jonathan, “Reckoning with Risk …” is the one I have in mind. Obviously I am a bit behind with my reading. Are the others as good?

Comments are closed.