Statistics

“Do doctors understand test results?”

by

The short answer in many cases is “no”:

In one session, almost half the group of 160 gynaecologists responded that the woman’s chance of having cancer was nine in 10. Only 21% said that the figure was one in 10 – which is the correct answer. That’s a worse result than if the doctors had been answering at random.
 
The fact that 90% of women with breast cancer get a positive result from a mammogram doesn’t mean that 90% of women with positive results have breast cancer. The high false alarm rate, combined with the disease’s prevalence of 1%, means that roughly nine out of 10 women with a worrying mammogram don’t actually have breast cancer.
 
It’s a maths puzzle many of us would struggle with. That’s because, Gigerenzer says, setting probabilities out as percentages, although standard practice, is confusing. He campaigns for risks to be expressed using numbers of people instead, and if possible diagrams.
 
Graphic showing “false positives” in mammogram tests
Even so, Gigerenzer says, it’s surprising how few specialists understand the risk a woman with a positive mammogram result is facing – and worrying too. “We can only imagine how much anxiety those innumerate doctors instil in women,” he says. Research suggests that months after a mammogram false alarm, up to a quarter of women are still affected by the process on a daily basis.
 
Survival rates are another source of confusion for doctors, not to mention journalists, politicians and patients. These are not, as you might assume, simply the opposite of mortality rates – the proportion of the general population who die from a disease. They describe the health outcomes of people who have been diagnosed with a disease, over a period of time – often five years from the point of diagnosis. They don’t tell us about whether patients die from the disease afterwards.

The linked article is worth reading despite its implicit pro-NHS boosterism. See also this. The poor education in statistical analysis of doctors, lawyers, journalists and members of other influential groups in our society is a significant problem.

(Via Mangan RT by heartiste on Twitter.)

UPDATE: Gerd Gigerenzer’s Books

The Limits of Expertise

by

I originally published this essay on the 18th of January, 2014 at The Scholar’s Stage. David Foster’s recent post on “credentialed experts” has prompted me to resurrect it here. I have not otherwise changed it from the original.

—-

Last month Tom Nichols, professor of national security affairs at the U.S. Naval War College  and a well regarded authority on Russian foreign policy and American nuclear strategy, published a thought-provoking essay on his blog titled “The Death of Expertise:”

…I wonder if we are witnessing the “death of expertise:” a Google-fueled, Wikipedia-based, blog-sodden collapse of any division between students and teachers, knowers and wonderers, or even between those of any achievement in an area and those with none at all.

By this, I do not mean the death of actual expertise, the knowledge of specific things that sets some people apart from others in various areas. There will always be doctors, lawyers, engineers, and other specialists in various fields.

Rather, what I fear has died is any acknowledgement of expertise as anything that should alter our thoughts or change the way we live. A fair number of Americans now seem to reject the notion that one person is more likely to be right about something, due to education, experience, or other attributes of achievement, than any other.

Indeed, to a certain segment of the American public, the idea that one person knows more than another person is an appalling thought, and perhaps even a not-too-subtle attempt to put down one’s fellow citizen. It’s certainly thought to be rude: to judge from social media and op-eds, the claim of expertise — and especially any claim that expertise should guide the outcome of a disagreement — is now considered by many people to be worse than a direct personal insult.

This is a very bad thing. Yes, it’s true that experts can make mistakes, as disasters from thalidomide to the Challenger explosion tragically remind us. But mostly, experts have a pretty good batting average compared to laymen: doctors, whatever their errors, seem to do better with most illnesses than faith healers or your Aunt Ginny and her special chicken gut poultice. To reject the notion of expertise, and to replace it with a sanctimonious insistence that every person has a right to his or her own opinion, is just plain silly. (emphasis added) [1]

I encourage visitors to the Stage to read Dr. Nichol’s entire piece. It was prompted by what has become a common experience every time he (or fellow UNWC professor and former NSA employee John Schindler) decides to publish a new essay or speak publicly about a pressing issue of the day. Soon after his work is published a flood of acrimonious tweets and e-mails follow, declaring that he does not really understand how American intelligence agencies, the Kremlin, or the Obama administration actually work. 

Most of these responses are misinformed. Many are simply rude and mean. They are not an impressive example of what laymen commentators can add to America’s political discourse. Dr. Nichols suggests four rules of thumb for engaged citizens that he believes would improve matters:

1.The expert isn’t always right.

2. But an expert is far more likely to be right than you are.

3. Your political opinions have value in terms of what you want to see happen, how you view justice and right. Your political analysis as a layman has far less value, and probably isn’t — indeed, almost certainly isn’t — as good as you think it is.

4. On a question of factual interpretation or evaluation, the expert’s view is likely to be better-informed than yours. At that point, you’re best served by listening, not carping and arguing. [2]

 ·


The trouble with this advice is that there are plenty of perfectly rational reasons to distrust those with political expertise.

Read more

Afghanistan and Pakistan.

by

I have been unhappy about our role in Afghanistan for several years. This goes back to at least 2009. Then there was this.

Watching the last two weeks or so in the White House, gives me the sense that the decision is going to be the wrong one. There are three possible choices that Obama has; one is to take his hand-picked general’s advice and send 40,000 more troops. It will stress our military and the logistical challenges are serious. Afghanistan is land-locked and the neighbors are not friendly. Russia will try to create problems, as they already have in Kyrgyzstan. They do not want us to succeed yet they may fear total failure. In the meantime, they are making serious trouble.

And then, this development.

it’s an open secret the Taliban are headquartered across the border in the city of Quetta, Pakistan, where they operate openly under the aegis of Pakistani intelligence — and the financial sponsorship of the Saudis.

Sending more troops to Afghanistan is a necessary, albeit unfortunate, rear-guard action against marauding Taliban fighters armed, trained, supplied and deployed from Quetta — and funded from Riyadh.

NATO and U.S. military command know this. They’ve complained about it over and over in military action reports. So have Treasury officials regarding Saudi funding of the Taliban.

“Saudi Arabia today remains the location where more money is going to terrorism — to Sunni terror groups and the Taliban — than any other place in the world,” testified Stuart Levey, Treasury undersecretary.

This is Viet Nam all over again. The enemy has a sanctuary and our allies are siding secretly with our enemies.

Well, today, there is another bit of information

Read more

The ghost of database past

by

Section 2, Amendment XIV:

Section 2. Representatives shall be apportioned among the several States according to their respective numbers, counting the whole number of persons in each State, excluding Indians not taxed. But when the right to vote at any election for the choice of electors for President and Vice President of the United States, Representatives in Congress, the Executive and Judicial officers of a State, or the members of the Legislature thereof, is denied to any of the male inhabitants of such State, being twenty-one years of age, and citizens of the United States, or in any way abridged, except for participation in rebellion, or other crime, the basis of representation therein shall be reduced in the proportion which the number of such male citizens shall bear to the whole number of male citizens twenty-one years of age in such State.

Article I, Section 2, U.S. Constitution:

Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers, which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.

For working on the paternal genealogy of Howard Ira Milligan, my mother’s father, U.S. Census records have proved to be an important primary source.

Read more

“Studies Show” – Widespread Errors in Medical Research

by

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice?

The arguments presented in this article seem like a good if somewhat long presentation of the general problem, and could be applied in many fields besides medicine. (Note that the comments on the article rapidly become an argument about global warming.) The same problems are also seen in the work of bloggers, journalists and “experts” who specialize in popular health, finance, relationship and other topics and have created entire advice industries out of appeals to the authority of often poorly designed studies. The world would be a better place if students of medicine, law and journalism were forced to study basic statistics and experimental design. Anecdote is not necessarily invalid; study results are not necessarily correct and are often wrong or misleading.

None of this is news, and good researchers understand the problems. However, not all researchers are competent, a few are dishonest and the research funding system and academic careerism unintentionally create incentives that make the problem worse.

(Thanks to Madhu Dahiya for her thoughtful comments.)