Some Chicago Boyz know each other from student days at the University of Chicago. Others are Chicago boys in spirit. The blog name is also intended as a good-humored gesture of admiration for distinguished Chicago School economists and fellow travelers.
I originally published this essay on the 18th of January, 2014 at The Scholar’s Stage. David Foster’s recent post on “credentialed experts” has prompted me to resurrect it here. I have not otherwise changed it from the original.
—-
Last month Tom Nichols, professor of national security affairs at the U.S. Naval War College and a well regarded authority on Russian foreign policy and American nuclear strategy, published a thought-provoking essay on his blog titled “The Death of Expertise:”
…I wonder if we are witnessing the “death of expertise:” a Google-fueled, Wikipedia-based, blog-sodden collapse of any division between students and teachers, knowers and wonderers, or even between those of any achievement in an area and those with none at all.
By this, I do not mean the death of actual expertise, the knowledge of specific things that sets some people apart from others in various areas. There will always be doctors, lawyers, engineers, and other specialists in various fields.
Rather, what I fear has died is any acknowledgement of expertise as anything that should alter our thoughts or change the way we live. A fair number of Americans now seem to reject the notion that one person is more likely to be right about something, due to education, experience, or other attributes of achievement, than any other.
Indeed, to a certain segment of the American public, the idea that one person knows more than another person is an appalling thought, and perhaps even a not-too-subtle attempt to put down one’s fellow citizen. It’s certainly thought to be rude: to judge from social media and op-eds, the claim of expertise — and especially any claim that expertise should guide the outcome of a disagreement — is now considered by many people to be worse than a direct personal insult.
This is a very bad thing. Yes, it’s true that experts can make mistakes, as disasters from thalidomide to the Challenger explosion tragically remind us. But mostly, experts have a pretty good batting average compared to laymen: doctors, whatever their errors, seem to do better with most illnesses than faith healers or your Aunt Ginny and her special chicken gut poultice. To reject the notion of expertise, and to replace it with a sanctimonious insistence that every person has a right to his or her own opinion, is just plain silly. (emphasis added) [1]
I encourage visitors to the Stage to read Dr. Nichol’s entire piece. It was prompted by what has become a common experience every time he (or fellow UNWC professor and former NSA employee John Schindler) decides to publish a new essay or speak publicly about a pressing issue of the day. Soon after his work is published a flood of acrimonious tweets and e-mails follow, declaring that he does not really understand how American intelligence agencies, the Kremlin, or the Obama administration actually work.
Most of these responses are misinformed. Many are simply rude and mean. They are not an impressive example of what laymen commentators can add to America’s political discourse. Dr. Nichols suggests four rules of thumb for engaged citizens that he believes would improve matters:
1.The expert isn’t always right.
2. But an expert is far more likely to be right than you are.
3. Your political opinions have value in terms of what you want to see happen, how you view justice and right. Your political analysis as a layman has far less value, and probably isn’t — indeed, almost certainly isn’t — as good as you think it is.
4. On a question of factual interpretation or evaluation, the expert’s view is likely to be better-informed than yours. At that point, you’re best served by listening, not carping and arguing. [2]
The trouble with this advice is that there are plenty of perfectly rational reasons to distrust those with political expertise. Mr. Nichol’s wiser readers, for example, may have heard of the research conducted by Philip Tetlock, presented most compellingly in his 2005 book, Expert Political Judgement: How Good is it? How Can We Know?. Louis Menard describes the results of Dr. Tetlock’s research program in a book review for the New Yorker:
“Tetlock is a psychologist—he teaches at Berkeley—and his conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living “commenting or offering advice on political and economic trends,” and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. Would there be a nonviolent end to apartheid in South Africa? Would Gorbachev be ousted in a coup? Would the United States go to war in the Persian Gulf? Would Canada disintegrate? (Many experts believed that it would, on the ground that Quebec would succeed in seceding.) And so on. By the end of the study, in 2003, the experts had made 82,361 forecasts. Tetlock also asked questions designed to determine how they reached their judgments, how they reacted when their predictions proved to be wrong, how they evaluated new information that did not support their views, and how they assessed the probability that rival theories and predictions were accurate.“[3]
Tetlock’s experts came from all sorts of backgrounds: included were media personalities, tenured academics, professional analysts in Washington think tanks, and employees of numerous government agencies (including those with access to classified materials). A wide range of political beliefs and styles of analysis were also included: the study included both registered Republicans and Democrats, Austrian economists and their Keynesian counterparts, specialists in game theory, realist IR, area studies, and every other analytic model that gained popularity during the test period. What were the results of their 82,000 predictions?
“The results were unimpressive. On the first scale, the experts performed worse than they would have if they had simply assigned an equal probability to all three outcomes—if they had given each possible future a thirty-three-per-cent chance of occurring. Human beings who spend their lives studying the state of the world, in other words, are poorer forecasters than dart-throwing monkeys, who would have distributed their picks evenly over the three choices.
Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study. Knowing a little might make someone a more reliable forecaster, but Tetlock found that knowing a lot can actually make a person less reliable. “We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly,” he reports. “In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.” And the more famous the forecaster the more overblown the forecasts. “Experts in demand,” Tetlock says, “were more overconfident than their colleagues who eked out existences far from the limelight…. The expert also suffers from knowing too much: the more facts an expert has, the more information is available to be enlisted in support of his or her pet theories, and the more chains of causation he or she can find beguiling. This helps explain why specialists fail to outguess non-specialists. The odds tend to be with the obvious” (emphasis added). [4]
In other words, the expert is not more likely to be right than you are! Rigorous examination of actual expert track records show the opposite to be true: the more famous, confident, and specialized an expert is, the less accurate their political judgments tend to be. It turns out what an expert truly excels at is explaining away his or her own horrible record. Dr. Tetlock describes what happened at the end of the study when participants were asked to explain their errors:
“When we recontacted experts to gauge their reactions to confirmation or disconfirmation of their predictions, we frequently ran into a awkward problem. Our records of the probability judgements made at the beginning of the forcast periods often disagreed with experts recollections of what they predicted. Experts claimed that they assigned higher probabilities to outcomes that materialized than they did. From a narrowly Bayesian perspective, this 20/20 hindsight effect was a methodological nuisance: it is hard to ask someone why they got it wrong when the think it is right. But from a psychological perspective the hindsight effect is intriguing in its own right… , so we decided in six cases, to ask experts to recollect their positions prior to receiving the reminder from our records. When we asked experts to recall their original likelihood judgments, experts, especially hedgehogs, often claimed that they attached higher probabilities to what subsequently happened than they did…. Experts [also] shortchanged competition. When experts recalled the probabilities they once thought their most influential rivals would assign t the future that materialized, they imputed lower probabilities after the fact than before he fact. In effect, experts displayed both the classic hindsight effect (claiming more credit for predicting the future than they deserved) and the mirror image effect (giving less credit to their opponents for anticipating the future they deserved.)“ [5]
When confronted with these truths very few experts would admit that their reasoning or their methods were wrong. Instead they would create elaborate justifications for each failed prediction, claiming that their predictions were “almost right,” that events had truly been “close calls,” or that their prediction was only thrown off by some “out of the blue” or “fluke” occurrence no one could have seen coming. Each failed prediction was successfully turned into a compelling story that not only justified the expert’s failures but made things seem as if they and their methods had been right all along.
Mr. Tetlock’s research deserves to be better known than it is. In addition to Louis Menard’s review in the New Yorker excerpted above, interested readers are encouraged to read the CATO Unbound issue devoted to the book, view Tetlock’s hour long presentation for the Long Now Foundation, or purchase the book itself. It is difficult to delve into his work and think about experts the same way again.
Of course, Mr. Tetlock is not the only person to put our faith in expertise to question. The last decade has seen a general assault on the role of experts in a wide variety of fields: research by John Ioannis has shown that much of–if not most of–the medical findings published every year are false, and Daniele Fanelli has used similar methods to critique scientific research as a whole. [6] Doctors like Ben Goldcare and Adrian Fugh-Berman have shown how expert medical practitioners are often influenced by both common human biases and industry connections to prescribe treatments not in their patients best interests.[7] Popular public intellectual Nassim Nicholas Taleb made his claim to fame tearing down the models and methods of Wall Street’s top financial experts, maintaining that all narratives used by experts to explain the world are post-hoc constructions used to hide their own lack of knowledge. [8] James C Scott has shown how, despite any amount of study, training, or advance knowledge managers are given, the complexity of the systems expert bureaucrats are asked to manage are often beyond understanding, and attempts to merely comprehend (much less manage) these stewardship are doomed to distort and weaken them. [9]
This list is not comprehensive: it is limited to a few of the more stunning examples that have emerged over the last decade (and does not include all of those). But in many ways this is a very old critique. Men like Friedrich Hayek [10], Hermann Kahn [11], and Paul Meehl [12] made similar points (or in Meehl’s case, conducted similar studies) decades ago. Their skepticism has deep roots in Western culture; the Western tradition of free inquiry began with a man condemned to death for exposing how little his society’s elites actually knew. [13] Thoughtful people have been rejecting expert opinion for two millenia.
All of this leaves us with an important question: if experts are no better at making sense of the world than the rest of us, what are they good for?
While this post has expressed some hostility towards expert judgment, I am not an intellectual Jacobin intent upon getting rid of our experts altogether. The best evidence suggests that there is a valuable role experts can play. Indeed, some expertise does not need to be questioned at all. In their review of recent research on expert intuition, psychologists Daniel Kahneman and Gary Klein summarize the findings of James Shanteau‘s “cross-domain” study of expert performance:
“The importance of predictable environments and opportunities to learn them was apparent in an early review of professions in which expertise develops. Shanteau (1992) reviewed evidence showing that [real, measurable] expertise was found in livestock judges, astronomers, test pilots, soil judges, chess masters, physicists, mathematicians, accountants, grain inspectors, photo interpreters, and insurance analysts. In contrast, Shanteau noted poor performance by experienced professionals in another large set of occupations: stockbrokers, clinical psychologists, psychiatrists, college admissions officers, court judges, personnel selectors, and intelligence analysts.” [14]
Mr. Kahneman and Klein conclude that an expert’s judgement is most valuable in predictable environments whose patterns can be recognized, memorized, and then internalized. They call these “high validity” domains:
“Validity, as we use the term, describes the causal and statistical structure of the relevant environment. For example, it is very likely that there are early indications that a building is about to collapse in a fire or that an infant will soon show obvious symptoms of infection…. In contrast, outcomes are effectively unpredictable in zero-validity environments. To a good approximation, predictions of the future value of individual stocks and long-term forecasts of political events are made in a zero-validity environment….Validity and uncertainty are not incompatible. Some environments are both highly valid and substantially uncertain. Poker and warfare are examples. The best moves in such situations reliably increase the potential for success.” [15]
Alas, “zero-validity environments” are exactly the kind environments at the heart of this discussion with. Zero validity environments tend to fall into two categories:
1) Environments whose size or complexity make it impossible for experts to recognize the patterns or relationships they need to understand in order to make valid judgements about the system (such as faced by economists, ecologists, and financial analysts).
2) Environments where experts must evaluate behaviors, attitudes, past history, and other personal idiosyncrasies to try and explain why individuals act as they do or how they will act in the future (such as faced by psychiatrists, college admissions officers, and court judges).
“Political analysis,” which usually attempts to explain how statesmen, politicians, soldiers, or other political actors will respond to or instigate changes in world affairs, the electoral arena, battlefields, or some other political environment, falls into both categories! The expert political analyst is expected to create a reliable psychological portrait of key decision makers and understand the the numerous contingencies and complexities of the environment in which these decisions makers work. Unfortunately for the expert, this task lies beyond the limits of human cognition. No human, no matter how learned, experienced, or credentialed, can overcome them.
This seems pretty dismal, but as I suggested earlier, our experts need not lose hope. Their advanced degrees were not attained in vain. Too many have simply been proclaiming their expertise in the wrong subjects. Experts have no advantage over the average reader of the New York Times when it comes to political judgements. They are no better–and sometimes worse–at predicting political outcomes, weighing the relative importance of events, or identifying cause and effect relationships than the rest of us. But analyst experts do have a significant advantage over the layman analyst in one domain: they know so much more. Experts can contribute a great deal to the conversation simply by telling us what they know.
An easy way to distinguish the difference between these two types of expertise is to think in terms of what and why. Experts are at their best when they explain the “what” of the systems they have spent so much time studying. This kind of expertise is about knowledge, not judgment or analysis; its explanations are not prescriptive or predictive, but descriptive. They focus on confirming disputed facts and explaining the significance of details that layman observers do not have the knowledge to properly understand or recognize. Their job is to find the truth and make it known.
This task is far more important than it initially seems. Consider the ‘red-line’ kerfuffle that made headlines back in September. Several weeks before sarin filled munitions were launched into a suburb outside of Damascus called Ghouta. The President of the United States had previously declared that if the Assad Regime used chemical weapons then the United States would intervene in the conflict. The world was left with a question: were these weapons launched by the Syrian Army or one of the groups opposing it?
In those days opinions on the subject flew off the presses. Most of this was political analysis par excellence–lots of talk about the various groups’ “motives” or “credibility,” and the “political logic” of the situation. But as the literature reviewed above suggests, once a certain level of quality was reached, none of these analyses were more reliable than the others. We had ran straight into the limits of human political judgment.
Fortunately, there was a more promising route to answering the question at hand. At the time I was closely following Eliot Higgins’s Brown Moses Blog. Mr. Higgins, now recognized as an expert in the munitions used in the Syrian war [16], put together all of the videos and pictures he could get that were in some way tied to the scene of the attacks and compared them to other munitions used in the conflict. His careful reconstruction both identified the exact munitions used in the attack and linked them to artillery and munitions used by Assad’s regime.
Mr. Higgins here accomplished something only an expert could. The number of Americans who have the background knowledge necessary to identify and distinguish the dozens of munitions used by Syrian militias is limited to a few dozen experts. These men have a true “expert’s advantage” that comes from gaining specialized knowledge in the what of an issue.
This advantage is not limited to the world of material objects. A recent post at Andrew Chubb‘s website is an excellent example of how expertise can be properly applied to human subjects. The question at hand: to what extant do the views articulated by vocal PLA ‘hawks’ like Luo Yuan, Dai Xu, and Zhang Zhaozhang represent the upper levels of the People’s Liberation Army as a whole? (I encourage those interesting in following this debate in full to read Mr. Chubb’s original essays for the Jamestown Foundation’s China Brief, a more recent review essay by Yawei Liu and Justine Zheng Ren for for the Journal of Contemporary China, and Chubb’s response to this essay at South Sea Conversations). This is essentially a debate about a fairly narrow “what”: what are the past and current institutional roles of the hawks? Are they in a position to know and articulate PLA strategic doctrine? [17]
As with the munitions in Syria, analysts who approach this question need a thorough command of things not commonly known: the structure of the PLA, the biographies of the officers in question, PLA-media relations procedure, and the ability to recognize those damn Chinese characters used in all of those press releases and interviews. Only those with a high level of expertise can even participate in this debate. Those of us who lack this knowledge are well advised to take Dr. Nichols’s advice to stop our “carping” and listen to what these folks have to say.
One suspects that the unpolished masses who flood Dr. Nichol’s and Schindler’s inboxes with insults and assertions do not do so because they have been reading Philip Tetlock’s most recent research. While we have reviewed plenty of compelling evidence that experts and expert claims should be viewed with skepticism, the great majority of this research is unknown among even the more educated sections of the body politic. What accounts for this widespread hostility towards expertise?
Dr. Nichols suggests the following:
Some of it is purely due to the globalization of communication. There are no longer any gatekeepers: the journals and op-ed pages that were once strictly edited have been drowned under the weight of self-publishable blogs (like, say, this one). There was once a time when participation in public debate required submission of a letter or an article, and that submission had to be written intelligently, pass editorial review, and stand with the author’s name and credentials attached. Even then, it was a big deal to get a letter in a major newspaper.
Now, anyone can crowd the comments section of any major publication with inane blather. We live in a huge high school boys’ room, where anyone with a marker can write anything on the wall. Sometimes, that kind of free-for-all spurs good thinking. Most of the time, it just means people can post anything they want, under any anonymous cover, and never have to defend their views or get called out for being wrong.
Another reason for the collapse of expertise lies not with the global commons but with the increasingly partisan nature of U.S. political campaigns. There was once a time when presidents would win elections and then scour universities and think-tanks for a brain trust; that’s how Henry Kissinger, Samuel Huntington, Zbigniew Brzezinski and others ended up in government service while moving between places like Harvard and Columbia.
…I also would argue that colleges have to own some of this mess. The idea of telling students that professors run the show and know better than they do strikes many students as something like uppity lip from the help, and so many profs don’t do it. Many colleges are boutiques, in which the professors are expected to be something like intellectual valets. This produces nothing but a delusion of intellectual adequacy in children who should be instructed, not catered to.
…Finally, the root of this collapse of standards lies in our manic reinterpretation of “democracy,” in which everyone must have their say, and it must not be, er, disrespected. (The verb to disrespect is one of the most obnoxious and insidious innovations in our language in years, because it really means “to fail to pay me the impossibly high requirement of respect I demand.”) [18]
All of these things may have an influence on current attitudes towards experts and educated expertise. [19] But I think Nichols has missed an important ingredient–perhaps the most important ingredient–of the hostility he and his compatriots receive.
To get a sense for what is missing from his explanation, please consider the following:
 |
||
| Reported Trust in Government since 1950s. Figure 2 from Doug Sosnik, “What Side of the Barricade are You On?“, Politico, 25 November 2013. |
 |
| The American Public’s Confidence in Assorted Public Institutions Figure 3 from Doug Sosnik, “What Side of the Barricade are You On?“, Politico, 25 November 2013. |
These two images were originally presented in an astute essay that political strategist Doug Sosnik wrote for Politico. These are just two of the many data points Sosnik uses to capture the sense of alienation and outrage the American public feels towards their leaders. The picture is pretty clear: Americans do not just doubt their experts. They doubt every institution American society has invested with authority! This crises in public confidence is a storm no authority figure can run away from, experts included.
It is not difficult to understand how experts are caught up in this great swell of hostility towards American elites. “Our Republic”, the narrative goes, “wasted a decade fighting wars our experts claimed would be done and over with in a few years. We still haven’t picked ourselves up from a recession these experts did not see coming. Why should we trust another expert again?”
It is unfair to blame all of America’s ills (or even the specific ones mentioned above) on our hapless experts. But the truthfulness of the narrative is irrelevant. What matters is how people who live far away from the ivory towers and government offices see the experts who dwell there. And in in this kind of political climate the experts who craft political analysis defending the powers that be should not be surprised when they are met with unrestrained hostility and extreme skepticism. It is the cost of claiming expertise.
It is not just the politicians that will be forced through the grinder. Everybody who talks politics or claims a position of special authority is going to be, or is already, facing this scrutiny. People are wondering what side of the barricade America’s credentialed class of experts belong.
[1] Tom Nichols. “The Death of Expertise.” The War Room (11 December 2013).
[2] Ibid.
[3] Louis Menard. “Everybody’s an Expert: Putting Political Expertise to the Test.” The New Yorker (5 December 2013).
[4] Ibid.
[5] Philip Tetlock. Expert Political Judgement: How Good is it? How Can We Know? (Princeton: Princeton University Press, 2005). pp. 137-138
[6] The key articles are John Iodannis, “Why Most Published Medical Research is False,” PLOS Medicine (30 August 2005); John Iodannis, et. al, “Persistence of Contradicted Claims in the Literature,” Journal of American Medical Association, Vol 298, No 21 (5 December 2007); Danielle Fanelli, “How Many Scientists Falsify and Fabricate Research? A Systematic Review and Meta Analysis of Survey Data,” PLOS One (29 May 2009); “‘Positive’ Results Increase Down the Hierarchy of Science,” PLOS One (7 April 2010).
I strongly recommend those unfamiliar with this line of research read David Freeman’s review essay for the Atlantic, “Lies, Damned Lies, and Medical Science” (4 October 2010).
[7] Ben Goldcare, “What Doctors Do Not Know About the Drugs They Proscribe,” presentation given at TED Med 2012 (June 2012, pub. Sep 2012);
[8] Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Imporbable. Rev. ed. (New York: Penguin Books, 2010). pp. 145-151.
[9] While James C. Scott’s books and articles are legion, I believe that his essay, “The Problem With the View From Above,” CATO Unbound (8 September 2010), is the best introduction to his work on this topic.
[10] Friedrich Hayek, “The Uses of Knowledge in Society,” American Economic Review. Vol. XXXV, No. 4(Sep 1945). pp. 519-30. American Economic Association.
[11] Herman Kahn, “The Expert and Educated Incapacity” in World Economic Development: 1979 and Beyond, (Boulder, CO: Westview Press, 1979). pp. 482-484
[12] William M. Grove, “Clinical vs. Statistical Prediction: The Contributions of Paul E. Meehl,” Journal of Clinical Psychology, Vol. 61, No. 10 (2005), pp. 1233124.
[13] Plato, Apologia, 21.c-23.b
[14] Daniel Kahneman and Gary Klein, “Conditions for Intuitive Expertise: A Failure To Disagree,” American Psychologist, vol 64, No. 6 (September 2009), p. 522.
[15] Ibid., p. 520, 524.
[16] I should note Mr. Higgins is a new entrant to the world of the expert, having no credentials beyond the incredible work he does. By some definitions he does not count. But most folks in the field seem to think he does. Both the New Yorker and the Huffington Post have published essays on this man and his remarkable methods that may profit those who have never heard of him:
Bianca Bosker, “Inside the One Man Intelligence Unit That Exposed the Secrets and Atrocities of the Syria War,” The Huffington Post (18 November 2013); Patrick Raden keefe, “Rocket Man: How An Unemployed Blogger Confirmed Syria Used Chemical Weapons,” The New Yorker (25 November 2013).
[17] The full citations for the articles discussed are: Andrew Chubb, “Propaganda as Policy? Explaining the PLA’s “Hawkish” Faction, pt 1” The China Brief: A Journal of Analysis and Information, Vol XIII, Issue 15 (26 July 2013), pp. 6-11; “Propaganda as Policy? Explaining the PLA’s “Hawkish” Faction, pt 2” The China Brief: A Journal of Analysis and Information, Vol XIII, Issue 16 (9 August 2013), pp. 12-16; Yawei Liu and Justine Zhang Ren, “An Emerging Consensus on the U.S. Threat: The United States According to PLA Officers,” Journal of Contemporary China (19 November 2013); Andrew Chubb, “Are China’s hawks actually the PLA after all? [Revised],” South Sea Conversations (or. pub 5 December 2013, rev. 17 December 2013).
[18] Tom Nichols, “The Death of Expertise.”
[19] The only one of his proposed explanations I outright disagree with is the ‘manic reinterpretation of democracy’ he speaks of. I find this view of the matter quite peculiar; familiarity with American political history and American political discourse suggests that this is not a reinterpretation of democracy’s meaning but a restoration of what the word meant for majority of this Republic’s history. The move towards expert empowerment (and the deference to expertise that went along with it) is a relic of the Efficiency movement and other reformer ‘movements’ of the Progressive Era. Tocqueville captured the mood of the nation before those movements:
“As for the effect which one man’s intellect can have upon another’s, it is of necessity much curtailed in a country where its citizens, having become almost like each other, scrutinize each other carefully and, perceiving in not a single person in their midst any signs of undeniable greatness or superiority, constantly return to their own rationality as to the most obvious and immediate source of truth. So, it is not merely trust in any particular individual which is destroyed, but also the predilection to take the word of any man at all.
Each man thus retreats into himself where he claims to judge the world”
-Alexis de Tocqueville, Democracy in America and Two Essays on America, trans Gerald Bevan (New York: penguin Books, 2003), p.494. See also pp. 493-503 generally.
This does not sound unlike the very attitudes that prompted Dr. Nichols to write his essay.
The men of 1834 and 2014, of course, are not quite the same. Every man should be an expert of his own community. The average antebellum man had a great deal of control over the political realities most important to his well being. The average American of the 21st century cannot claim the same.
[20] Doug Sosnik, “What Side of the Barricade are You On?“, Politico (25 November 2013).
Comments are closed.
“If done properly, a PhD certifies that you are capable of conducting research to particular standards in your field, that you have contributed new knowledge to your field, and that you have an independent ability to frame questions and conduct serious, long-term analytical projects to answer them.” The qualification at the beginning of that sentence is vital. After all, the “Climate Scientists” have PhDs but much of what they have to say is rubbish. It happens that I have expertise enough to have formed that judgement years ago; I have, however, been impressed since by how many people, initially without any expertise, have managed to come up with telling criticisms of that religion.
In other words, to disparage all expertise is just plain silly, but it’s also worth recognising how many fake claims to expertise there are around.
I prefer the quotation from Nevil Shute’s novel ‘Trustee from the Toolroom’ which went:-
‘An Engineer is a man who can do for fifty cents what any bloody fool can do for five dollars’ (currency terms translated from the original)
Your link to “John Ioannis” links to someone called Ioannis “John” Papapolymerou, who probably is not who you meant.
I assume you really meant John Ioannidis and his study here: http://dx.doi.org/10.1371/journal.pmed.0020124
dearieme
In other words, to disparage all expertise is just plain silly, but it’s also worth recognising how many fake claims to expertise there are around.
Many people who have Ph.Ds. assume that their doctorates give them an expertise in subjects far removed from the topics of their doctoral dissertations. The consequence of these faux experts claiming expertise in fields in which they know relatively little is to further undermine trust in “experts.”
The Democrats have for decades proclaimed that they have the expertise by which Government of the Democrats, by the Democrats and for the Democrats will bring Heaven on Earth. An illustration of this point of view is seen in a Christmas letter from 2008:
The snafus embodied in Fast and Furious, the Stimulus Package, Obamacare, and foreign policy have done nothing but increase mistrust in “experts,” a.k.a. “cadre of the best,” who time and again have rolled out policies that have not performed as promised.
Obama’s staffing the National Security Council with 20-something spinmeisters is both a cause and a symptom of the loss of trust in “experts.”
The mark of a real expert:
An interesting post. The comment box seems to be having issues but, here goes. The issue of experts in medicine is a good one. I spent a year at Dartmouth in 1994-95 learning some technology that I hoped to put to use in measuring healthcare quality. I had retired after 25 years of practice followed by a 14 hour back surgery. Dartmouth and Jack Wennberg who ran the program then, have studied the role of variation as a measure of quality and effectiveness in medicine. They have many publications on this topic and anyone interested can quickly find many of them. They also did an Atlas of Healthcare based on Medicare spending. One finding is that, when a condition has one obvious treatment, like a broken hip for example, there is little variation in how it is treated. Other conditions, like pneumonia or congestive heart failure, where there is little consensus, find considerable variation in treatment.
One method trying to achieve consensus is the use of Guidelines. There are two ways to establish guidelines. One is the randomized clinical trial. Here, a treatment, usually a drug, is tested against a placebo or another drug. The patient, and usually the doctor, does not know which drug the patent got and results are compared. Sometimes the results are a great surprise, like this cute story and surgery, for obvious reasons, is usually not studied since it is pretty obvious who got the surgery. There have been a few examples but not many.
Most doctors are wary of what are called “consensus guidelines,” which are created by a group of experts thinking about their opinions and then handing “down” the guideline. Here the tyranny of experts reigns. These are often, in fact usually, influenced by economic factors and the cheapest alternative always seems mysteriously to be chosen. Obamacare comes to mind.
I learned, after spending some years at it, that quality was very low on the list of priorities of health care organizations and eventually went back to teaching part time. One example of my interests at the time was this proposal which was cancelled by the UCI hospital administrator. He was not interested in improving care for anybody.
There’s a problem with the comments formatting for this post, probably my fault. Comments display correctly but fill the width of the page. You will need to scroll down a bit to find the comment entry box, but comment functionality is otherwise unchanged.
Related: And of course, one of the great and most horrible of social crimes in this benighted age is to be “judgmental”. Gasp Oh, the shock, the horror, of one who would be “judgmental”!
But as one of the few remaining wise souls said, I know not who: “To not judge is to not think”.
I dunno, we seeing any “not thinking” out there of late?
From the commented-on article: “But mostly, experts have a pretty good batting average compared to laymen: doctors, whatever their errors, seem to do better with most illnesses than faith healers or your Aunt Ginny and her special chicken gut poultice.”
That’s a dishonest framing of the issue. Even in the actual medical situation that he mentions immediately afterwards (vaccines) it is misleading to think of it as doctors vs. outsiders, because the divide runs more nearly through the gray shading of expertise into non-expertise than through the sharp legal line of having the privilege of legally performing medical services. And the usual situation is not particularly analogous to modern doctors vs. outsiders.
Two reasons that usual expert situation is not particularly analogous to medicine are (1) that doctors generally delight in demonstrating reliable performance at doing demanding things to reach a desirable end result, like sewing up my shredded lower lip a few years ago, or transplanting various people’s kidneys (in which they are similar to other respected experts like computer programmers, pilots, auto repairmen, and various kinds of engineers) and (2) doctors’ status is propped up by sharp draconian legal barriers to entry so e.g. if you want various centrally important medical services (antibiotic, strong painkiller, X-ray…) you must go through a doctor and the number of doctors is sharply restricted (in which they are less similar to those other expert specialties).
And in the controversies that seem to be driving his personal lament, he has two particular problems. One problem is that he wants to dodge demonstrating reliable performance, and appeal to expertise in plausibly-related things. Another problem is that he glosses over ridiculous levels of politicization.
Demonstrating reliable expertise would be, e.g., bragging about how the excellent record of foreign policy experts being reliably correct. Instead he brags about e.g. how he spent time in Russia, acquiring special knowledge. That is the strategy of doctors centuries ago, when they couldn’t reliably demonstrate superior outcomes. Instead they appealed to e.g. how they had learned more than their rival the barber about about the writings of the ancient Greeks and Arabs, and/or about anatomy. And lo, when today’s foreign policy experts base their prestige on a superior-training-not-superior-outcomes foundation similar to that used by doctors centuries ago, by some bizarre coincidence they achieve status similar to the status of doctors centuries ago. It reflects badly on this political scientist’s savvy about the workings of humans and societies, and about the validity of expertise that he is publicly ruminating on, that the analogous distinction and the analogous outcome elude his notice.
(This comment is already long, so I will just leave the politicization as an exercise for the reader.)
He also writes “as I say so often, there’s a reason that articles and books are subjected to ‘peer review’ and not to ‘everyone review.'” Bah. There are good and important reasons that science advanced just fine before peer review became common; there are also important reasons that people have tried to elevate peer review to sacred status, but those reasons are generally stupid and/or outright evil.
The most severe barrier to entry in medicine is the need for teaching hospitals and clinical experience. The US license thousands of foreign doctors who have no clinical experience at all. Mexican medical schools, for example, have none. I teach the clinical introduction of medicine to second year students and we have far less clinical experience for the class that is three times the size of my class 50 years ago. When I was a student, the LA County hospital had 3500 beds mostly full and my class was 66. Last year;s class at USC was 168 and the County hospital has 600 beds. Much of this is the trend to outpatient medicine but class size has been increased tremendously compared to 50 years ago. In 1962, voters in California decided to end the osteopathic board and give all osteopaths the option of becoming MDs on the theory that they would get better training. Osteopathic medical schools came back 25 years ago and there are quite a few. They have small clinical facilities. Most of the talk about “barriers” is about a myth.
>>the more facts an expert has, the more information is available to be enlisted in support of his or her pet theories, and the more chains of causation he or she can find beguiling.
I find that convincing. Many processes respond disproportionately to one or two factors.
There is no expertise in government, politics, or morality. The use of the words “bioethicist” makes my skin crawl, because that word is sure to be followed by some outrage about nudging useless mouths to death.
An expert in ethics knows about ethics, but is not himself ethical.
I think there are a few different things going on here:
1) In many quarters there is a virtual worship of “science.” The quotation marks are because what is being worshipped is not the scientific method, but rather the conclusions/assertions of certain credentialed scientists. It is basically an Argument from Authority, little different from the medieval practice of looking to Aristotle as the source of scientific truths. (A giveaway to this attitude is the phrase “science says.”)
Much of this is due to people who do not themselves have any scientific education at all, even 8th-grade chemistry, but *have* spent years in academic environments where agreeing with the authorities is strongly incentivized.
2) Certain fields of study have “borrowed” credibility from other fields which have more right to such credibility. In the 1960s, for example, there had been huge and fairly recent advanced in nuclear energy, missile & space technology, electronics, antibiotics, etc etc. Social scientists asserted that they could provide the same kind of progress in the understanding & improving of society.
3) Celebrity culture is very strong in the US today, and assertions of very-well-known people, even about fields of which they have no knowledge or experience whatsoever, are taken seriously by millions of people.
4) People who feel a strong religious impulse, but have rejected organized religion (“I’m spiritual but not religious” is the tag line) are often attracted to pseudoscientific beliefs such as astrology, magical crystals, etc.
5) Credential-worship has led to excessive reverence for conceptual tools (such as those learned in the typical MBA program) at the expense of experience; also for quantitative information at the expense of hands-on and necessarily-subjective information. (A “Big Data” analysis of your customer, versus actually spending some time in the stores and seeing with your own eyes what is going on)
I have several posts relevant to this topic:
Drucker on Management Mentalities
Myths of the Knowledge Society
Management Education and the Role of Technique
The B-School Debate
An anecdote about Kellstadt. In 1958, I was working part-time for Sears in their east LA store, called the “Boyle Street store.” One day, Charles Kellstadt, the Sears Chairman, came through the store on a visit. As he walked through the mens’ department, he saw a gaudy tie with dyed feathers on it. He said, “That tie is awful. Get rid of it.” and walked on. Nobody had the heart to tell him that the tie was our best seller in that Hispanic neighborhood store.
MikeK….reminds me of Koestler’s comment that if you asked a centipede to think about what sequence he moved his legs in, he probably wouldn’t be able to walk at all. His point was that it can be dangerous for a higher-level function to interfere excessively in the operations of a lower-level function.
From the B school piece: “Why did that revolution begin with the Spinning Jenny in the mid-18th centruy and not in the mid-first century AD when Hero of Alexandria developed a steam engine?”
Joel Mokyr has written several books about that very question and his conclusion, which I accept, is that there was no legal framework to allow inventors to profit from their inventions until 17th century England. The Industrial Revolution began in the nation that had the best legal system to protect the private economy. After the revocation of the Edict of Nantes in 1685 by Louis XIV , the Industrial Revolution of France moved to England with the Huguenots as they fled the persecution of Protestants.
This may explain why there is no fracking in Europe where mineral rights belong to the state, not the property owner. Some European countries are reassessing their decision as Russia becomes more aggressive but, even then, the action will be sluggish and inefficient.
I doubt management as we know it was a fundamental cause of prosperity.
Well, people had to learn how to operate complex and often geographically-distributed organizations, the railroads being a case in point. The much-earlier development of accounting also played an important part by reducing opportunities for fraud and providing something of a common language for evaluating business results.
I think the concept of “federal decentralization” (as Drucker calls it)…ie, multiple business units running to a considerable degree as separate businesses within an overall corporation…has been of considerable value…imagine GE, for example, if there was a purely functional organization, ie one common engineering organization for all products ranging from dishwasher to locomotives to jet engines, one common sales organization for all of same, etc…failure pretty much guaranteed.
And the development of Lean management principles, as at Toyota, has been of considerable value, although lots of not-so-good approaches also advertise themselves under that umbrella.
Another good example of a large corporation with multiple divisions running almost independently is 3M and that has created a lot of innovation.
What side of the barricades will the experts be on?
The Strongest which is unfortunate, since most of them merit extinction.
The rule of experts has placed civilization at peril of a another Dark Age. Our money and debt for instance…
Now what should one make of this? “But the truthfulness of the narrative is irrelevant. What matters is how people who live far away from the ivory towers and government offices see the experts who dwell there.”
No actually the Truth of the Matter is at last becoming important again, comprehensive bankruptcy and looming war have that effect.
Hummm…
My latest History Friday column actually hits on this melt down of credentials as well.