A Lie in a Lab Coat

105 thoughts on “A Lie in a Lab Coat”

1. Falluja was excluded because it was an outlier. This is perfectly standard practice. Even excluding Falluja, the majority of the excess deaths were from violence — you screwed up your calculations.

2. Stay on these guys, baby.

3. You think that `corruption’ is evident because in the totals they omitted a cluster that would have shown much higher death rates?

4. Tim, What role might researcher bias be playing in the Lancet study?

   -Steve

5. Tim Lambert,

   “Falluja was excluded because it was an outlier. This is perfectly standard practice”

   But the exclusion should have been total. It wasn’t, The summary in particular flips back in forth from including the Falluja data to excluding it and then back again. If the Falluja cluster is in fact an outlier then it has no place in study period at all.

   Cluster sampling was a poor design choice expressly because it was likely to produce just such an outlier. They should have created criteria for excluding a cluster a priori but I see no evidence they did so. Deciding to selectively exclude evidence after it is measured because it doesn’t fit is indicative of bad design.

   “Even excluding Falluja, the majority of the excess deaths were from violence — you screwed up your calculations.”

   Including Falluja, the study measured a delta of +96 deaths (146 deaths post invasion – 46 deaths pre-invasion) Of those deaths, 73 (52 in Falluja + 21 elsewhere) or 76% were the result of violence.

   Excluding Falluja, the study produced a delta of +44 (146 deaths post invasion – 52 deaths in Falluja – 46 deaths pre-invasion). Of those deaths 21 or 48% resulted from violence which means that 52% did not. Last time I looked 48% did not define “most.”

   Based on the data in table 2, most of the excess deaths resulted from violence only when the Falluja cluster is counted.

6. “You think that `corruption’ is evident because in the totals they omitted a cluster that would have shown much higher death rates?”

   No I think that corruption is evident because they selective include or exclude the cluster based solely on whether doing so makes the Coalition look worse.

   They exclude the cluster when its extreme numbers would obviously invalidate the study but they include it in order to make the majority of excess deaths result from violence or to skew the age and sex ratios towards women and children.

   The cluster should be in or out. The authors shouldn’t literally switch from including and excluding the Falluja from one sentence to the next.

   Scientist shouldn’t pick and choose their data. That they did so is dishonest.

7. You need to adjust for the fact that the time periods before and after the war were unequal.

8. Shannon,
   Steel yourself.

   Your typos will be used to anti-intellectually attack your incredibly wise responses.
   -Steve

9. “You need to adjust for the fact that the time periods before and after the war were unequal.”

   You maybe right now that I look at it. Increasing the number of deaths pre-invasion from non-violence means will skew the percentage of excess deaths due to violence post-invasion upward. I will rerun my numbers tomorrow and correct the post if necessary.

   But

   That still does not explain the misstating of the gender and age ratio based on the Falluja cluster, nor does it explain the statement that “after the invasion violence was the primary cause of death.” When the study clearly shows that, absent the Falluja cluster, the major cause of death is still illness and accident.

   This is supposed to be scientific paper in a major publication. It is supposed to be clear and concise not riven with spin.

10. God this is weak. I would advise anyone following Shannon’s links to “Cluster Sampling, explained” to read the comments thread in detail, where it is explained that Shannon’s past critique is entirely incorrect. This post, however, is not just incorrect but actually dishonest.

    The fact that a Falluja cluster even got studied raises red flags. The studies authors bemoan the lack of security in the nation and explain that it caused them move the original site of other clusters yet they managed to get one cluster in downtown bandit central, in the one city in the entire country that had absolutely no Coalition presence at the time the study was conducted. Riiiiiiiiiight, I’ll by that

    This is stupid. The clusters were chosen at random. There was no “moving of other clusters” and a simple reading of page 2 of the study makes it clear that this is the case. In fact, the Lancet team also took a cluster from Sadr City, the other main area of high violence, and as it happened found zero violent deaths there. This accusation of dishonesty in the selection of clusters is, in my non-professional opinion, very seriously defamatory.

    The Lancet study does a good job of summarising the following facts:
    1) In the Kurdish North, the death rate fell
    2) In most of the rest of Iraq, the death rate rose by somewhat more than 50%, the average being pulled down by the Kurdish governorates
    3) There were also isolated areas like Fallujah in which the death rate increased by much more than 50%, these additional deaths being entirely the result of violence.

    The table is presented so that anyone who wants to can see exactly what happened, and it is downright stupid as well as dishonest to claim that the decision to give figures with and without the Fallujah cluster, saying at every stage that this is what was being done, constitutes “whoring out of science”. An example of “whoring out of science” might be someone pretending that cluster sampling tended to overestimate rare effects when they knew it didn’t. This is what Shannon Love did, and that makes Shannon Love, by the standard of this post, a liar.

    The bottom line here is that when bombs are dropped in clusters, they tend to land in clusters, and therefore it is stupid to entirely exclude high-violence “outliers”, even if it is also wrong to incldue them in the computation of summary statistics. Anyone with an ounce of sense, or with any experience at all in practical work, can see that the sensible thing to do is to report all the numbers so that an intelligent reader can see the underlying facts; an increase in the baseline death rate of around 50%, plus the deaths in the high-violence clusters.

11. “Cluster sampling was a poor design choice expressly because it was likely to produce just such an outlier.”

    Leaving aside the fact that this is simply wrong (cluster sampling is likely to miss the small neighbourhoods where the carnage is most heavily concentrated), what design would you have chosen in the difficult circumstances faced by the researchers?

12. Kevin. I don’t know if you are asking a rhetorical question, so please pardon the response with that in mind.
    If the survey can’t be done other than in a flawed fashion because of security concerns, then it ought not be done at all, or its results listed as something other than the Lancet did.
    Or, to approach the question from another direction, that the study was difficult does not retroactively make flawed numbers valid.
    Without going into the methodology, averaging one of the worst battlefields into the mix as if it represents some kind of standard is suspect.
    Outliers ought to be “out”, described and studied as single events.

13. I don’t fully buy Shannon’s analysis but she does raise several good points that critics like DSquared simply ignore in their haste to call her a “liar.”

    DSquared claims that it is “stupid as well as dishonest” to claim that selective inclusions of the Fallujah data were misleading. I disagree and I also disagree that the authors were “saying at every stage that this is what was being done.” The point isn’t that they shouldn’t have analyzed the data both ways. The point is that they often began their interpretation by presenting results *without* the Fallujah data and then slid immediately into conclusions that *did* include this data without mentioning this shift. By their interpretation of death rates, one could easily come to believe that the Americans were rattling about the countryside indiscriminately shooting women and children! Indeed, this is *exactly* how some opponents of the war characterize American behavior. And it is just dead wrong.

    It is not “stupid as well as dishonest” to point this out and DSquared is obviously smart enough to know better.

    The two biggest issues that I see with this study – one of which isn’t dealt with here at all – are (A) the reliance on circumspect data on pre-war death rates (not discussed here) and (B) reliance on data from subjective interviews of a population made up largely of people who wanted to see the Sunni Baathists returned to power. It is simply incomrehensible that anyone can take the Fallujah data seriously when there is zero control for alternative explanations of the data (such as political manipulation). Hell, I’ve had to run entire experiments over from scratch due to less serious data collection issues.

    This is like interviewing the Third Reich in their bunkers in the days before the fall of Germany in WWII and asking them what they thought of the moral standing of the Allies. What the hell do you expect them to say?

    We should all recognize that any attempt to topple a ruthless dictator will cost innocent lives. It has been true since the beginning of human civilization that one must employ violence to beat back those who would destroy that civilization. This is a very important debate: just how long a leash is a despot given before he must be removed, knowing that this will cause grievous suffering and innocent lives lost? It is an excruciatingly difficult question but one that a mature and responsible intellectual class should be able to debate. Unfortunately the Left appears only capable of posturing and posing. Apparently, in their view, there is *no* justification for war and any number of innocent lives lost discredits the entire enterprise. Of course, to convince the rest of us, they stoop to the obvious and clumsy distortions introduced in the Lancet’s conclusions to inflate the number. But, truly, what kind of “idealism” lets the thug and the tyrant flourish and the people be crushed underfoot? What kind of “peace” movement protests the horrible war while absolving the criminal gang of its responsibility for starting it? What kind of people refuse to soften their hearts at the prospects of millions of Iraqis choosing their own future and instead hails the Jihadist fanatics as “modern patriots and minutemen”?

    Not the Left that I grew up with and not a Left that I’m willing to side with ever again.

14. I do not withdraw the epithets “stupid” or “dishonest” and will not do so while Shannon continues to pretend that the cluster sampling critique is valid. I also stand by my assessment of the summary and conclusions; it is perfectly obvious to any reader which conclusions are based on the ex-Fallujah results and which are based on the cum-Fallujah results.

    reliance on data from subjective interviews of a population made up largely of people who wanted to see the Sunni Baathists returned to power

    Two errors here. First, the population of Iraq was not and is not made up largely of people who wanted to see the Ba’ath party returned to power. Second, the data was not “from subjective interviews”; it was checked against death certificates on a sample basis (two certificates per cluster, except in those clusters which did not have two deaths). When asked, 81% of respondents were able to produce death certificates.

    This is like interviewing the Third Reich in their bunkers in the days before the fall of Germany in WWII and asking them what they thought of the moral standing of the Allies

    No it isn’t. It’s like interviewing the Third Reich and asking them if a member of their household had died in the last eighteen months. There were simply no questions about “moral standing” asked in the Lancet survey, and I respectfully request that you read the damn thing before continuing to comment on it.

    And finally, Richard Aubrey:

    Outliers ought to be “out”, described and studied as single events.

    It is not at all obvious that Fallujah was a “single event”; it stands proxy for Sadr City, Mosul, Samarra and Najaf, none of which were sampled.

15. (erratum: of course Sadr City was sampled, however by sheer fluke the survey picked a cluster which had seen zero deaths despite being in the middle of the fighting, a fine example of the tendency of cluster sampling to underestimate the frequency of rare events).

16. Dsquared said:

    “The bottom line here is that when bombs are dropped in clusters, they tend to land in clusters, and therefore it is stupid to entirely exclude high-violence “outliers”, even if it is also wrong to incldue them in the computation of summary statistics.”

    I don’t understand this argument. OFI and the aftermath have seen the most concentrated use of smart weapons in history. This has the effect of greatly reducing collateral damage – innocent civilian deaths. Given the weaponry, I become suspicious of claims of large numbers of innocent civilians being killing by American bombing. It doesn’t mean that it isn’t happening but if it is there could be other reasons. For example, if the terrorist-insurgents are in a mosque and have taken a group of hostage/human shields with them then a bomb that takes out the terrorists will probably take out the hostages.

    If Dsquared is claiming that the U.S. engages in indescriminate bombing – that’s what I infer from his statement – then I’d say he’s wrong. Of course, in a fog-of-war situation in which CAS missions are carried out in an urban environment, as was the case in Falluja, mistakes will be made. Still innocent civilian casualties will be lower for urban combat than ever before in history.

    What is really significant about the battle against the terrorist-insurgents is that it has shown that the old cannards about urban combat are wrong. Unlike the Russians, who leveled Grozny and really did kill innocent civilians by the bushel, U.S. urban warfare doctrine has kept civilian casualties very low.

    I think the Lancet study needed to be more discriminating in determining exactly what it was trying to show. A more interesting study would be to compare civilian casualties in historical urban combat situtations with current U.S. doctrine.

17. “The bottom line here is that when bombs are dropped in clusters, they tend to land in clusters”

    9 out of 10 bombs dropped in Iraq are laser guided, precision weapons. Artillery shells, main battle tank shells and mortar shells are also precision weapons (especially if you’re to avoid “blue on blue”).

    All the talk about the statistical study will never explain is why there are no mass graves digged today, why Iraqis voted “en masse” (pardon the french), why “resistance” has not picked up (if it were a massacre, it should have) etc etc…

18. By the way, if the body count ratio is 1 to 10 in favor of the US Army it means that 15.000 terrorists/thugs were killed in combat/bombing/etc. (US casualties: 1500 => 15.000 terrorists. Excluding accidental death and IEDs, maybe it’s only 10.000 terrorists killed)
    Of course they’re all “civilians” because none wore a uniform.

19. Richard Aubrey,

    I am not asking a rhetorical question. If critics of the Lancet study consider cluster sampling inappropriate then it is hardly unreasonable to ask just what methodology they favour. Your only response here is:

    “If the survey can’t be done other than in a flawed fashion because of security concerns, then it ought not be done at all, or its results listed as something other than the Lancet did.”

    I don’t know what you mean by listing of results in this context. On my reading, the main purpose of the Lancet study was to determine whether Iraq’s mortality rate was raised or lowered as a result of the forcible regime-change. The result is unequivocal.

    Of course security concerns meant that the study would be flawed, which for you means it should not have been done. Perhaps you would argue that studies which attempted to quantify the loss of life in the Rwanda genocide or the war in DR Congo should not have been done. I don’t agree and you present no reason why I should.

20. It’s tommorow, where’s the correction RE: your inability to correct for time periods?

21. Kevin, you clearly (accidentally, of course) missed my caveat. If one is going to present a study whose results could be flawed by the inability (due to security or any other reason) to do a first-class job of sampling, then the results should be presented as at the very least, coming from a flawed sampling process and thus should be taken as substantially different from other studies whose credibility is not so damaged.
    You implication that I would oppose other studies is absurd, resulting from a non-accidental misreading of my statement and adds a retroactive flavor to your objections.

22. Dsquared:
    As to Fallujah standing proxy….
    I’m tempted to say “prove it”, but of course you can prove Fallujah is used to stand proxy. My point is the implication, supported by little, that there is any useful or significant similarity between Fallujah and the other cities you mention. IMO, that needs to be demonstrated, not asserted.

23. I am learning a lot just by reading the back and forth here. I hope all involved can keep up the informed debate and limit the insults. This kind of fact-checking is what blogging is all about.

24. Shannon, You’re trying to herd cats!

    Researcher bias kills the Lancet report in its crib, as far as this American is concerned.

    Debating the minutiae of its sampling errors, use of clusters and selective in/exclusion of data only dignifies its promulgators with your valuable reason, time and verbage.

    Good luck!
    -Steve

25. Can anyone tell me if the Lancet ever published a response to the criticism leveled at this report? If so, what was the gist of it?

26. It’s tommorow, where’s the correction RE: your inability to correct for time periods?

    C’mon, guys, this is a hobby. If you want to pay us a few thousand per post then we’ll whip up a schedule and stick to it. Otherwise you’re just going to have to wait.

    James

27. Richard,

    I am not deliberately misreading you, if I am accidentally misreading you then I respectfully submit that this may be the fault of your writing rather than my reading. A straightforward reading of your earlier comment is that the study should not have been carried out in view of the obstacles presented by the security situation. It now appears that is not your view; so much the better.

    When you now say “the results should be presented as at the very least, coming from a flawed sampling process” do you actually mean to imply that the study does not alert the reader to the flaws? For my part, having read the study I am quite satisfied that it treats the reader fairly in that regard.

    My question to Shannon Love is a simple one: if cluster sampling is the wrong approach, what is the right one?

28. If cluster sampling was an appropriate way to measure excess deaths, then why not publish the result that your sampling technique actually gave you?

    It’s extremely tricky logic to argue that a study was flawed for unknown and uncontrolled-for reasons, but that exclusion of a single data point has corrected for these errors, although they remain unknown and uncontrolled-for. It’s much more likely that you consciously or subconsciously massaged the data until a ludicrous result entered the high side of believability.

    It’s extremely tricky to exclude data after you’ve collected it. Unless you have a very good idea of why (and how) the data isn’t measuring what you wanted to measure, the only real answer to outliers is to conduct a better study and publish that.

29. Pingback: Deltoid

30. There are 4 very serious problems with this study:

    1) The study says nothing useful. The only meaningful (i.e. supported by a 95% confidence level) statement that can be made based on the data is that between 8,000 and 194,000 people were probably killed.

    2) The pre-war death rate the study uses seems unusually low, lower than Iran’s or the EUs. Iran has a similar population age with a standard of living 3 times higher. When a more reasonable pre-war death rate is used, the already hard-to-detect effect disappears completely, or suggests the war saved lives.

    3) Published reports say the authors only agreed to do the study on the condition it would be published before the U.S. elections. Political motivations lead to bad science.

    4) The authors failed to survey some of the heavily Kurd and Shia provinces, which benefitted the most from the war. This is a very serious flaw which would tend to produce misleading results, and one suspects this is not unrelated to point 3.

31. When I was in the Army, I had the disagreeable duty of notifying next of kin on several occasions. I kept up with two of the families (one mine) and discovered that both parents of one guy killed had heart attacks within a year of his death, and in another, the mother had one, but not the father.
    In a country with substandard health care, these might have been deaths.
    My question regarding the Lancet’s inclusion of non-combat deaths is whether this has ever been done before? Did anybody ever count the number of parents who died of grief after Pearl Harbor?
    Of course not. How about those who died because so many physicians were in the military during WW II that homefront health care declined? Of course not.
    This inclusion might seem scholarly, but it reeks of an attempt to punch up numbers which, without it, could be anticipated to be too small to be useful.

32. Zach, everyone knows that outliers cause headaches for researchers. But doing a whole new survey isn’t a sensible response. There is nothing shameful about reporting the results with and without the outlier and letting readers draw their own conclusions.

    The answer to your “why not” question is, they did publish the result their sampling gave them. Don’t take my word for it; read the study.

33. For me, the real significance of the Falluja results is not merely that they were absurd on the surface – and I feel safe in saying that, considering that they pointed to hundreds of thousands of death in a city where the rebels themselves had claimed only hundreds of innocent deaths. It’s also that there really was no more reason to exclude those results than any of the results. Methodology-wise, that is. They looked ridiculous so – heck, must be an outlier. But methodology-wise, the other results were arrived at the same way. That is, the absurdity of the Falluja results implies very strongly that the other results might be just as absurd.

    I aslo agree with someone up above that the “excess deaths” ratio depends on accepting their calculation of pre-war deaths. I’ve heard a lot of people say – what, wasn’t Hussein killing a lot of people? Where were those violent deaths? But that’s not the point for me; it may well be true that Hussein was not actually killing thousands of people a year before the war. The point, from what I’ve read, is that numbers from pre-war Iraq were incredibly unreliable; and that the death rates reported compare favorably to the USA, in a land that was still suffering under sanctions and a dead economy.

34. J. Bennett asks:

    “Can anyone tell me if the Lancet ever published a response to the criticism leveled at this report?”

    The lead author has responded to e-mails requesting clarification of some points. If you click on Tim Lambert’s name in the comments above, you will find that he has examined the criticisms in some detail; he also provides numerous links.

    You should note that much of the criticism has consisted of name-calling which no sensible person would bother responding to. Also, many of the critics haven’t read the study and/or are are innumerates posting gibberish about sampling, confidence intervals etc. which they have picked up from other innumerates.

35. Although I have read some persuasive arguments about why this study is correct, and I accept them, I have yet to see a persuasive argument about why the lead researcher on this project retroactively lowered the pre-war mortality rates, after having argued for years that they were incredibly high during the sanctions of the 1990s.

    The methodology of this study may be perfect, but if the basic premise (that prewar rates were as low or lower than Iraq’s neighbours) is wrong, then no amount of scientific rigour *during* the survey will save it.

    Anyone care to address this?

36. I love how Kevin, with no hint of irony, complains about namecalling and then does it himself. I guess as a sensible person I shouldn’t respond to it.

37. Philip, this issue has been addressed; I can’t go into all the relevant points, but the main point to note is that the survey examines both pre-war and post-war mortality. Even if the pre-war figure is biased downwards, which is certainly possible, that does not affect the results if there is a similar bias in the post-war figures. That is part of the reason for designing the study in this way.

38. While I’m taking the risk Kevin will call me names again, on the results I’d just like to point out that had they found a 95% confidence interval of say, 50,000 to 150,000, then we could say with 95% confidence that at least 50,000 people were killed. That would be both meaningful and useful to their cause. Saying at least 8,000 were killed isn’t all that useful to their cause, which is why they gave the point estimate to which no confidence level is or can be attached.

    I think the pro-war folks can take great comfort in the fact that even a politically biased study using terrible methodology that oversampled Sunnis and used a an incorrect pre-war death rate, they still only proved 8,000 people were killed.

39. Assuming that there is a similar bias in the post-war death numbers seems like a pretty thin reed. Why would that be the case? I mean it might be, but I can think of a lot of reasons why it wouldn’t be.

    Anyway, the main point about the pre-war deaths is that like the Falluja results, they seem ridiculous on their face. And you don’t have to adjust them upwards very much at all toward what might seem more reasonable (not proposing scientific rigor here, just observing something about math) before the “excess” figure melts away substantially.

40. Hi Shannon,

    I completely agree that the selective treatment of Fallujah is the single worst problem with this study. You’ll find that I’ve commented on that extensively.

    It’s one thing for death rates to be higher, and quite another for coalition forces to have killed tens of thousands of innocent women and children through bombing, an accusation that is completely unsupported by the actual data of the Lancet study outside of the problematic Fallujah cluster.

    After Iraq’s liberation, hundreds of thousands of cars were imported into Iraq. It’s quite possible that there is a direct connection with more road traffic fatalities today (the numbers are really too small in the Lancet study, but accidents are up from 4 to 13, much more than would be expected from the different time spans covered).

    However, it’s hard to argue that these extra deaths should be classified as “war dead”, even if there is a causal connection with the end of Saddam’s regime, or that these deaths are morally equivalent to women and children shot by careless coalition soldiers.

    Many war critics seem to regard US foreign policy as a greater threat/problem than the fascists and islamist terrorists in Iraq, but you are a little crass in the way you phrase this fact.

41. You know, I’ve been out of town for a week, too busy to keep up with Chicagoboyz. And so when I checked this blog last night I was met with a feeling of warm nostalgia. The topic that got me reading this blog in the first place, the Lancet article, had been revisited! I’ll try not to rehash what I’ve already said in past comments on past posts, but here’s a few points:

    1.) I highly object to the characterization of the Lancet paper as dishonest. Echoing Dsquared, the authors did a very honest and conventional thing when they presented all data in a table and also gave results with and without Fallujah. This way the reader can see what’s going on in their data and what a more conservative and less conservative estimate may be.

    2.) The authors NEVER pretended that their results were 100% the absolute truth of final. The discuss the limitations extensively, and they emphasize the need for future research in the paper. Not only that, but when the authors are interviewed they go on to state that their findings are not final but just a beginning in the important effort to discern what’s really going on.

    3.) The authors of this study don’t wear lab coats.

42. TallDave,

    In my reply to Philip I wasn’t referring to this thread, particularly. From the beginning the debate has been muddied by false claims about statistics, in particular “Kaplan’s fallacy”: the notion that numbers far from 98,000 are just as good as those closer to it. So I had an entire category of Lancet critics in mind, not just one member of that class. But if the cap fits by all means wear it.

    Having said that I have come to see the wisdom of d-squared’s criticism, way back then, of the way the study was presented and the highlighting of that 98,000 central estimate. It is better to focus on the main finding of the study: Iraqi mortality has risen at least a little and probably quite substantially. That finding seems quite robust.

    Some people will say: so what? That’s a bit OT, but worth a brief comment. It depends on what your expectations were. Some people supported the war on the grounds that it would benefit the Iraqi people. They face greater difficulty making their case in the light of the Lancet findings; which is not to say that their case collapses of course. For those who supported the war on other grounds the Lancet findings may well be irrelevant.

    That is why you are wrong to say that the low estimate of 8,000 “isn’t all that useful.” Please note: it is an estimate of excess deaths, over and above the quota which Saddam could be expected to cause. To the “humanitarian war” school it is undoubtedly an embarrassment, which is why they hate the Lancet so.

    Such is life. If an estimate of -98,000 had emerged (negative excess mortality), no doubt the loony left would be jeering: “Ha! They found no deaths in Sadr City! What a crock!”

43. Very nicely stated, Kevin. Thank you.

44. Kevin –

    I don’t think any war supporter or semi-supporter would shy away from an observation that in the chaos of the first two years after the invasion, there were arguably more deaths for all kinds of reasons as compared to the situation of tortured stability before.

    It’s that “100,000 excess deaths, most from violence and most from Coaliton air activity” that falls apart under close examination, a fact which you seem to be akcnowledging while trying to minimize, and which also is the main thing that has political meaning.

    And I still haven’t seen anyone directly address the issue of the low death rate in pre-war Iraq, the figure which is all-important for any calculation of excess deaths.

45. chel,

    “the authors did a very honest and conventional thing when they presented all data in a table”

    Actually the table is incomplete as it leaves out the deaths from non-violence that were reported in Falluja. Further the authors do not report the breakdown of such critical observations as the number of people reported killed in airstrikes in Falluja and elsewhere. As in several instances, they blend Falluja in without pause creating false impressions.

46. They were, however, so kind as to provide that data in an email.

    Outside of Fallujah, the violent deaths are made up of 12 non-coalition caused deaths, 3 deaths from coalition small arms fire and 6 deaths (in 3 incidents, one with 3 people dead, one with 2 people dead and 1 with a single death) from the use of coalition aerial weaponry.

    Of course, this means that one single self-reported incident is responsible for half of all ex Fallujah bombing fatalities, and also fully half of all ex Falluja child and woman deaths.

    This single incident may be a “fluke”, it may be completely made up, or it may have been misattributed, for example the real cause of it may have been an insurgent mortar rather than coalition aerial weaponry.

    Outside of Fallujah, the data are just too thin, and in no way sufficient to support the (most killed through violence and most of those through coalition air power and most of those women and children) conclusion that is so inflammatory about the study.

47. Hi Kevin,

    of course, the humanitarian argument is vital. It’s the main reason I supported the action.

    My estimates of what’s happened so far:

    Infant mortality (too little information, probably little changed)

    Other non-violent adult mortality (likewise)

    Crime (probably up by about 10,000 deaths)

    Baathists/islamist (probably about 25,000 deaths)

    Innocent Iraqis or Iraqi police/national guard killed by the coalition (around 3,000)

    Innocent Iraqis or Iraqi police/national guard killed by the “resistance” (around 10,000)

    Torture (down, but still a problem)

    Freedom of expression, general political freedom, such as freedom of assembly (much improved)

    Economic well-being (generally improved, but patchy in some respects, eg availability of gasoline)

    These numbers by no means destroy the humanitarian case, assuming that future Iraqi governments can contain the “insurgency” and consequently massively improve the wellbeing of Iraqis.

    So far, in my judgment the suffering and the gains for the Iraqi people are still in balance. I think there is already a net gain, but it is precarious.

    Finally, the original decision was finely balanced. It was built on a belief that removing Saddam could be a catalyst for change in Iraq and the wider Middle East, and for the better. But it was a decision reasonable people could disagree over.

    What I find objectionable in the anti-war movement is an unwillingness to accept that the decision was genuinely made with the well-being of Iraqis in mind, and also an overwillingness to consider “imperialism” as a greater danger than true fascisms, as practised by the Baathists, and true religious extremism as shown by Zarqawi’s ilk.

    One telling example of this (for me at least) is the issue of gay rights. The US is one of the most liberal countries in the world in this regard. There are only four countries (parts of Canada and the US, and Belgium and the Netherlands) where gay marriage is legal. Yet, in half the world’s countries homosexuality is outlawed and in seven it carries the death penalty.

48. “The authors NEVER pretended that their results were 100% the absolute truth of final. The discuss the limitations extensively, and they emphasize the need for future research in the paper. Not only that, but when the authors are interviewed they go on to state that their findings are not final but just a beginning in the important effort to discern what’s really going on.” – chel

    It is time to enter the real world. The 100,000 figure is treated as a proven fact by the left and the Arab press. There is no debate at all, it is just accepted as truth. If you want me to copy and paste a few instance, feel free to ask.

    When I heard that study determined there was between 8,000 and 194,000 casualties, and that the 100,000 was some sort of average, I was shocked.

    This 100,000 figure is being used to prove that Americans and coalition forces are showing a callous disregard for Iraqi lives, when a few of the soldiers I have talked to tell me they take great pains, and put themselves at higher risk trying to prevent civilian deaths.

    This 100,000 is used to raise money for terrorist attacks, it is used to recruit terrorists from Jordan, Egypt, Saudi Arabia, etc.

    This study has consequences. I would be embarassed if I was a part of researching this study. When you say the authors have mentioned the need for furthur study, and that the results are not final, that is the first I have heard of it. And for someone who follows the Arab media as well as the usual papers and journals here, I can guarantee you they have not reported that those numbers are not final either.

49. Heiko Gerhauser:
    about the US:
    how will this do for trying to measure how selfless the US is?
    – How many wars have there been in the world the over the last 100 years (that lasted more than a week)?
    – in how many of those wars did the US intervene?
    – in how many of those wars where the US intervened, did the US have “personal” interest?

    about gay rights in the US:
    I don’t really follow this subject much, but as far as I know only of few states in the US allow gay marriage. Also I remember something about Mr. Bush trying to make gay marriage illegal.
    Am I missing something?

50. “The authors NEVER pretended that their results were 100% the absolute truth of final. The discuss the limitations extensively, and they emphasize the need for future research in the paper. Not only that, but when the authors are interviewed they go on to state that their findings are not final but just a beginning in the important effort to discern what’s really going on.” – chel

    Link

    Over 100,000 Iraqi deaths since war
    Al Jazeera

    The high death rate is mainly due to violence, with many of the victims women and children, according to US public health experts.

    “Making conservative assumptions, we think that about 100,000 excess deaths or more have happened since the 2003 invasion of Iraq,” researchers from Johns Hopkins’ Bloomberg School of Public Health in Baltimore, Maryland, said in a report published online by The Lancet medical journal on Thursday.

    “”Violence accounted for most of the excess death and air strikes from coalition forces accounted for the most violent deaths,” the report added.”

    Link

    Thousands Protest US-Led Invasion of Iraq
    Arab News

    Thousands of people gathered in London brandishing posters denouncing the “war on terror” and the US-led invasion of Iraq. Protesters placed a coffin in front of the US Embassy with “100,000 dead” written on it.

    Link

    Anti- war demonstrations in London and U.S. cities
    Arabic News

    Protesters placed a coffin outside the American embassy in London with the words “100,000 dead” written on it. In November last year, “The Lancet” British medical journal reported a medical study that said that”Making conservative assumptions, we think that about 100000 excess deaths, or more have happened since the 2003 invasion of Iraq. Violence accounted for most of the excess deaths and air strikes from coalition forces accounted for most violent deaths.”

    Link

    Counting the Iraqi Dead
    Fairness and Accuracy in Reporting

    According to a study published in the respected British medical journal The Lancet (10/29/04), about 100,000 Iraqi civilians have died as a result of the war. The majority of deaths were due to violence, primarily as a result of U.S.-led military action. One of the researchers on the project said that the estimate is likely a conservative one (New York Times, 10/29/04). It’s certainly a more scientific estimate than the Iraq Body Count figure cited by ABC , which is, as that project’s website notes, a “compilation of civilian deaths that have been reported by recognized sources…. It is likely that many if not most civilian casualties will go unreported by the media.”

    [Ahem] Is it me, or did I miss where someone was “pretending” these figures were 100% absolute truth, or that these figures were final?

    If you want a few more sources of how this study is being used, just ask. Although some of them would need to be translated.

51. I’m sure Heiko Gerhauser can take care of himself, but Rune Petersen seems to equate law in countries that can not decide whether homosexuals should be thrown from buildings or building should fall upon them in terms of execution with an argument that legalization in other states should not be determined by a single states’ court. This is not a question of making “illegal” but only making “legal” only if a state has chosen to do so. Nor did Heiko exaggerate: he quite aptly noted that gay marriage is accepted in some parts of the United States and Canada. He was accurate. A homosexual is quite likely to find America a more comfortable place than many places in the world. Heiko has a sense of proportion. Such lack of proportion makes it hard to take Petersen’s arguments seriously.

    I don’t see an argument that America is altruistic; I do see an argument that conditions in Iraq were bad for its citizens as well as for our country, if for no other reason than it was not pleasant to keep pilots stationed there to patrol the no-fly zone continuously. (Of course, because of those no-fly zones, the Kurds had a relatively comfortable and relatively democratic and relatively just life.) That combination of motives impel most actions – a desire to benefit oneself and a desire for a higher good.

52. Cog,

    I am not, have not, and will not defend how the media can create false impressions of science through simplified reporting.

    However I stand by my statement. If you read the article, it discusses many limitations of the study and emphasizes that future research is necessary. Furthermore, when an author is actually interviewed, he states in simple terms what he thinks the study does show and very honestly states that they do not claim this is 100% final truth.

    Check out Dr. Gil Burnham’s response to the following question in The New Republic:

    TNR: How confident are you that you have a sample size that’s adequate?

    BURNHAM: We calculated our sample size, and the number of people that we did have in the study is well within the range that would detect a change in mortality rates — we were looking for half, or 60 percent change. It was well within that range. Now, I don’t want people to be thinking that this is the definitive study on mortality in Iraq. Basically, as you can imagine, in Iraq, doing any kind of survey is in itself very dangerous. So we wanted to do something that would give us an idea about what changes in mortality would be, as a rough indicator of where mortality was for the civilian population. Now, the final word on this is going to be done by a proper census-type survey, and this is by no means intended to be the definitive survey. But it is a way that, within its statistical limits, can tell us with some confidence: Is there an increased mortality rate [due to the intervention] or is there not? And I think that from the results we can say with confidence that yes, there is definitely an increased mortality rate.

    Now, you can argue, is this increased mortality rate 70,000? Is it 60,000, is it 150,000, is it 200,000? Our best guess, on a conservative side, is 100,000. But it could be less and it could be more. Because just by the statistical nature of this thing, the kind of zone around this number where we are sure this answer truly lies is fairly broad. It’s a national survey, it’s a massive survey, but it’s not a national census.

53. This is not a “false impression”. This is an outright distortion, with very serious consequences.

    The 100,000 excess deaths figure has been proffered as fact every since the study was released. The authors of the study have done next to nothing to dispel that myth, and clarify what their study actually shows.

    And it has resulted in very serious consequences, and more excess deaths as a result. Unless you think that the Arab media, and left’s characterization of the study has no effect on terrorist recruiting and financing.

    That is the fatal flaw of both the Lancet study and the IBC. Holding a terrorist accountable for killing and injuring civilians is beyond the scope of what they are trying to prove.

54. Make every, ever above.

55. From the law end, I find the argument misses one large point. Let’s say the Lancet boys are in the dock giving evidence about what they saw in a brawl. Tough cross-examination, points made and lost.
    Ok, but then what happens is their counsel stands up and says “I call the other witness.” The US Army climbs into the dock. Lawyer says “OK, that’s what they say happened. What do you say happened?”
    Silence
    “I’m sorry, I can’t hear you.”
    “We don’t do body counts.”
    “So you were there, you had a full opportunity to produce your own estimate, you had every motive to do so, I put it to this court that it’s virtually impossible to believe that you didn’t do so, and you won’t share your estimate with this court. Aren’t we entitled to believe that you feel it would not be advantageous to you? That, indeed, it would be less favourable to you than the Lancet estimate?”
    If I was in the jury I know it’d sway me.

56. Chris. You really need swaying?
    The US doesn’t do body counts because the anti-war left used that to accuse the US of being interested solely in killing. The anti-war left uses the lack of body counts to accuse the US of being callous.
    Can’t win, which is a given, but the second course is probably less troublesome.
    I know you’d like body counts so as to be able to accuse the US of wanting to kill everybody it can find. Too bad, huh?
    We have a discussion right on point. Your implication of nefarious motives for not doing body counts is, 1, obvious, and, 2, nowhere nearly as effective a point to make to the really dumb people who buy this crap as being able to point to body counts to demonstrate the US wants to kill everything it sees.
    Thanks for the demonstration.

57. I still disagree about the “do a better study, and publish that” comment. The tremendous error bars, plus the enormous sensitivity of the study’s result to a single cluster indicate that the study has almost no utility in differentiating between numbers of dead.

    As an example, consider what’s probably the best estimate of civilian deaths, (direct deaths, not excess) Iraqbodycount.com. They measure by direct enumeration, have a well-defined minimum and maximum. Currently, they report a minimum of 17,085 civilian deaths, and a maximum of 19,457. Of those two numbers, which is the closest to the truth? This study won’t tell you. The error bar is too big. How about 18,271 and 36,542? That’s the Iraqbodycount.com mean and twice the mean. Using the Iraqbodycount.com methodology, 36,000 deaths is utterly absurd. Using the results of this study, it’s almost as likely as 18,000.

    If you wanted to get a useful estimate of excess dead, a far better approach would be to examine morgue records, and perhaps do a statistical sampling to determine what percentage of dead in certain areas don’t get brought to the morgues. That might give you an error of 10,000 or so — certainly you’d get something comparable to the Iraqbodycount.com error bar. Doing the study this way — even stipulating that the researchers did it with the purest of intentions — gives you error bars that are too big to do anything useful with them, like tell the difference between a credible and a ridiculous number of civilian dead.

58. The IBC count is replete with flaws as well. I am not sure if you followed the war in Afghanistan, but after each encounter, the Taliban claimed scores of coalition forces were killed, and an equal number of civilians killed and wounded. All were the fault of the coaltion of course.

    Now with many of the Qaeda training actually posted online, you can get a glimpse at how they have stepped it up to the next level complete with audio/video presentations, a distribution network, and even their own graphics.

    Many of their claims are accepted as fact, as was the 100,000 Lancet figure, without question by the Arab media. Many of these reports have filtered through the IBC figures.

    “That is the fatal flaw of both the Lancet study and the IBC. Holding a terrorist accountable for killing and injuring civilians is beyond the scope of what they are trying to prove.”

59. Zach,
    Why even do a “study” in the first place? What possible motivation would drive the need to fund the salaries, stipends, data collection and distribution fees for such a study?

    Is it because the Pew Charitable Trust and Ralph Neas told you to?

    If you’re cleaning the blackberry brambles out of your backyard, do you conduct a “casualty count” of the chopped stocks and ripped leaves as you load your wood-chipper?

    No, you wash your hands, put ointment on your thorn-wounds, and get on with business. Unless, oh, I forgot, America is the “bad guy.” ‘Tsk, tsk.”

    Why do you hate us?
    -Steve

60. Oh, surely true. But they have clear criteria for what gets in, clear criteria for what gets left out, and they stick to it. They could be off systematically if people are gaming the system, but you could correct for it if you really wanted to. By using direct enumeration, they get error bars that are appropriate for the number they’re measuring — total reported civilian deaths. If you used a more reliable source, like the morgue records I recommended, you’d get correspondingly lower error bars.

    If the Lancet study reported 100,000 +- 5,000, I’d be confident they knew what they were doing. 100,000 +- 10,000, well, war is tough for everyone. 100,000+-92,000? What the error bars are telling you is that they don’t understand how to measure excess deaths well enough to give an accurate answer.

61. There’s plenty of reasons to do a study. Not least, because people want to know. And I’m sure that they weren’t planning on 90,000-odd error bars when they planned the study. The fact that the error bars got that big indicates that the issue was quite a bit more complicated than they figured on it being — so much more complicated that you have to doubt whether the answer they got when they were finished means anything.

62. Warning: Mega-length post, apologies in advance.

    From debating the Lancet study at length at Tim Lambert’s blog, I became convinced that the study methodology was sound, however I believe valid criticism can be made of the authors’ stated conclusions, and the inherent ability of the study to provide a reliable excess death number. On key points, these conclusions are either deliberately misleading (and inflammatory, as Heiko points out), or incorrect.

    The authors have pegged the deaths from coalition air strikes at approximately 30,000 (of the total 100,000 ex-Falluja excess death estimate), yet the actual study data allows for approximately 17,000 deaths from air strikes. Such a large discrepancy for the number one cause of death (as cited by the authors) has to have a bearing on the study’s reliability, even though this discrepancy does not affect the aggregate excess death figure of 100,000.

    This also leads to the issue of volatility within the individual violent death subsets. The aggregate number is composed of the sum of the subsets (both violent and non-violent). The aggregate number relies on the integrity and accuracy for extrapolation purposes of theses subsets. While I realize I am committing statistical sacrilege in saying this, these subset numbers are very small, and I can’t see how this doesn’t make them prone to volatility if this survey was repeated using identical sample size and methodology.

    To put this another way, there are certain causes of death (particularly violent death) that have a much greater bearing on the excess death figure than other causes of death. If the most influential of the death subsets are prone to volatility, this has to have a profound effect on the ability to cite the 100,000 estimate with confidence.

    As Heiko points out, the study’s stated number one cause of excess death (coalition air strikes) is based on 6 fatalities from 3 incidents. The effect of any error or misrepresentation here (such as an insurgent mortar being responsible for one of the attacks) has a huge bearing on attribution of responsibility, when one realizes the authors are attributing 30,000 deaths from these 6.

    There are other examples of the shortcomings posed by extremely small subset numbers. Of the 21 excess violent deaths that occurred outside the Falluja cluster, 1 of these was a killing attributed to Saddam regime agents, however it occurred during regime change, and therefore served to raise ever so slightly the post-invasion death rate. There was only 1 violent death recorded in the study for the pre-invasion period. We don’t know if this was a criminal homicide, or a Saddam regime murder. If it isn’t a Saddam regime killing, then ironically, the study fails to account for any pre-invasion killing by Saddam, but Saddam regime terror does tack on an extra 3,000 or so deaths to the post-invasion excess death extrapolation.

    The study also fails to pick up the death of a single Iraqi regular army soldier, even though it seems apparent that at least several thousand, perhaps many more, were killed during conventional fighting at the time of invasion.

    There’s one other point I wanted to make concerning the death estimate from coalition bombing.

    Well known Canadian law professor and anti-war activist Michael Mandel lead a campaign to attempt to indict Bill Clinton and other western political and military leaders over the 1999 NATO intervention in Kosovo. In his statement to the International Court of Justice, Mandel claimed that NATO had dropped 25,000 bombs on Serbia, killing between 500 and 1,800 civilians. Human Rights Watch castigated NATO with great fanfare for killing 500 civilians, by its count. Serbia, for its part, claimed as many as 2,000 civilian bombing deaths.

    The Serbian bombing campaign went on for 78 straight days, with 25,000 bombs dropped. Yet the highest civilian death toll claimed is 2,000. I’m not suggesting that the bombing campaign carried out in Serbia mirrors the coalition bombing activity in Iraq. However, it was a very heavy bombing campaign by anyone’s standard, and does illustrate how uncertain and questionable it is to extrapolate a nation wide figure for bombing deaths from this particular cluster survey. Moreover, it calls into question the extremely high death figure attributed to bombing by the Lancet study.

    We also do not have enough media or hospital reports of large scale death tolls from coalition bombing to support either the 17,000 or 30,000 figures for bombing deaths. The April 2004 Falluja offensive appears to be the single deadliest event for Iraqi civilians. By virtually all Iraqi accounts, official and otherwise, we have approximately 600 civilians killed. There are no other similar reports that I am aware of, anywhere else in Iraq, mentioning even a hundred deaths resulting from a single air strike, or resulting from multiple air strikes in an area. A bombing death toll of the magnitude suggested by the study authors, if plausible, should have numerous corresponding individual reports of large scale bombing deaths, with many exceeding 100 deaths per incident. We simply don’t see this corroboration coming from hospitals, morgues, even anti-coalition sources.

    The single greatest flaw I see in relying on the 100,000 excess death estimate is this; if the study were repeated with the exact same survey sample size and the same methodology, the second survey is capable of providing a radically different figure. And I don’t mean to imply that this only includes a lower value. This type of survey is capable of providing an excess death toll of 300,000, and one that would not be qualified and reducedby the presence of any identifiable outliers. All that would have to occur would be for the survey to pick up say 2 neighbourhoods that experienced 10-15 bombing deaths each. And why should that be so unlikely? If defenders of the study are to be believed, then there must be hundreds of neighbourhoods in Iraq that suffered similar fates, if we’re to meet the bombing death estimate provided by the authors. If the authors saw fit to extrapolate 30,000 bombing deaths from 6 actual recorded deaths, how many would they extrapolate from 30?

    What you would then have is two identical studies, one with an excess death toll of 100,000 that has been stridently defended as accurate, even an underestimate by many. The other comes in at 300,000, which clearly cannot be defended by any reasonable person, based on the information coming from Iraq. How can anyone say one is more accurate than the other, if they’re identical surveys?

    The most accurate method for determining at least the violent death component of the toll from regime change is an actual count. Obviously, the current security situation in Iraq precludes this. However, such a count is underway in the Former Yugoslavia, by Bosnian war crimes investigator Mirsad Torkaca, now that a cessation of hostilities has allowed one to proceed. Torkaca has found that the much quoted figure of 250,000 dead is inaccurate. He expects the final total will exceed 100,000, but not reach 150,000. Of course, Torkaca makes it clear that his team is counting only violent deaths, not excess deaths as in the case of the Lancet study in Iraq.

    Conditions will someday allow a count to take place in Iraq. The results will be very difficult to dismiss, whatever they may turn out to be.

63. A few points:

    1) I would still like to see some formal repudiation of the “cluster sampling critique” by Shannon Love. As far as I am aware, he is unwilling to defend this critique but continues to make it. That’s dishonest.

    2) Tall Dave et al: Iran does not have a similar age structure to Iraq. From the CIA Factbook, Iran’s age structure is 28% below 15 yrs, 67.2% 15-64 years, 4.8%> 65 years or more. For Iraq, the figures are 40.3% 64 yrs. That isn’t really all that similar.

    3) Anyone saying “8,000 deaths” is wrong. The figure would have to be “8,000 deaths excluding Fallujah”. Since the number of deaths in Fallujah is unlikely to be a figure that is small relative to 8,000, this matters (note that this entire post is about the inclusion or exclusion of Fallujah! However I will continue to assume that this mistake is being made honestly).

    4) The majority of excess deaths in Iraq will have occurred in places like Fallujah, not places like Basra (Najaf, Samarra, Ramallah and Mosul are like Fallujah in that they saw heavy fighting). Therefore I remain to be convinced that complete deletion of the Fallujah data is a sensible way to deal with the data.

64. It don’t see how we could trust the pre-war data and even post invasion data is suspect, as pointed out by the authors, but if the study is sound, it could be used to measure future improvements.

65. Aaron,

    The study does not lend itself to the use you have in mind. One of its strengths is that it samples pre- and post-invasion data so as to identify the trend in mortality. So if there is a bias affecting both in a similar way, it cancels out.

    The kind of data you are after requires a census, not a survey. The Lancet cannot be expected to do that. The really bad news here is the failure to follow up with the more comprehensive data-gathering which only a government can do, which isn’t happening because the situation is too chaotic and the political will isn’t there.

66. On smart weapons (several posts above):

    I’m not convinced that smart weapons necessarily have the effect of greatly reducing the number of civilian casualties. Many of the pilots are reservists, and lack of adequate training has been implicated in for instance the
    
    tragic death of Matty Hull.

    One of the points made by several (angry) commentators after the event was that this kind of occurance doesn’t necessarily point to incompetence on the part of the US military. UK armored vehicles were identified with a small Union Jack insignia, easy to miss in an A-10 flying at 3-400 mph. I imagine it’s even harder to distingish between a terrorist and a civilian. This is not to impugn the the US military who I know are doing a difficult job in difficult circumstances – more to say that we shouldn’t be surprised by these kind of casualty figures.

67. It isn’t likely that the bias pre and post cancel out. It is more plausible that they compound the error.

    So, I think we have established that the study is useless.

    Census will be used to measure future changes in Iraq, rather than more cluster studies. So it won’t be used as baseline. Its comparison of pre and post is imprecise and methodology biased therefor it’s useless.

68. It isn’t likely that the bias pre and post cancel out. It is more plausible that they compound the error.

    Really? What’s your statistical argument here, Aaaron?

69. Patrick. You miss the point about smart munitions.
    You also miss the point about reservists. Some aviation reservists are actually better than their regular counterparts because of low turnover in the unit. This is not ordinarily true of ground combat units who need, among other things, a level of conditioning civilians are unlikely to keep up.
    Smart munitions hit their targets a very large part of the time, and their ability to do so is designed to reduce the errors inevitable in dropping a bomb which then falls free on an unguided ballistic. So smart bombs actually make it easier for less-skilled pilots to hit than more-skilled pilots with dumb bombs. Best to have high skill levels and smart bombs, but you see the point, by now, I hope.
    The big factor with smart munitions is that you only need one. No longer do we need to drop dozens or hundreds (see WW II’s thousand-plane raids) to accomplish the goal. Worst case is that one or two smart bombs miss their targets. Compared to having one or two bombs of fifty hit the target and the remainder go someplace else, this is an improvement of many orders of magnitude in protecting civilians.

70. Deaths, especially violent deaths, under-reported pre-war and over represented by the post war methodology.

71. Yes, but *why* do you think deaths were under-reported before the war and over-reported afterward? Do you have any reason to believe that they were, other than that this would give a lower number?

72. The unwillingness of survey correspondents to fully report pre-invasion deaths the unavailability of persons affected by pre-invasion deaths (intimidation, desire to skew the results by participants, the dead don’t talk, etc.), uncertainty in pre-invasion records, and manipulation of the sample by occupying forces in some regions etc…

73. Taking in order:

    The unwillingness of survey correspondents to fully report pre-invasion deaths

    You have no evidence for this; recall bias was reported as a possible source of bias in the study but there is no reason to suppose it was a serious problem.

    the unavailability of persons affected by pre-invasion deaths (intimidation, desire to skew the results by participants, the dead don’t talk, etc.)

    Persons affected by post invasion deaths would be just as unavailable (because just as dead) as persons affected by pre invasion deaths. If you believe that a random sample of 7800 Iraqis hate us enough to fake the deaths of family members to make us look bad in the Lancet, fair enough, but this is pure speculation untouched by evidence.

    uncertainty in pre-invasion records,

    Since the study was based on a survey rather than records, this cannot possibly be relevant. Have you read the thing?

    and manipulation of the sample by occupying forces in some regions

    Unless occupying forces can manipulate a random number generator or a GPS system, this could not have happened.

    etc

    This implies you have other arguments; I suggest you use them.

74. “You have no evidence for this; recall bias was reported as a possible source of bias in the study but there is no reason to suppose it was a serious problem.”

    Read a fucking news paper.

    “Persons affected by post invasion deaths would be just as unavailable (because just as dead) as persons affected by pre invasion deaths. If you believe that a random sample of 7800 Iraqis hate us enough to fake the deaths of family members to make us look bad in the Lancet, fair enough, but this is pure speculation untouched by evidence.”

    They types of deaths are very different. Systematic killings are less likely to come out in a survey than the more random casualties of a war.

    “Since the study was based on a survey rather than records, this cannot possibly be relevant.”

    I thought the survey result needed to be supported by records.

    “Have you read the thing”

    I can’t seem to access it.

    “Unless occupying forces can manipulate a random number generator or a GPS system, this could not have happened.”

    Yeah, sure, the on the ground work was completely unaffected by outside factor. A random number generator does not control the factors within the sample it selects.

75. Richard: You miss my point about smart munitions. There are two factors here, first identifying a target, and second hitting it. Smart munitions only help with the second part. It’s far better to have a smart pilot with dumb weapons, since at least the target identification part can work correctly, and indeed dumb weapons can be remarkably accurate in the hands of a skilled pilot – I have immense respect for the skill of the best USAF and USN pilots. A dumb pilot w/ smart weapons may be able to hit what he’s decided to hit 100% of the time – but has that initial decision been made correctly?

    On training, Air Force reservists certianly weren’t trained to identify UK tanks in battlefield conditions – their training was patently inadequate in this area. It’s well known the military are under stress just to field required manpower. Training is one of the things that often slips in these circumstances. I can’t make a categorical statement that training was and is inadequate, but I’ve heard many examples of rushed and poor training of troops in Iraq.

    To recapitulate, heavy civilian casualties are consistent with these problems. This isn’t “callousness” – there was a perception in the US in 2003 that urgent action was required in Iraq on account of WMD etc., and that this outweighed other factors, such as ensuring an adequate troop deployment and adequate training of the troops. I suspect that there was also an expectation that there would be say another 50,000 odd troops of non-US nations from the get-go, but unfortunately clumsy diplomacy meant that this didn’t materialize. The coalition forces went in undermanned and undertrained. The situation got out of control – it’s a testament to their professionalism, ability and equipment that they’ve done this well, and are sticking with the task, and perhaps now making things better. The sad artefact of all of this is that a lot of civilians are dead, quite possibly more than 100,000. It’s an open question as to whether this was entirely avoidable, but it certainly seems like a lot of deaths, and the heads of the US adminstration should not be sleeping soundly at night right now.

76. Pingback: No Illusions

77. This forum is almost useless. The strategy of Shannon Love and the pro-war ‘see-no-dead’ crowd is the same the fanatical right-wing crowd uses in every discussion: lie, lie, lie, someone will then have the chance to use that lie as ‘proof’ that “some people believe the study is incorrect”. They use it all the time (anti-darwinism is probably the best example). It’s useless to discuss with them. Their dishonesty should be apalling to anyone who believes in intellectual debate. Or in decency.

78. I think that’s rather a harsh assessment, though it does trouble me that Shannon continues to link to an argument about cluster sampling which is wrong, which he knows to be wrong (assuming he reads my posts, which I suppose might not be true) and which he is not prepared to defend.

79. dsquared,

    Could you summarise what you believe to be wrong, and why this should be obvious to Shannon?

    She may just disagree that you made a convincing argument, be it because she thinks it’s entirely mistaken, or deals with points she doesn’t think are truely important.

80. dsquared,

    I’m glad I gave you the chance to look like a diplomat (although it looks from your critique of Dr. Love that you might be the John Bolton type of diplomat… :-)

    I still stand by my point on this thread. Love doesn’t even bother on answering your questions, or Tim Lambert critique. Her (or his?) only mission is to feed the underlings with enough rotten arguments to run around the playground repeating her half-cooked, fully-dishonest talking points: the Lancet report is wrong, the authors are suspect. That’s all she/he wants.

81. dsquared and Palo,

    Your imputation of bad motives to someone because he disagrees with you makes clear that you are both idiots.

    See? This point was as valid as are your mean-spirited remarks about Shannon’s motives. Based on your behavior, the most gentle inference I can make about your motives is that you are profoundly overconfident about your own judgment. You probably don’t appreciate the irony.

82. Jonathan, I have made no comment about anyone’s mental states except my own; that I am troubled by the following facts:

    1. In the post “Bogus Lancet Study”, Shannon wrote “Since all sources agree that violence in Iraq is highly geographically concentrated, this means a cluster sample has a very high chance of exaggerating the number of deaths”.

    2. A cluster sample does not have a “very high chance of exaggerating the number of deaths” if deaths are rare and concentrated. It has a very low chance of exaggerating the number of deaths (by a large amount) and a very high chance of underestimating the number of deaths. This is a fact of mathematics, not the sort of thing that there can be reasonable disagerement over.

    3. After considerable discussion in the comments threads on the various Lancet posts, we reached a point at which everyone left in the debate had understood point 2 above.

    4. The issue of cluster sampling has been discussed repeatedly on my own site, Tim Lambert’s and Brad DeLong’s. Shannon has participated in a number of these discussions, but has not made any argument that I am aware of which would justify the statement that “a cluster sample has a very high chance of exaggerating the number of deaths”.

    5. Nevertheless, Shannon continues to link to his previous posts on the subject of cluster sampling (see the sixth paragraph of the original post).

    Shannon’s actions with respect to the “cluster sampling critique” are highly likely to mislead your readers into believing that there is any merit to this statistical argument when there is not. I keep mentioning this, but the link above remains, despite other errors of fact (and this is an error of fact) having been corrected. When someone appears to be misleading people on purpose, what am I meant to conclude? Particularly when the context is an article in which that same person is prepared to accuse other people of being propagandists for fascists, liars and scientific whores?

    If my and Palo’s (whose remarks I thought were “harsh”, not “inaccurate”) remarks about Shannon’s motives are “mean spirited”, then what would they be if instead of saying that it is intellectually dishonest to make accusations that you are not prepared to defend or abandon, we had said that the author of this post was an apologist for mass murder, a liar, and a whore?

    Your position at the moment appears to be that of a man sitting in a pigsty and complaining about the muck.

83. Off topic: No wonder Susan Estrich complains of insufficient female voices; after extremely long threads in which dsquared has squared off against Shannon, he still refers to her (as in “his previous posts”) in masculine terms. Obviously, ideas are not gendered and I prefer the general “he”, but it doesn’t increase dsquared’s credibility.

    As an observer with no statistical background, I would think that your statement:

    A cluster sample does not have a “very high chance of exaggerating the number of deaths” if deaths are rare and concentrated. It has a very low chance of exaggerating the number of deaths (by a large amount) and a very high chance of underestimating the number of deaths

    would indicate that such a project needs a high level of honesty (both with us and I suspect with themselves – always the more difficult task) in both those who set up the study and who developed the data.

84. dsquared [cough, HACK, cough], what you lack in substance you make up for in volume.

85. It is of course downright stupid to think that Shannon Love’s genitalia have anything to do with either standard deviations or deaths in Iraq, but actually, you’ll see that I used all manner of paraphresis to avoid making a gendered reference until Matt Weiner, on my own blog, told me that Shannon was male. Since the name is ambiguous, cursory checks around Chicago Boyz didn’t turn up any clues, and since I’m not a webstalker, I thought what the hell and took Matt’s word for it. If I’m wrong, I obviously apologise. Because that’s what I do when it appears I made a mistake. If you see me in future writing things like “Shannon Love is, of course, a man”, and linking them to this thread, you have my permission to criticise that. I think I may have used “she” in previous discussions with Shannon, for what it’s worth, because I think of it more as a female name than a male one. Luckily, however, there is no gender-based statistical convention that it is OK for women to make misleading statements about cluster sampling, so nothing of consequence turns on this.

    On your substantive question, Ginny, could I ask what point you are trying to make? Any study needs honesty, clustered or not. By “honesty”, I mean the kind of good character and regard for the truth that stops people from saying things like “a cluster sample has a very high chance of exaggerating the number of deaths” when it’s not true. Are you trying to say that, in fact, I am wrong on the mathematics and a cluster sample does indeed have a very high chance of exaggerating the number of deaths? Or are you trying to imply without saying that the Lancet team might have had a cluster study which exaggerated the number of deaths because they are dishonest? If it’s the second, then that’s not a very honest thing to do. Could I respectfully suggest that, as an observer with no statistical background, all you are ever likely to contribute to this debate will be evidence-free accusations, and that as a result, you would be better off not bothering?

86. 3. After considerable discussion in the comments threads on the various Lancet posts, we reached a point at which everyone left in the debate had understood point 2 above.

    That’s your interpretation. My interpretation is that most of the people debating you gave up because it was obvious that, no matter what they argued, you would defend the Lancet study’s methodology and that you have a low threshold for impugning their motives. It’s as though you believed you could win the argument by attrition.

    By emphasizing analytical arcana you have avoided the central issue, which is data quality. Will you agree that it’s possible for a data sample to be of such poor quality as to be useless no matter how skillfully it is analyzed? I submit (again) that the Lancet study is either clearly based on such a sample or that there is substantial justified uncertainty about whether the sample is adequate for the uses to which it is being put.

    So why the vehement defense of what is at best marginal-quality data? Why not accept that the study is inconclusive and should be redone? Conditions in Iraq are gradually improving, so it should become easier with time to do high-quality surveys.

87. Jonathan,

    I call dishonesty when I see it, without any mean spirit. Quite the contrary, to call on a dishonest argument actually makes me feel happy.
    Dishonesty is to keep throwing the same arguments (in a mean spirited way, you would say) without any acknowledgement of the the very strong criticism your arguments received, even when you definitely know they exist. That is dishonest.
    Dishonesty is going around the forum on comments to your own article and choosing to answer only the easy stuff, once again ignoring the people that expose the weakness of your argument.
    Dishonesty is repeating over and over and over your partisan criticism of the Lancet study claiming to be fool-proof but not to have the guts to send your paper to a peer-review journal, probably to avoid, once again, the exposing of your weak argument.

    I don’t know Shannon Love. I have nothing to say about her other that she advances dishonest arguments. But I did suggest the unsupported idea that her only goal is to feed the underlings some talking points. That I regret, even when I have the feeling you can only prove me right.

88. Palo, your point is taken, though I am confident that you are wrong about Shannon’s motives. Perhaps even people you disagree with are arguing from a sincere desire to debunk what they see as faulty arguments.

89. Jonathan,
    I’m all in favor of respecting disagreement. Not in respecting dishonesty. I’m sure you’ll respect my belief that Shannon Love’s postings are dishonest, and that they only serve to provide statistically ignorant people tools to hang on to the propaganda that the Lancet study is flawed and its authors crooks.
    In fact, no matter how many times dsquared has shown that the statistical analysis by Love is hopelessly wrong, you keep arguing about your right to believe her argument. Fine. Do so. It doesn’t say anything about the study, but a lot about you.

    PS: nice way to lecture me on the right to disagree after calling me an idiot!

90. Patrick. You still miss the point, and you bring up another which is also incorrect.
    The point you still miss is that with dumb bombs, you have to drop a lot of them to make sure you get at least one on target. The point is not whether one goes astray. The point is what about all those others. One smart bomb going astray may be a problem. Twenty dropped just in case is a bigger problem. And most smart munitions hit their targets. With smart bombs you drop one or two. If you hit, end of problem. No more bombs are necessary. With dumb bombs, you drop ten or twenty or whatever, and probably hit.
    Munitions left over from dumb bomb days were difficult to smart up by providing seeker heads and guidance fins because they had been made with their center of mass slightly off from the longitudinal line running from the nose back to the tail in order to “spread” a salvo. No more, except by design in the CBUs. The point is, there was a need to spread the things out. Hitting with one was good enough, if you could do it, but missing with all was a wasted trip. So they figured by getting a few mils’ spread, one of the things would accidentally make up for the errors inevitable in the bomb run, and the metereology, and so forth. The others…? What do you think they did?
    I inadvertently gave you an opportunity to misrepresent the case when I mentioned dumb pilots and you ran with that. Well-trained pilots–which is what we have–and smart munitions do excellent work and in the process reduce civilian casualties.
    As to target identification, that has been a problem since the invention of the thrown rock. Stonewall Jackson was killed by his own troops. For example.
    Two days running, the air prep for the breakout from Normandy hit US troops.
    This will never go away. The training to recognize ours vs. theirs is actually pretty good. The problem is that one frequently opens fire from a distance at which distinctions cannot be made regardless of recognition training. The villain in almost all cases is getting locations wrong. Those indistinct vehicles at that location must be enemy. If they’re ours, they forgot to tell somebody where they are, or got the coordinates wrong, or somebody up the chain dropped the ball. Or the shooter mistakes where he is, thinking everybody in front of him is the enemy, but he’s not as far forward as he thinks. Or somebody else is farther forward than he ought to be and doesn’t know it.
    The Marines,whose air is primarily a ground-support arm, both to motivate their pilots and to avoid such things, insist that fighter pilots complete their Infantry Officer’s Basic Course.
    In any event, these issues are irrelevant to the question of civilian casualties, which was the original question.

91. Jonathan,

    What grounds have you for believing that the Lancet study is based on a data sample “of such poor quality as to be useless no matter how skillfully it is analyzed”?

92. Jonathan, I can only argue with what is actually put in front of me. A one-handed boxer could count the number of people who have bothered to make specific comments about the data-gathering process (he would use the thumb for Mike Harwood and the mitt for Heiko Gerhauser).

    Shannon, on the other hand, has titled posts “Bogus Lancet Study” and “A Lie in A Labcoat”. The “cluster sampling critique” is nothing to do with data quality; it’s a (incorrect) statement that the authors chose a flawed methodology. Nor is this “IFC/EFC” rubbish anything to do with data quality, and it comes wrapped up in a disgusting accusation that the authors intentionally misrepresented their results in order to provide propaganda for fascists.

    I don’t, as it happens, have a particularly high threshold for impugning people’s motives; a glance at this thread reveals that I haven’t done it to Heiko, Amac or Mike, and I’ve actually left off arguing with Aaron rather than allow the temperature to rise. I haven’t impugned your motives, either. In general, if you don’t start accusing people of lying, then I’m unlikely to bring it on. If, on the other hand, someone accuses people of “whoring out science”, then their own motives have to be fair game. Nor do I ever give out what I can’t take.

93. “Any study needs honesty, clustered or not.”

    Which is exactly contrary to how the authors have characterized this study to the media. A fact that you conviently ignore in your laughable defense of this study.

    Reason enough to abandon any discussion with dsquared is one visit to his weblog.

    http://d-squareddigest.blogspot.com/

94. “Making conservative assumptions, we think that about 100,000 excess deaths, or more have happened since the 2003 invasion of Iraq. Violence accounted for most of the excess deaths and air strikes from coalition forces accounted for most violent deaths.” – researchers from Johns Hopkins’ Bloomberg School of Public Health in Baltimore, Maryland, said in a report published online by The Lancet medical journal.

    Sounds like an honest characterization to me. Even if they came forward an corrected the record at this point, this quote by the authors has been played several thousand times by the Arab media.

95. Cog,

    You say the authors have characterised the study dishonestly, then you say their characterisation sounds honest to you. Are there two Cogs debating the issue here?

96. Palo,

    If you reread my comment you will see that I did not call you an idiot. But you seem to jump to conclusions in this area too.

97. “then you say their characterisation sounds honest to you”.

    Sarcasm?

98. “Making conservative assumptions, we think that about 100,000 excess deaths, or more have happened since the 2003 invasion of Iraq.”

    Is it me, or are the “conservative assumptions” of 100,000 excess deaths neither conservative or assumptions as reported several thousand times by the MSM and Arab press?

    How many people who reference the 100,000 figure mention it is an average based on sampling? And the authors promoted this mischaracterization by saying their findings were conservative, when they obviously were not.

    The sad fact is, this study most assuredly lead to more excess deaths. And if you want to see terrorists mention the Lancet study, visit memritv.org. I am sure they are aware of this study’s limitations, that the number cited was only an average between 8-190,000, and that the results were preliminary.

    Oops, that damn sarcasm again.

99. “Sarcasm?” Possibly. The effects produced by a real master of rhetoric can be so subtle as to defy critical analysis. Others would be well advised to just say what is on their minds.

    The 98,000 excess deaths figure is conservative in the sense that it excludes Fallujah and the authors do not mischaracterise their results by saying so. It is true that many reports of the Lancet study were misleading, but that goes both ways. Some presented 100,000 as a minimum figure, others suggested 8,000 was equally likely – both claims are false.

    So far as the merits of the study itself are concerned it is beside the point.

100. The sad fact is, this study most assuredly lead to more excess deaths.

     Is this based on actual data or is it blind, indecent political propaganda?

101. “It is true that many reports of the Lancet study were misleading, but that goes both ways. Some presented 100,000 as a minimum figure, others suggested 8,000 was equally likely – both claims are false.”

     Really? The first mainstream news source or Arab news source I saw reference the 8000 figure was the last article in Slate, when they mentioned this was an average.

     I guarantee you this was not reported as such by the Arab media en masse. And while many in the MSM now consider this study discredited, thankfully, there are still a few hangers on.

     “So far as the merits of the study itself are concerned it is beside the point.”

     Yes, completely beside the point. In a study of excess deaths, a study used by terrorists to recruit and finance attacks which lead to excess civilian deaths is beside the point. The “proven” 100,000 figure has shown several dozen of times on Al Jazeera, and twice on terrorist videos that I have watched. It has also been reported incessantly in Turkey, almost as a mantra against the US forces.

     And the authors of this study have made it clear that this study is an average of 8,000 and 194,000? Time to enter the real world.

102. “And the authors of this study have made it clear that this study is an average of 8,000 and 194,000?”

     Of course not, since it isn’t. The mid-point of a confidence interval is not obtained by averaging the two end-points. Cog, since you speak of the real world, you must know that real-world statisticians calculate the CI from the estimate, not the other way round. Pretending otherwise is just dragging down the level of the debate, which is about whether a paper is up to scratch or whether it is “a lie in a lab coat” as our host claims, but fails to demonstrate.

     If all you are saying is that the press reported it badly all I can say is, what else is new?

103. “oh. My. God.”

104. Richard:

     I agree with the content of your entire last post, but for the last comment – that this is irrelevant to civilian casualties. We both agree that hitting a target is good, and smart weapons do a good job of this, and this probably helps reduce civilian casualties. The question is how does one identify that a potential target is a good one to hit? How precisely does a pilot travelling at signifigant speed distinguish between an insurgent and a civilian?

105. Warm welcome to Alnemat TheGrace Arabic Christian Internet Magazine, We love you! Please visit us at:

     http://www.TheGrace.com

     http://www.TheGrace.net

     http://www.TheGrace.org
     سلام لكم في محبة الله.نتأمل زياراتكم الكريمة لموقع النعمة موقع مجلة النعمة يقدم كلمة الله الكتاب المقدس الإنجيل رسالة السيد يسوع المسيح قراءات مختارة مواضيع مصيرية قصص واقعية شهادات شخصية ترانيم ممتازة ردود مؤكدة كتب بنّاءة رسوم تسالي تأملات يوميات
     Bible Read search in Arabic Studys Stories Testimonies Acappella Hymns and Poems Answers Books Links Daily devotions Acappella Music Graphics /Alnemat Journal Arabe Chrétien La Grâce la Revue Arabe sur Internet offre La Sainte Bible Al-Injil L’Evangile de Jésus Christ gratuit, Bienvenue a La Grâce.

Comments are closed.