Wow – what a terrible review of that study – let’s fact check it:
> By serendipity via link on MyDD, a Democratic Partyesque blog,
> http://www.chicagoboyz.net/archives/002543.html
> October 29, 2004
> Bogus Lancet Study
>
> Via The Command Post comes this study published in Lancet (free reg)
> which purports that 100,000 Iraqi have died from violence, most of it
> caused by Coalition air strikes, since the invasion of Iraq.

Wrong – The study calculates excess deaths FROM ALL CAUSES postwar as
opposed to prewar. The 95% Confidence Intervals is wide, from 8,000
to 194,000. Luckily we are starting to get larger sample studies with
smaller CIs, such as Iraq Living Quality Study – which largely
vindicate the lancet study.

Needless
> to say, this study will become an article of faith in certain circles
> but the study is obviously bogus on its face.
>
> First, even without reading the study, alarm bells should go off. The
> study purports to show civilian casualties 5 to 6 times higher than
> any other reputable source. Most other sources put total combined
> civilian and military deaths from all causes at between 15,000 to
> 20,000. The Lancet study is a degree of magnitude higher. Why the
> difference?
>

What source is this? If its from Iraq body count (that has numbers in
that range) then that would be a study of violent deaths – not deaths
from all sources. The Iraq Living Quality Survey
http://www.iq.undp.org/ILCS/overview.htm has recently been touted as
disproving the Lancet study because it only shows 24,000 violent
deaths from warfare. The Lancet study number of deaths from war
violence is 33,000 – however the lancet study covered 18 months after
the invasion while the ILQS covered only one year after the war – the
month average death by war is similar but slightly higher the much
larger ILQS study.

> Moreover, just rough calculations should call the figure into doubt.
> 100,000 deaths over roughly a year and a half equates to 183 deaths
> per day. Seen anything like that on the news? With that many people
> dying from air strikes every day we would expect to have at least one
> or two incidents where several hundred or even thousands of people
> died. Heard of anything like that? In fact, heard of any air strikes
> at all where more than a couple of dozen people died total?
>

Again – the Lancet study covers all causes of deaths not just war
related violence – Comparing the the lancet study numbers to Iraq Body
Count one can have a rough estimate that roughly one out of three
deaths are not reported in the media – a not so unlikely proposition.

> Where did this suspicious number come from? Bad methodology.
>
>From the leading figures in the field of epidemiology. For example the
Chronicle of higher education
(http://chronicle.com/free/2005/01/2005012701n.htm) notes

“Les has used, and consistently uses, the best possible methodology,”
says Bradley A. Woodruff, a medical epidemiologist at the U.S. Centers
for Disease Control and Prevention.

Indeed, the United Nations and the State Department have cited
mortality numbers compiled by Mr. Roberts on previous conflicts as
fact — and have acted on those results.

> >From the summary:
>
> Mistake One:
>
> “A cluster sample survey was undertaken throughout Iraq during September, 2004″
>
> It is bad practice to use a cluster sample for a distribution known to
> be highly asymmetrical. Since all sources agree that violence in Iraq
> is highly geographically concentrated, this means a cluster sample has
> a very high chance of exaggerating the number of deaths. If one or two
> of your clusters just happen to fall in a contended area it will skew
> everything. n fact, the study inadvertently suggests that this
> happened when it points out later that:
>

> “Violent deaths were widespread, reported in 15 of 33 clusters…”
>
> In fact, this suggest that violent deaths were not “widespread” as 18
> of the 33 clusters reported zero deaths. if 54% of the clusters had no
> deaths then all the other deaths occurred in 46% of the clusters. If
> the deaths in those clusters followed a standard distribution most of
> the deaths would have occurred in less than 15% of the total clusters.
>
> And bingo we see that:
>
> “Two-thirds of all violent deaths were reported in one cluster in the
> city of Falluja”
Falluja was considered an outlier and was exculded when deriving the
conclusions had they included it the final result would have been
higher. the Economist on this issue:
“The Fallujah data-point highlights how the variable distribution of
deaths in a war can make it difficult to make estimates. But Scott
Zeger, the head of the department of biostatistics at Johns Hopkins,
who performed the statistical analysis in the study, points out that
clustered sampling is the rule rather than the exception in
public-health studies, and that the patterns of deaths caused by
epidemics are also very variable by location.”
http://www.economist.com/science/displayStory.cfm?story_id=3352814

>
> (They also used a secondary grouping system (page 2, paragraph 3) that
> would cause further skewing.)
>
> Mistake Two:
>
> “33 clusters of 30 households each were interviewed about household
> composition, births, and deaths since January, 2002.”
>
> Self-reporting in third-world countries is notoriously unreliable. In
> the guts of the paper (page 3, paragraph 2) they say they tried to get
> death certificates for at least two deaths for each cluster but they
> never say how many of the deaths, if any, they actually verified. It
> is probable that many of the deaths, especially the oddly high number
> of a deaths of children by violence, never actually occurred.
They state that they got 81% of the death certificates they asked for,
those who didn’t have them had believable reasons why, and the
interviewers were asked to judge if they thought the person was lying
- which they didn’t find any.

>
> So we have a sampling method that fails for diverse distributions, at
> least one tremendously skewed cluster and unverified reports of
> deaths.
>
> Looking at the raw data they provide doesn’t inspire any confidence
> whatsoever. Table 2 (page 4) shows the actual number of deaths
> reported. The study recorded 142 post-invasion deaths total with with
> 73 (51%) due to violence. Of those 73 deaths from violence, 52
> occurred in Falluja. That means that all the other 21 deaths occurred
> in one of the 14 clusters where somebody died, or 1.5 deaths per
> cluster. Given what we know of the actual combat I am betting that
> most of the deaths occurred in three or four clusters and the rest had
> 1 death each. Given the low numbers of samples, one or two fabricated
> reports of deaths could seriously warp the entire study.
>
> At the very end of the paper (page 7, paragraph 1) they concede that:
>
> “We suspect that a random sample of 33 Iraqi locations is likely to
> encounter one or a couple of particularly devastated areas.
> Nonetheless, since 52 of 73 (71%) violent deaths and 53 of 142 (37%)
> deaths during the conflict occurred in one cluster, it is possible
> that by extraordinary chance, the survey mortality estimate has been
> skewed upward. ”

How many times do we need to remind that Falluja was an excluded outlier!

>
> Gee, you think? It’s almost as if military violence is not randomly
> distributed across the population of Iraq but is instead intelligently
> directed at specific areas, rendering a statistical extrapolation of
> deaths totally useless.
>
> In the next paragraph they admit:
>
> “Removing half the increase in infant deaths and the Falluja data
> still produces a 37% increase in estimated mortality.”
>

Again! How many times do we need to remind that Falluja was an excluded outlier!

In short this review cherry picks quotes to distort what the lancet
study said. Of course the wide confidence Interval makes larger
studies a neccessity, but lets not shoot the messenger.

–
Andy Barenberg

PS. More indepth discussion of the lancet study here:
http://cgi.cse.unsw.edu.au/~lambert/cgi-bin/blog/science/LancetIraq/

___________________________________
http://mailman.lbo-talk.org/mailman/listinfo/lbo-talk

–
Michael Pugliese

Having a number gut is really important, but it’s also part of general range number “truth”. It’s interesting to look at the Japan-China flap over the Rape of Nanking. In the newer nationalist book, the number of casualties is “disputed”, and no range is given. 300 000? 500 000? 50 000? The fact that we will never have calculator 9 digit accuracy for war deaths means all such numbers are estimates. The 6 million/10 million Jews/all murdered in Nazi death camps is a similar issue — without 11 digit accuracy, it’s ‘only an estimate’.

Most human truths about mass events are only estimates; even the 2500-3000 killed in the WTC. The media has failed to emphasize and explain this fact so that more folk understand.

(When I read that the Lancet 100 k estimate included a range of between 8 000 and 200 000, it was clearly of little value to me. Just another stick for the Bush-haters.)

Heinlein and his wife did their calculations based on observation— i.e. ‘we went to these cities, and saw and talked to this many people, and based on the physical size of the place, with the density we observed, the city has this many people.’ But the reported number of people was five to ten times that— and they knew that couldn’t be right.

So Heinlein talked to a friend of his, a military strategist, and asked him the population of Moscow. The friend thought a minute and came up with a number very like Heinlein’s estimate. When asked, he said he’d envisioned the physical map of the place, and from a knowledge of infrastructure such as roads and sewers, determined that the city could withstand no more than the number he quoted before it started breaking down. Which only reinforced the idea that the atlases and other world information sources were using bogus information, given out by the USSR.

(One can, of course, figure out many reasons for such a misleading set of figures.)

Of course, now I have to wonder how they went about correcting those figures, and how much of a problem it was when the correction was made. (“The former Soviet Republic lost over half its population last year?”)

The Lancet study is the equivalent of 500 Mai Lais over two years or almost two every three days.

Great analysis, both here and previous posts.

Lex:

Has anyone created a gallery of pictures of post-war Tokyo or Berlin side-by-side with post-war Baghdad?

Would this ratio come close to believability?

I wouldn’t know where to go to locate the military’s numbers on this, but we heard again and again during the active ground campaign how much ordinance was used, and how often our aircraft were coming back with almost full compliments of missiles and bombs because of the lack of targets.

Enrico Fermi was famous for being able to estimate the neutron capture probability of newly discovered isotopes based on the skimpiest of data. He estimated the mega-tonnage of the Trinity nuclear explosions by scattering bits of tissue paper in the air at the “flash” and measuring how far they were carried by the power of the later-arriving “bang”. And he demanded his students practice problems like piano-tuner estimations.

You don’t know the population of Chicago, but you know it’s smaller than New York and you know THAT. You don’t know the percentage of families with pianos — but take a guess. Less than half, more than one percent, somewhere. You don’t know how often a piano has to be tuned — take another guess: less than once a week, surely? How many pianos can a piano tuner tune if the tuner turns tunes every day? Etc. The Fermi notion was that guesses tended to be just as likely high as low, and offset.

Another principle of the Fermi problem was that the math should never be better than the data. Guessing that the population of Chicago was 3 million, and average family size was 4 — 3 million people divided by 4 people per family equals about 1 million families. If about 3 percent own pianos then 1 million times .03 is — call it 36500 pianos. If one piano tuner tunes one a day for a year it would take him 100 years to tune each. If a piano holds a tune for, what? two years each and all tuners stay busy then there must be about 50 tuners in Chicago.

Doing Fermi problems is like sit-ups — builds a strong gut.

One curious tool for such estimation:

The number of seconds in a year is within about 1/2% of pi x 10E7.

Link roundup of good stuff.

‘Number gut’ is why in the late 1980s my math teacher made us first use sliderules and then, later, calculators.

I can certainly tell the difference with folks now who haven’t ever done lots of calculation by hand. Oh and of course before we were let anywhere near the sliderules, we had to first learn how to do everything totally by hand with log tables and interpolation (of several kinds!)

THEN we could use sliderules (the one I have now is a nice circular one), then calculators, and then computer software.

Did you know that it takes about 6 hours to calculate and plot a natal astrological chart using only a log table, an ephemeris, a pencil and a straightedge? Less than a second on even ‘slow’ computers, and the output is nicer, and the math is more accurate. All of which is now starting to get harnessed, to the tune of several percent productivity increase a year now that we have all these computers talking to each other.

Oy, can you tell I’m a math major?

James