Lancet Letters

Although sampling of 988 households randomly selected from a list of all households in a country would be routinely acceptable for a survey, this was far from the method actually used–a point basically lost in the news releases such a report inevitably engenders. The survey actually only included 33 randomised selections, with 30 households interviewed surrounding each selected cluster point. Again, this technique would be adequate for rough estimates of variables expected to be fairly homogeneous within a geographic region, such as political opinion or even natural mortality, but it is wholly inadequate for variables (such as violent death) that can be expected to show extreme local variation within each geographic region. In such a situation, multiple random sample points are required within each geographic region, not one per 739 000 individuals.

In my opinion, such a flaw by itself is fatal, and should have precluded publication in a peer-reviewed journal. However, the authors’ sampling technique is also questionable in other ways. When a town or village was selected from the “cumulative population lists for the Governorate”, the survey team then “drove to the edges of the area and stored the site coordinates”. In personally deciding what constituted the edges of the grid rectangle for “the area”, the team leader could potentially introduce bias into the selecting of an area to sample. In any case, it seems quite likely that the grid rectangles created by driving around in a war zone were much smaller than the original census tracts used in the “cumulative population lists”. Additionally, there is no way to verify that the 30 closest households were those actually interviewed. It is always easier to interview the more vocal households, and there is some leeway in deciding which doorways are closest. The claimed 99·5% response rate makes it seem highly suspect that the 30 closest doorways were actually those rigorously selected.

Lastly, when interviewing a “household” about violent deaths, one is likely to hear reports included from an extended family unit. Individuals might be included from a network of family by marriage, including many dozens if not hundreds of individuals. Although such individuals might have even lived in the household at some point, that does not mean they would have actually been living at that location at the time of the survey if they had not been killed. Such a phenomenon is much more likely when reporting violent deaths (due to the extreme emotional import of such occurrences) than when reporting natural deaths.

I think such considerations should have been uncovered during a thorough review by statistical experts (as I assume occurred before electronic publication of this article immediately before a US presidential election). If not precluding publication altogether, these fairly obvious points should have been included in a fair editorial analysis accompanying the article.

I declare that I have no conflict of interest.

Stephen Apfelroth

Department of Pathology, Albert Einstein College of Medicine, Jacobi Hospital 1E12, 1400 Pelham Parkway, Bronx, NY 10461-1101, USA

1 Roberts L, Lafta R, Garfield R, Khudhairi J, Burnham G. Mortality before and after the 2003 invasion of Iraq: cluster sample survey.  Lancet 2004; 364: 1857-64. [Text]

Authors’ reply

The ability of a 33-neighbourhood sample to portray adequately the mortality experience of an entire country has seemed problematic to many critics of our study in Iraq. However, most mortality surveys in war zones only contain 30 clusters, as do the recommended approaches of United Nations agencies and the US Agency for International Development.1 Ample evidence suggests that 30 locations are reasonably adequate for measuring the level of malnutrition or immunisation coverage in an entire country.2

Unfortunately, as Stephen Apfelroth rightly points out, our study and a similar one in Kosovo,3 suggest that in settings where most deaths are from bombing-type events, the standard 30-cluster approach might not produce a high level of precision in the death toll. But the key public-health findings of this study are robust despite this imprecision. These findings include: a higher death rate after the invasion; a 58-fold increase in death from violence, making it the main cause of death; and most violent deaths being caused by air-strikes from Coalition Forces. Whether the true death toll is 90 000 or 150 000, these three findings give ample guidance towards understanding what must happen to reduce civilian deaths.

All sampling methods include the potential for bias. We feel that sampling from a random point was less susceptible to bias than any other method practical in this setting. In the one wartime setting where the usual “spinning the bottle in the centre of a village” approach was compared to our random point selection approach, the two surveys produced nearly identical results.4 As we noted in the paper, by storing the randomly picked point in a global positioning system (GPS) unit and visiting the nearest 30 households as defined by the GPS, there was little subjectivity in the choice of households.

We also stated in the paper that people had to have been sleeping under the same roof with a family for 2 months before their death to be considered a household death. This strict definition of household member may have prevented the recoding of some deaths, particularly among former military members who did not live with any household in the weeks before their death, but it ensured that the type of overestimation that concerns Apfelroth did not occur.

Before publication, the article was critically reviewed by many leading authorities in statistics and public health and their suggestions were incorporated into the paper. The death toll estimated by our study is indeed imprecise, and those interested in international law and historical records should not be content with our study. We encourage Apfelroth and others to improve on our efforts. In the interim, we feel this study, as well as the only other published sample survey we know of on the subject,5 point to violence from the Coalition Forces as the main cause of death and remind us that the number of Iraqi deaths is certainly many times higher than reported by passive surveillance methods or in press accounts.

We declare that we have no conflict of interest.

*Les Roberts, Gilbert Burnham, Richard Garfield

les@a-znet.com

*Center for International Emergency Disaster and Refugee Studies, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21287, USA (LR, GB); and School of Nursing, Columbia University, New York, NY, USA (RG)

1 Standardized Monitoring and Assessment of Relief and Transitions (SMART). http:// www.smartindicators.org (accessed Jan 29, 2005).

2 Binkin N. Rapid nutrition surveys: how many clusters are enough?  Disasters 1991; 16: 97-103. [PubMed]

3 Spiegel PB, Salama P. War and mortality in Kosovo, 1998-99: an epidemiological testimony.  Lancet 2000; 355: 2204-09. [Text]

4 Roberts L, Hale C, Belyakdoumi F, et al. Mortality in eastern Democratic Republic of Congo: results from eleven mortality surveys. New York: IRC, 2001. http://intranet.theirc. org/docs/mortII_report.pdf (accessed Jan 29, 2005).

5 Hoge CW, Castro CA, Messer SC, et al. Combat duty in Iraq and Afghanistan, mental health problems and barriers to care.  N Engl J Med 2004; 351: 13-22. [PubMed]

Les Roberts and collaborators1 compared mortality figures before and after the invasion of Iraq in 2003. A 2·5-fold increase in the risk of death, mainly due to the extremely violent situation in Fallujah, was revealed. Even after the Fallujah outlier was removed from the analyses and the measured increase in infant mortality halved (to account for a possible, though unlikely, recall bias regarding infant deaths), the risk of violent death still soared by 37% after the war set off. When all data were analysed, the increase reached 58%. Coalition Forces may have targeted suspect fighters, but civilians were most frequently killed. The figure for the whole country (100 000 excess deaths since the coalition invaded Iraq) is regarded by Roberts and colleagues as an underestimate.

In January, 2003, two open letters to the Prime Minister of the UK were published in leading medical journals.2,3 More than 500 signatories from the London School of Hygiene and Tropical Medicine (LSHTM) warned about the public-health implications of war on Iraq. Based on estimates anticipating 50 000 to 250 000 deaths, they expressed their opposition to military intervention. Their views can now be contrasted with actual facts. The UN and WHO were sadly right in their warnings, and the political leaders of the Coalition Forces wrong in their choice of military action. The claimed legality of their decision is at least dubious; its legitimacy is implausible.

The effects of the invasion on public health and humanitarian aspects other than direct mortality figures await scientific scrutiny. Reports about the physical and psychological suffering of other victims (the injured; the refugees; a deprived population enduring shortages of food, safe water, medical care and supplies, sanitation, shelter, personal safety, and mental-emotional stability; the soldiers; the tortured and humiliated prisoners; those kidnapped by terrorists–in summary, all those whose rights are violated plus their families) will appear in due course. They should be compulsory reading for all those who keep promoting collective violence.

Finally, Roberts and colleagues highlight how their results were achieved with modest funding and lots of nerve and commitment, making a brilliant case against those who (clearly trying to avoid accountability) hide behind claims that valid mortality data cannot be obtained in war environments.

I declare that I have no conflict of interest.

Fernando Abad-Franch

fernando.abad-franch@lshtm.ac.uk

*London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK; and Fiocruz, Manaus, Brazil

1 Roberts L, Lafta R, Garfield R, Khudairi J, Burnham G. Mortality before and after the 2003 invasion of Iraq: cluster sample survey.  Lancet 2004; 364: 1857-64. [Text]

2 Stephens C, on behalf of 500 signatories of the London School of Hygiene and Tropical Medicine. Open letter to the Right Honourable Tony Blair, Prime Minister of the UK: public health and humanitarian effects of war on Iraq. Lancet 2003; 361: 345.