A fabulous post by Marko Milanovic on the blog of the European Journal of International Law talks about many of the issues I raised in my own “Narrative in the Israeli/Palestinian Conflict” post a few years back. I highly recommend it — as well as t…

Thanks for your thoughtful reply and for raising these important issues. A few points in response:

1. On re-reading our exchange, it strikes me that we may have been talking past each other, insofar as I did not address the importance of the context in which empirical claims are received and interpreted. It seems fair to say that the relative weight given to certain claims in the face of contradictory evidence varies incredibly depending on other contextual factors (political, historical, etc.): e.g., the inclination of some people in Serbia to believe that a massacre of Serbs occurred at Srebrenica, or the inclination of many Americans to believe that Saddam Husayn possessed WMD.

2. That being said, and in contexts where people do not feel some kind of personal or atavistic stake (as with many though by no means all readers of western newspapers re Israel/Palestine), I stand by the point that falsifiability and transparency should count for something. I agree that one can create a misleading impression of rigorous method and of course do not expect everyone to personally scrutinize the raw data used by HR NGOs. But the _availability_ of raw data and methods, in contrast with what governments or other actors (in this particular case, Israel) provide to support their claims should and I think in some cases does factor into people’s credibility judgments.

And more importantly, it should certainly factor into how journalists treat such claims — methodology is important precisely because the relationship between NGO claims and the newspaper reader is mediated by others in the public sphere who can and do cross-check them (though I admit this point is more relevant when it comes to case studies than statistical tallies). In the example that you cite — the ‘massacre’ of Serbs at Srebrenica — I’m sure the claims were also subjected to independent scrutiny by someone with more time/resources than you and me and treated accordingly, even if their intended audience was uninterested in potential refutations (cf. point above about context).

The limitation (and perhaps you were alluding to this, I wasn’t entirely clear) of the approach you outline — ask who has reason to lie — is that it is always possible to imagine cynical motives, as frequently occurs: HR groups allegedly lie because of their political agendas, hatred of people x, or desire to inflate casualty numbers for attention and funding. Given the imperfections of both approaches, I would rather combine them rather than go only with one.

3. On ad hominem attacks, I agree that human rights organizations must hold themselves to very high standards and that such attacks can work. I also agree that your approach — ask who has reason to lie — is a useful starting point here (though by no means does not exhaust the well of refutations available), as NGO Monitor is run by a consultant to the Israeli government.

Best,
Darryl

Many thanks for your comment. On the specifics of the Israeli/Palestinian conflict, I agree with much of what you have said. In principle, I also agree with your conclusion that one should weigh competing claims to the best of one’s ability. But the limits of that ability are quite, quite narrow, even for those with a modicum of expertise in the field.

So, I personally would venture into having an opinion on some of the legal issues, which you quite correctly say are interlinked with the factual issues, e.g. is a Hamas policeman a civilian. But I am at a loss beyond that. You thus say that the B’tselem and the PCHR reports are more trustworthy than the Israeli gov’t one because (1) they give a list of names, and (2) they use a good methodology.

As for (1), the experience of the Yugoslav wars amply shows just how easy it is for ‘experts’ of one ideological stripe or another to produce a list of names. You wouldn’t believe the number of such lists circulating, e.g. about Serbs killed in Srebrenica, all supposedly based on verifiable information. As for (2), do you honestly believe that a regular person reading the news etc is capable of assessing the merits or the rigor of such a methodology? That this reader has both the necessary expertise, access to the necessary information, and above all the TIME required to properly assess a methodology?

I freely admit that I do not. I trust the B’tselem casualty count more than the Israeli gov’t account (and I do; I did not say that I am equally skeptical towards all sources in my original post) not because of its better methodology, but because I choose to trust a human rights NGO over a state which has every reason to cover up its misdeeds. And, frankly, I think that this goes for most people. But I am aware of the limits of placing my trust in that way.

To again briefly refer to the ongoing ad hominem attacks against you personally and other HRW staffers, as in the recent NGO Monitor report – these attacks can work and should be taken seriously precisely because HRW and other human rights NGO operate vis-a-vis the general public on the basis of trust and faith. In other words, just like Caesar’s wife, you must strive to be above all suspicion. If you lose your reputation and credibility, you’ve lost everything, better methodologies notwithstanding.

(Apologies for being preachy. I usually am not.)

I would reframe the issue slightly: the overall number of deaths is actually not particularly controversial. The ratio of civilians is, but that’s also not as straightforwardly empirical as you suggest, since it involves questions about legal standards that you and others have explored extensively on this blog.

But more importantly: I agree that we should always be careful and honest about one’s biases and their relationship with the conclusions (political, otherwise) that are drawn from factual claims. But I fear your post goes too far in implying that even competing factual claims should be treated equally, without considering their merits.

As a matter of disclosure, I have worked in Israel/Palestine for Human Rights Watch, B’tselem, and the Palestinian Centre for Human Rights. I feel I have a pretty good sense of their methodologies and the pressures under which they operate, though obviously I write here solely in my personal capacity.

1. Methodology

Israel’s claims about casualties in the January war are unverifiable because they haven’t released a list of names and have not discussed their methodology in any detail. In contrast, the human rights groups use methodology that is comparatively much more transparent and, importantly, falsifiable, allowing people to independently check those claims (which a cottage industry of pro-Israel bloggers zealously do, with largely unimpressive results). In this case, B’tselem has sent its entire list to the IDF and used birth certificates, hospital records, etc. (PCHR also published its list several months back).

Without getting into the limits of Popper’s arguments about falsifiability, as a practical matter it would seem odd to accord equal weight to a zero-transparency claim versus one that provides considerable detail on methods and results.

2. Conceptual and definitional debates

In addition, the factual disputes that _do_ exist often hinge on issues that are clearly identifiable and allow one to narrow (or at least more clearly define) the field of disagreement.

One such issue is that Israel classifies all police personnel as combatants, as well as individuals who it deems to be members of armed groups, even if their participation does not meet the ICRC “continuous function” test and they are not engaged in hostilities (in short, anyone who ever signed up for an armed group or posed for a macho photograph can be retroactively counted as a combatant). This single issue accounts for a significant portion of the discrepancy between the IDF and the human rights groups. In other words, the fact that there are often divergent interpretation of the same data does not necessarily invalidate that data itself.

[this also presents a contrast with the Yugoslav example -- in Israel/Palestine, there is generally little disagreement about the overall numbers of deaths, the dispute is about how to classify those deaths; in the former Yugoslav wars, my impression is that variance on total deaths far greater, probably for reasons of scale]

3. Knee-jerk skepticism

The danger of believing only what we are predisposed to believe is indeed great and demands constant vigilance. But we should also bear in mind that a position of knee-jerk skepticism towards all claims (“Israel/Palestine is very complicated, so all sides are equally right/wrong”) requires just as little thought as believing whatever one’s own “side” proclaims.

In the Gaza case, it is simply not true that information is “monopolized” by those with reason to lie; yes, Israel and Hamas impose restrictions, as do parties in any conflict. Yet human rights NGOs can and do cross-check each other’s work to come up with reasonably coherent narratives and analyses (hence the B’tselem report works better as evidence of a strengthening consensus rather than an example of deep epistemological uncertainty). Israel/Palestine may appear more ‘controversial’ or ‘polarizing’ than another conflicts; but the quantity of hard data about deaths is probably far greater per capita than for any other conflict in the world because of the relatively small numbers and relatively intense scrutiny from various parties.

Rather than simply acknowledging that everyone has an agenda, it would seem a more important step towards rationality would be to actually independently engage and weigh competing claims to the best of one’s ability.