The Wikipedia Approach to Reality

Author: Marko Milanovic
Filed under: EJIL Analysis

Wednesday
Sep 9,2009

I was reading the news today, and was again struck for the umpteenth time by the ease with which people slip into what I now like to call the Wikipedia approach to reality – a phenomenon that I’m sure psychologists have defined in a much more sophisticated way as some form of cognitive bias or another. Take a look at this CNN report regarding a new B’Tselem report on casualties during the Gaza conflict (on which see more from our blog here, here, here, and here):

A human rights group says more than half of the Palestinians killed during Israel’s three-week offensive against Hamas in Gaza earlier this year were civilians, contradicting an Israeli military claim.

Israel had said that more than 60 percent of those killed in the military campaign were “terror operatives.”

The Israeli human rights organization B’tselem released figures Wednesday, saying that, of the 1387 killed, 773 “did not take part in the hostilities” and, that of those, 320 minors were killed.

Israel says it launched the military operation in late December and early January in Gaza to stop Hamas and other militant groups from firing rockets into the southern parts of the country.

The accounting, which B’tselem says was based on “months of meticulous investigation and cross checks with numerous sources,” is at odds with official numbers released by Israel.

In March, the Israeli military spokesman’s office said that 1,166 Palestinians were killed in the Gaza operation, also known as Operation Cast Lead, and that 709 of them were “identified as Hamas terror operatives, amongst them several from various other terror organizations.”

The rest included 162 names that had “not yet been attributed to any organization,” according to the military. Israel said 295 Palestinians who were uninvolved in the fighting were killed, among them 89 children younger than 16.

At the time, the military said it was releasing the findings to counter false information from various Palestinian sources, and to remove doubt about the number of Palestinians killed in the operation.

B’tselem said it was it was unable to compare its list of names with that compiled by the Israeli military, which has declined to release the list of those Palestinians its says were killed in the fighting or the methodology used to compile the names.

[snip]

In a written statement, the Israeli military said the B’tselem report “is not based on fact or on accurate statistics” and that the group “does not have the tools, nor the intelligence capabilities with which it can within a necessary degree of confidence know the causes of death or the affiliations of these casualties.”

Since the end of Operation Cast Lead, the Israeli government has consistently said that it went to great lengths to minimize civilian deaths and has blamed civilian deaths on Hamas, which it alleges used Gaza’s civilians as human shields to launch attacks against Israel.

The Israel Defense Forces said in April that its forces “operated in accordance with international law” during the operation, but there were a few incidents in which “intelligence or operational errors” occurred.

According to a statement posted on IDF’s Web site in April, Israeli investigations “revealed a very small number of incidents in which intelligence or operational errors took place during the fighting. These unfortunate incidents were unavoidable and occur in all combat situations, in particular of the type which Hamas forced on the IDF, by choosing to fight from within the civilian population.”

This is but one of many examples of how competing realities can be constructed regarding any politically controversial issue. In principle, there is only one true, empirically verifiable account of how many people died during the Gaza conflict, and how many of these people were innocent civilians. But this one true reality is equally non-ascertainable. The two sides in the conflict or their immediate allies can thus easily produce their own versions of reality. Two empirically verifiable items of information – the total number of people killed, and the ratio of civilians – are easily distorted, depending on the broader narrative of each side.

For the Israeli side, the number of civilians killed, and particularly the number of children killed, the most innocent civilians of them all, is small and fits into the overall picture of the Gaza conflict as a defensive operation conducted in accordance with international law, with some possible mistakes or isolated incidents. For the Palestinian side, more civilians were killed than combatants, and this fits the broader view of the conflict as a punitive operation by Israel, a part of the broader Israeli oppression of Palestinians. As we have seen, the report by B’Tselem, an Israeli NGO, is closer to the Palestinian version.

Now, it is perfectly possible for one to believe as a general matter that Israel fights the good fight against Palestinian terrorism, but that nonetheless the Gaza operation had numerous instances of excessive use of force. Or, one could also believe that Palestinians are as a general matter oppressed by Israel, but that this particular operation in Gaza was as a whole conducted in accordance with the rules of the law of armed conflict. But very few people will take either route. Rather, most people who individually care about the Israeli-Palestinian conflict have for one reason or another already chosen ‘their’ side to the conflict, and their consequent bias will lead them to accept the factual account which is closer to their own preconceptions. Thus, those who believe in the ‘right of Israel to defend itself’ will also believe that the Gaza operation was entirely kosher, while those who think that the Israeli occupation of Palestine is a form of apartheid etc. will also believe that the whole point of the Gaza operation was to brutalize Palestinian civilians.

This is of course nothing new. My only point is that this mental process has nothing to do with knowing the facts, but with believing the facts as presented by others. Indeed, it is essentially not different from believing in Jesus, Buddha, or what have you. The overwhelming majority of us were actually not there in Gaza to see what was going on with our own eyes, nor have we counted the bodies. Because we do not really know what went on, we have to believe somebody who (supposedly) does, yet at the same time we as a species undoubtedly have the tendency to believe what we want to hear. And those who actually were there or who did count the bodies frequently have a vested interest in lying to us, or in first lying to themselves and then to us.

The actual, ‘true’ reality hence becomes unknowable. Another similar example would be the various casualty counts during the Bosnian conflict – for instance, the narrative constructed by Bosniak/Bosnian Muslim politicians that 200,000 Bosniaks were killed during the war, or the Bosnian Serb narrative about the 3,200 Serb civilians killed in the Srebrenica area during the conflict, which is being used to relativize the 1995 massacre of 8,000 Bosnian Muslims as a quite ‘natural’ instance of revenge (see more here). Or, to go a bit further into history, one of the most effective methods used to inflame ethnic nationalism in the former Yugoslavia was to manipulate the number of Serbs killed by Croatian fascists in the Jasenovac camp during World War II. Hence Serbian nationalists to this day tend to exaggerate the number of victims, while Croat nationalists tend to downplay it. These conflicting versions of reality become embedded in public discourse and generally accepted as an absolute truth, particularly when they are very effectively imparted on the succeeding generations through school textbooks.

For instance, I invite readers linguistically able to do so to read the entries on Jasenovac in the Serbian, Croatian, and English versions of Wikipedia, and to compare the general tenor of articles as well as the factual specifics. Wikipedia famously works so that while everyone can edit an entry, a consensus eventually builds that produces a definitive account. Edit wars on controversial topics are frequent, and have to be moderated by outsiders without a vested interest in the topic. But in part because language can be such an effective barrier to outside intervention, Wikipedia demonstrates how easy it is to construct alternative versions of reality, that are of course much more extreme outside Wikipedia itself, which is by design meant to be as impartial as possible.

We all depend on most of our ‘knowledge’ outside our area of immediate expertise on the accounts by others that we believe possess such expertise. In other words, just like the law, the human perception of reality is an authority-driven process – most of us believe that the Earth revolves around the Sun, or that man landed on the Moon, or that Earth is more than 5,000 years old, or in the theory of evolution, not because we have independently evaluated the evidence and conducted the necessary observations, but because we believe what our schoolteachers told us. But when it comes to politically or ideologically charged questions, authorities must be treated with skepticism. If we truly wish to assess a particular situation rationally and dispassionately, we should only believe those authorities who have the least reason to lie to us, and even then we should always keep an open mind.

Thus, for instance, the account on the number of casualties during the Bosnian war that I choose to believe in is the one which goes against the dominant narratives of both the Bosniak and the Serbian side. But do I know how many people died in Bosnia – no, I do not. I equally have no idea how many civilians died in Gaza – and neither do most people who so fervently claim that they do. It is particularly in those situations, like Gaza, when the access to information is basically monopolized by entities who have every reason to misrepresent it (viz. Israel and Hamas), that the only rational response is to profess ignorance. UN investigators and tribunals, human rights NGOs and the like are of course more credible than either of the warring parties, but they are not themselves free from any political agenda, nor are they free from doubts, justified or unjustified, as to their impartiality (see here and here). And if we must ultimately choose whom to trust, let us at least admit openly that those whom we are because of our own biases inclined to believe the most in a dispute are usually those who should be trusted the least. That, I think, would be the necessary first step towards rationality.

4 Responses for "The Wikipedia Approach to Reality"

Darryl Li September 10th, 2009 at 2:30
Dear Marko,

I would reframe the issue slightly: the overall number of deaths is actually not particularly controversial. The ratio of civilians is, but that’s also not as straightforwardly empirical as you suggest, since it involves questions about legal standards that you and others have explored extensively on this blog.

But more importantly: I agree that we should always be careful and honest about one’s biases and their relationship with the conclusions (political, otherwise) that are drawn from factual claims. But I fear your post goes too far in implying that even competing factual claims should be treated equally, without considering their merits.

As a matter of disclosure, I have worked in Israel/Palestine for Human Rights Watch, B’tselem, and the Palestinian Centre for Human Rights. I feel I have a pretty good sense of their methodologies and the pressures under which they operate, though obviously I write here solely in my personal capacity.

1. Methodology

Israel’s claims about casualties in the January war are unverifiable because they haven’t released a list of names and have not discussed their methodology in any detail. In contrast, the human rights groups use methodology that is comparatively much more transparent and, importantly, falsifiable, allowing people to independently check those claims (which a cottage industry of pro-Israel bloggers zealously do, with largely unimpressive results). In this case, B’tselem has sent its entire list to the IDF and used birth certificates, hospital records, etc. (PCHR also published its list several months back).

Without getting into the limits of Popper’s arguments about falsifiability, as a practical matter it would seem odd to accord equal weight to a zero-transparency claim versus one that provides considerable detail on methods and results.

2. Conceptual and definitional debates

In addition, the factual disputes that _do_ exist often hinge on issues that are clearly identifiable and allow one to narrow (or at least more clearly define) the field of disagreement.

One such issue is that Israel classifies all police personnel as combatants, as well as individuals who it deems to be members of armed groups, even if their participation does not meet the ICRC “continuous function” test and they are not engaged in hostilities (in short, anyone who ever signed up for an armed group or posed for a macho photograph can be retroactively counted as a combatant). This single issue accounts for a significant portion of the discrepancy between the IDF and the human rights groups. In other words, the fact that there are often divergent interpretation of the same data does not necessarily invalidate that data itself.

[this also presents a contrast with the Yugoslav example -- in Israel/Palestine, there is generally little disagreement about the overall numbers of deaths, the dispute is about how to classify those deaths; in the former Yugoslav wars, my impression is that variance on total deaths far greater, probably for reasons of scale]

3. Knee-jerk skepticism

The danger of believing only what we are predisposed to believe is indeed great and demands constant vigilance. But we should also bear in mind that a position of knee-jerk skepticism towards all claims (“Israel/Palestine is very complicated, so all sides are equally right/wrong”) requires just as little thought as believing whatever one’s own “side” proclaims.

In the Gaza case, it is simply not true that information is “monopolized” by those with reason to lie; yes, Israel and Hamas impose restrictions, as do parties in any conflict. Yet human rights NGOs can and do cross-check each other’s work to come up with reasonably coherent narratives and analyses (hence the B’tselem report works better as evidence of a strengthening consensus rather than an example of deep epistemological uncertainty). Israel/Palestine may appear more ‘controversial’ or ‘polarizing’ than another conflicts; but the quantity of hard data about deaths is probably far greater per capita than for any other conflict in the world because of the relatively small numbers and relatively intense scrutiny from various parties.

Rather than simply acknowledging that everyone has an agenda, it would seem a more important step towards rationality would be to actually independently engage and weigh competing claims to the best of one’s ability.
Marko Milanovic September 10th, 2009 at 9:09
Darryl,

Many thanks for your comment. On the specifics of the Israeli/Palestinian conflict, I agree with much of what you have said. In principle, I also agree with your conclusion that one should weigh competing claims to the best of one’s ability. But the limits of that ability are quite, quite narrow, even for those with a modicum of expertise in the field.

So, I personally would venture into having an opinion on some of the legal issues, which you quite correctly say are interlinked with the factual issues, e.g. is a Hamas policeman a civilian. But I am at a loss beyond that. You thus say that the B’tselem and the PCHR reports are more trustworthy than the Israeli gov’t one because (1) they give a list of names, and (2) they use a good methodology.

As for (1), the experience of the Yugoslav wars amply shows just how easy it is for ‘experts’ of one ideological stripe or another to produce a list of names. You wouldn’t believe the number of such lists circulating, e.g. about Serbs killed in Srebrenica, all supposedly based on verifiable information. As for (2), do you honestly believe that a regular person reading the news etc is capable of assessing the merits or the rigor of such a methodology? That this reader has both the necessary expertise, access to the necessary information, and above all the TIME required to properly assess a methodology?

I freely admit that I do not. I trust the B’tselem casualty count more than the Israeli gov’t account (and I do; I did not say that I am equally skeptical towards all sources in my original post) not because of its better methodology, but because I choose to trust a human rights NGO over a state which has every reason to cover up its misdeeds. And, frankly, I think that this goes for most people. But I am aware of the limits of placing my trust in that way.

To again briefly refer to the ongoing ad hominem attacks against you personally and other HRW staffers, as in the recent NGO Monitor report – these attacks can work and should be taken seriously precisely because HRW and other human rights NGO operate vis-a-vis the general public on the basis of trust and faith. In other words, just like Caesar’s wife, you must strive to be above all suspicion. If you lose your reputation and credibility, you’ve lost everything, better methodologies notwithstanding.

(Apologies for being preachy. I usually am not.)
Darryl Li September 10th, 2009 at 15:39
Marko,

Thanks for your thoughtful reply and for raising these important issues. A few points in response:

1. On re-reading our exchange, it strikes me that we may have been talking past each other, insofar as I did not address the importance of the context in which empirical claims are received and interpreted. It seems fair to say that the relative weight given to certain claims in the face of contradictory evidence varies incredibly depending on other contextual factors (political, historical, etc.): e.g., the inclination of some people in Serbia to believe that a massacre of Serbs occurred at Srebrenica, or the inclination of many Americans to believe that Saddam Husayn possessed WMD.

2. That being said, and in contexts where people do not feel some kind of personal or atavistic stake (as with many though by no means all readers of western newspapers re Israel/Palestine), I stand by the point that falsifiability and transparency should count for something. I agree that one can create a misleading impression of rigorous method and of course do not expect everyone to personally scrutinize the raw data used by HR NGOs. But the _availability_ of raw data and methods, in contrast with what governments or other actors (in this particular case, Israel) provide to support their claims should and I think in some cases does factor into people’s credibility judgments.

And more importantly, it should certainly factor into how journalists treat such claims — methodology is important precisely because the relationship between NGO claims and the newspaper reader is mediated by others in the public sphere who can and do cross-check them (though I admit this point is more relevant when it comes to case studies than statistical tallies). In the example that you cite — the ‘massacre’ of Serbs at Srebrenica — I’m sure the claims were also subjected to independent scrutiny by someone with more time/resources than you and me and treated accordingly, even if their intended audience was uninterested in potential refutations (cf. point above about context).

The limitation (and perhaps you were alluding to this, I wasn’t entirely clear) of the approach you outline — ask who has reason to lie — is that it is always possible to imagine cynical motives, as frequently occurs: HR groups allegedly lie because of their political agendas, hatred of people x, or desire to inflate casualty numbers for attention and funding. Given the imperfections of both approaches, I would rather combine them rather than go only with one.

3. On ad hominem attacks, I agree that human rights organizations must hold themselves to very high standards and that such attacks can work. I also agree that your approach — ask who has reason to lie — is a useful starting point here (though by no means does not exhaust the well of refutations available), as NGO Monitor is run by a consultant to the Israeli government.

Best,
Darryl
The Debate Link September 11th, 2009 at 20:46
Narrative in the Israeli/Palestinian Conflict, II…

A fabulous post by Marko Milanovic on the blog of the European Journal of International Law talks about many of the issues I raised in my own “Narrative in the Israeli/Palestinian Conflict” post a few years back. I highly recommend it — as well as t…