The Peer-Review Cartel

For the on-going debate, please see the RHS bar under Also See

Preface

Vijay and I seem to have scoped a vast canvas on which to paint our debate. Rather than each post addressing all the issues of the prior post of the other side (and thereby not allowing us to go deep enough into the foundational framework), I will take one specific item of contention at a time and try to present my position. I am starting with item 4 on my list of themes (which Vijay kindly accepted via a private email as being a reasonable way to proceed), titled, Power and Knowledge in India related studies. 

This article is the first of a series to lay the ground work for my positions on this theme. One of the fundamental issues at stake, which I shall argue below, is that the nature of the peer-review process in humanities / liberal arts is creating a knowledge production cartel that gives the Western academy neocolonialist control over the means of production of knowledge. Any critique from outside the elite neo-Brahmin cartel is sidelined (especially if it is seen as a serious enough threat) by invoking the "peer-review" as a silver bullet. One of the most cherished myths of the Western-controlled liberal arts intellectual apparatus is that its peer-review is a fair system. This essay demolishes this myth...

Prof. Wendy Doniger, Prof. Paul Courtright and others have alleged that the criticism I have made of their scholarship is illegitimate because their writings have been peer-reviewed. Therefore, they claim, my writings must be classified as "attacks" on them, and not as fair criticism, because they do not emanate from within the scholarly world. 

The implication here is that those who are not licensed by their academic system should not be allowed to argue with their positions, and certainly not as equal partners in dialog. This attitude is, in my view, part of a larger problem in academic discourse, especially in anthropology, sociology and the study of religion, where it is assumed that (i) the non-academician can only be positioned as a native informant, and (ii) the native informant should not talk back. 

At a major world conference on academic Religious Studies in Delhi in December, 2003, sponsored by The Infinity Foundation, a few Indian scholars are reported to have closed ranks to emphasize the schism between "we the scholars" and "you the ordinary people." To defend the monopoly of the Western academic fortress over the discourse on Indian society, one of their central planks has been that peer-reviewed scholarship cannot be criticized by ordinary people. 

Clearly, the peer-review process has acquired tremendous symbolic value. It is, after all, what separates an academician’s writings from whatever we ordinary folks might ever produce, and what distinguishes the guild, for which the entrance fees are steep and time-consuming, from the rest of us. 

I am glad that scholars have the peer-review system, as this provides a critique of scholarly works by their peers prior to publication, and thereby provides some level of checks and balances. But they should not use it as the final word to close the case on contentious issues, because it is, as I argue below, fallible and often biased in ways that insiders to the guild are not easily able to see. 

This essay is particularly critical of the over-confidence in the peer-review process in India-related scholarship. This blind spot in the academy prevents it from much-needed self-reflection. As long as scholars claim immunity from criticism by others, on the grounds of status and authority alone, intellectual deadlocks will continue. 

To divert from this issue, some academicians have raised the red flag of censorship to describe the role I have tried to play in contesting them, although I have never called for or endorsed censorship of any kind, but have simply insisted on the right to debate and contest views promulgated by scholars that do not accord with a different and perhaps more grounded perspective. 

Sokal’s Hoax  

I’d like to begin my critique of the peer-review system by citing Sokal’s Hoax, a famous instance of exposure of the lack of quality controls in liberal arts scholarship. Alan D. Sokal, a well-known physics professor at New York University, played a famous hoax that has become very embarrassing to scholars. Every liberal arts scholar, as well as everyone wishing to argue with them, should study this case and its implications. Unfortunately, many liberal arts professors do not include it in their reading assignments. Here are its highlights. 

Prof. Sokal submitted an article titled, Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity, to a scholarly journal, Social Text, which prided itself on its postmodern and avant-garde point of view. The article was a typical cut-and-paste, tongue-in-cheek construction of a high-flown thesis using scientific jargon and literary theories to claim that quantum physics supports radical left-wing ideas. After it was published, Sokal exposed his hoax in another article published in Lingua Franca. He wrote: 

"To test the prevailing intellectual standards, I decided to try a modest (though admittedly uncontrolled) experiment: Would a leading North American journal of cultural studies - whose editorial collective includes such luminaries as Fredric Jameson and Andrew Ross - publish an article liberally salted with nonsense if (a) it sounded good and (b) it flattered the editors' ideological preconceptions? The answer, unfortunately, is yes..." 

Sokal showed how the editors and peer-reviewers of this important academic journal had been easily duped by nonsense that was deliberately fabricated just to test their competence. 

He explains the significance of his hoax: 

"Throughout the article, I employ scientific and mathematical concepts in ways that few scientists or mathematicians could possibly take seriously... I assert that Lacan's psychoanalytic speculations have been confirmed by recent work in quantum field theory. Even nonscientist readers might well wonder what in heavens' name quantum field theory has to do with psychoanalysis; certainly my article gives no reasoned argument to support such a link... I intentionally wrote the article so that any competent physicist or mathematician (or undergraduate physics or math major) would realize that it is a spoof. Evidently the editors of Social Text felt comfortable publishing an article on quantum physics without bothering to consult anyone knowledgeable in the subject..." 

It is important to understand the seriousness of the hoax in Sokal’s own words: 

The fundamental silliness of my article lies, however, not in its numerous solecisms but in the dubiousness of its central thesis and of the "reasoning"' adduced to support it... I assemble a pastiche - Derrida and general relativity, Lacan and topology, Irigaray and quantum gravity - held together by vague rhetoric... Nowhere in all of this is there anything resembling a logical sequence of thought; one finds only citations of authority, plays on words, strained analogies, and bald assertions... 

What's more surprising is how readily they accepted my implication that the search for truth in science must be subordinated to a political agenda, and how oblivious they were to the article's overall illogic... 

The results of my little experiment demonstrate, at the very least, that some fashionable sectors of the American academic Left have been getting intellectually lazy. The editors of Social Text liked my article because they liked its conclusion: that ‘the content and methodology of postmodern science provide powerful intellectual support for the progressive political project.' They apparently felt no need to analyze the quality of the evidence, the cogency of the arguments, or even the relevance of the arguments to the purported conclusion... 

I resorted to parody for a simple pragmatic reason. The targets of my critique have by now become a self-perpetuating academic subculture that typically ignores (or disdains) reasoned criticism from the outside. In such a situation, a more direct demonstration of the subculture's intellectual standards was required. But how can one show that the emperor has no clothes? Satire is by far the best weapon; and the blow that can't be brushed off is the one that's self-inflicted. I offered the Social Text editors an opportunity to demonstrate their intellectual rigor. Did they meet the test? I don't think so. I say this not in glee but in sadness. After all, I'm a leftist too..." 

Sokal concludes: 

"Social Text’s acceptance of my article exemplifies the intellectual arrogance of Theory - meaning postmodernist literary theory - carried to its logical extreme." 

Sokal angered the whole liberal arts establishment because he had exposed its pretentiousness. But one of his supporters cynically remarked, "What passes for theory in academic circles is the intellectual equivalent of bubble gum, churned out solely in order to keep the otherwise useless at work." 

Alan Sokal played his remarkable hoax to illustrate the point that without better checks and balances in place, patently false information and analysis is being disseminated and accepted as ‘true.’ His hoax shows serious weaknesses in the peer-review process itself. These weaknesses are not restricted to journals such as Social Text. They are pervasive in the academy, and especially in the treatment and understanding of India and its culture, as I will be arguing. 

Furthermore, the problem also exists in reverse: Many articles are not published even after they have been critiqued (and even acclaimed) by the world's foremost authorities in some of the disciplines involved, simply because they undermine reputations of some academic icons.

This essay does not take any stand on either side of the universalism/relativism debate in philosophy that Sokal is involved in. My reason for starting it with the Sokal Hoax is merely to illustrate the fallibility of the peer-review system, in order to convince the reader not to dismiss my thesis simply because it raises the very real possibility that many who pride themselves on having been vetted by "peer-review" are on shaky ground. 

Errors despite peer-reviews 

Let me outline some of the major sources of errors in scholarship about India. Each source of error is separate and distinct and, even if a given reader accepts only some of these arguments, it would puncture the largely unquestioned credibility of scholarship in Indology and South Asian Studies.

Not scientifically verifiable or reproducible or universal: 

While I am primarily interested in criticizing the study of India and its culture, there are many overlaps between the issues concerning India Studies and those that apply to liberal arts and the humanities in general. Peer-reviews in these disciplines simply cannot be as rigorous as those in science, because empirical verification is unavailable. The conclusions they claim are not easily provable, especially as universal assertions. The liberal arts use a wide range of fashionable "theories" to reason and to reach scholarly consensus, but this process tends to be heavily political and deeply influenced by cultural biases as shown later. 

What they produce should be seen as consensus and not truth. Like any consensus, it becomes in part a matter of who the players are in reaching the consensus, and what forces are at work, including funding and politics. The possibilities for blindness here are increased by this problem of method and verification. 

To illustrate the non-reproducible nature of the work using anthropology as an example, one must note that there is more glamour and recognition for an anthropologist to go to study an obscure community "where no one has gone before." A "good" anthropologist tends to spend years, possible decades, returning to the same locality to become the Western academy’s expert on it. The result of this ultra-specialization is that this scholar’s work has to be taken at face value, as there is no other expert to contest any findings concerning that specific tribe or community. The native informants from within the community being studied are simply unable to argue back, given the imbalance of power, and nor are they given a translated account of the scholar’s reports published in the West. So there is never any independent verification of the data or interpretations of the scholar, because other experts work in different cultural contexts. One has to depend a great deal on the "reputation" of the scholar, and this becomes a matter for the most part of politics and personality. 

This kind of scholarship is non-verifiable and non-reproducible. Western readers often fail to contextualize the narrative as the perspective of an "outsider" that may be skewed in the following ways: (1) The native informant’s vested interests and intentions distort, just like any measurement perturbs the system being measured. (2) The scholar’s understanding, both literally and cognitively, is distorted by the scholar’s private framework. (3) The scholar has a propensity towards conclusions that support the particular political, religious or other institutional frameworks s/he is operating in. (4) The generalizations made lead to stereotypes, given the enormous diversity of the Indian experience. 

Arbitrary choice of theories: 

While anthropologists do acknowledge many of the problems in their discipline, less well known, though in my view even more problematic is that the theories used in the research are entirely Western, privileging an embedded worldview. No such theory is value-free or neutral. 

"Being critical is being political..." says one popular introduction to the fashionable theories used in the liberal arts. It goes on to say: "From Marxism onwards, critical theory has been very closely linked to political positions." So how do these scholars claim objectivity?

Here is the issue: The selection of the theory or theories to be used in a given instance is entirely arbitrary, and may be compared to picking ad hoc tools from a toolbox. The introductory guide makes this clear in its explanation to the newcomer: "The cultural analyst can pick or mix from the catalog of theories to put together synthetic models for whatever the task may happen to be." 

The proponents of liberal arts proudly claim that they no longer study literature, art or culture in and of themselves; rather, these are "objects" to be processed via specific "theories." This makes the legitimization and promotion of particular "theories" a very serious business, indeed. Whoever controls the "theories" controls the discourse. Since Sanskrit-based literary theories and hermeneutics got marginalized a long time ago, among many other non-Western paradigms, to be a scholar today one must use Western sanctioned theories. 

I want to advance analogies to business here, which, though they do not always apply, can illustrate what is happening in this situation where there are so few serious controls or challenges to the dominant paradigms. In most industries, there is an overabundance of product choices, but very few can be distributed through the available channels, and this critical bottleneck of distribution often determines which products win. If Wal-Mart gives your product shelf-space, that product will make money, and if it rejects your product the chances are increased that it will fail. Critical bottlenecks are inherent in every distribution channel – media, catalogs, retail, sales force, etc., because the product availability vastly exceeds the limited bandwidth of the channel. 

Therefore, the important question before us is: Who has the power, and by what authority, to decide which among the theories that are on the market shall belong to the catalog that is approved for scholarly usage? To what extent is popularity (by virtue of trendiness, money and powerful backing) the dominant criterion, analogous to the way internet search engines use the number of hits in their algorithms to rank web sites for a given search? Does this suggest a vicious cycle, whereby usage of the theory by intellectuals promotes that theory to gain market share, a process of assigning value that is not commensurate with merit? 

To what extent is the power of funding the application of certain theories (via individual research, book projects, conference/seminar themes, "institutes" and "area studies") equivalent to web sites being able to buy top spots in search engines, or PR agencies being able to get a new author on the Oprah Show, or a publisher being able to buy a prestigious display spot from Barnes & Noble? Why has the academy not wanted to inquire into such issues pertaining to the way market share is won for liberal arts theories?