The Peer-Review Cartel

For the on-going debate, please see the RHS bar under Also See

Preface

Vijay and I seem to have scoped a vast canvas on which to paint our debate. Rather than each postaddressing all the issues of the prior post of the other side (and thereby not allowing us to go deep enoughinto the foundational framework), I will take one specific item of contention at a time and try to present myposition. I am starting with item 4 on my list of themes (which Vijay kindly accepted via a private email asbeing a reasonable way to proceed), titled, Power and Knowledge in India relatedstudies. 

This articleis the first of a series to lay the ground work for my positions on this theme. One of the fundamental issuesat stake, which I shall argue below, is that the nature of the peer-review process in humanities / liberalarts is creating a knowledge production cartel that gives the Western academy neocolonialist control over themeans of production of knowledge. Any critique from outside the elite neo-Brahmin cartel is sidelined(especially if it is seen as a serious enough threat) by invoking the "peer-review" as a silver bullet.One of the most cherished myths of the Western-controlled liberal arts intellectual apparatus is that itspeer-review is a fair system. This essay demolishes this myth...

Prof. Wendy Doniger, Prof. Paul Courtright and others have alleged that the criticism I have made of theirscholarship is illegitimate because their writings have been peer-reviewed. Therefore, they claim, my writingsmust be classified as "attacks" on them, and not as fair criticism, because they do not emanate fromwithin the scholarly world. 

The implication here is that those who are not licensed by their academic system should not be allowed toargue with their positions, and certainly not as equal partners in dialog. This attitude is, in my view, partof a larger problem in academic discourse, especially in anthropology, sociology and the study of religion,where it is assumed that (i) the non-academician can only be positioned as a native informant, and (ii) thenative informant should not talk back. 

At a major world conference on academic Religious Studies in Delhi in December, 2003, sponsored by TheInfinity Foundation, a few Indian scholars are reported to have closed ranks to emphasize the schism between"we the scholars" and "you the ordinary people." To defend the monopoly of the Western academicfortress over the discourse on Indian society, one of their central planks has been that peer-reviewedscholarship cannot be criticized by ordinary people. 

Clearly, the peer-review process has acquired tremendous symbolic value. It is, after all, what separatesan academician’s writings from whatever we ordinary folks might ever produce, and what distinguishes theguild, for which the entrance fees are steep and time-consuming, from the rest of us. 

I am glad that scholars have the peer-review system, as this provides a critique of scholarly works bytheir peers prior to publication, and thereby provides some level of checks and balances. But they should notuse it as the final word to close the case on contentious issues, because it is, as I argue below, fallibleand often biased in ways that insiders to the guild are not easily able to see. 

This essay is particularly critical of the over-confidence in the peer-review process in India-relatedscholarship. This blind spot in the academy prevents it from much-needed self-reflection. As long as scholarsclaim immunity from criticism by others, on the grounds of status and authority alone, intellectual deadlockswill continue. 

To divert from this issue, some academicians have raised the red flag of censorship to describe the role Ihave tried to play in contesting them, although I have never called for or endorsed censorship of any kind,but have simply insisted on the right to debate and contest views promulgated by scholars that do not accordwith a different and perhaps more grounded perspective. 

Sokal’s Hoax 

I’d like to begin my critique of the peer-review system by citing Sokal’sHoax, a famous instanceof exposure of the lack of quality controls in liberal arts scholarship. Alan D. Sokal, a well-known physicsprofessor at New York University, played a famous hoax that has become very embarrassing to scholars. Everyliberal arts scholar, as well as everyone wishing to argue with them, should study this case and itsimplications. Unfortunately, many liberal arts professors do not include it in their reading assignments. Hereare its highlights. 

Prof. Sokal submitted an article titled, Transgressing the Boundaries: Toward a TransformativeHermeneutics of Quantum Gravity, to a scholarly journal, Social Text, which prided itself onits postmodern and avant-garde point of view. The article was a typical cut-and-paste, tongue-in-cheekconstruction of a high-flown thesis using scientific jargon and literary theories to claim that quantumphysics supports radical left-wing ideas. After it was published, Sokal exposed his hoax in another articlepublished in LinguaFranca. He wrote: 

"To test the prevailing intellectual standards, I decided to try a modest (though admittedlyuncontrolled) experiment: Would a leading North American journal of cultural studies - whose editorialcollective includes such luminaries as Fredric Jameson and Andrew Ross - publish an article liberally saltedwith nonsense if (a) it sounded good and (b) it flattered the editors' ideological preconceptions? The answer,unfortunately, is yes..." 

Sokal showed how the editors and peer-reviewers of this important academic journal had been easily duped bynonsense that was deliberately fabricated just to test their competence. 

He explains the significance of his hoax: 

"Throughout the article, I employ scientific and mathematicalconcepts in ways that few scientists or mathematicians could possibly take seriously... I assert that Lacan'spsychoanalytic speculations have been confirmed by recent work in quantum field theory. Even nonscientistreaders might well wonder what in heavens' name quantum field theory has to do with psychoanalysis; certainlymy article gives no reasoned argument to support such a link... I intentionally wrote the article so that anycompetent physicist or mathematician (or undergraduate physics or math major) would realize that it is aspoof. Evidently the editors of Social Text felt comfortable publishing an article on quantum physicswithout bothering to consult anyone knowledgeable in the subject..." 

It is important to understand the seriousness of the hoax in Sokal’s own words: 

The fundamental silliness of my article lies, however, not in its numerous solecisms but in the dubiousnessof its central thesis and of the "reasoning"' adduced to support it... I assemble a pastiche - Derrida andgeneral relativity, Lacan and topology, Irigaray and quantum gravity - held together by vague rhetoric...Nowhere in all of this is there anything resembling a logical sequence of thought; one finds only citations ofauthority, plays on words, strained analogies, and bald assertions... 

What's more surprising is how readily they accepted my implication that the search for truth in sciencemust be subordinated to a political agenda, and how oblivious they were to the article's overall illogic... 

The results of my little experiment demonstrate, at the very least, that some fashionable sectors of theAmerican academic Left have been getting intellectually lazy. The editors of Social Text liked myarticle because they liked its conclusion: that ‘the content and methodology of postmodern science providepowerful intellectual support for the progressive political project.' They apparently felt no need to analyzethe quality of the evidence, the cogency of the arguments, or even the relevance of the arguments to thepurported conclusion... 

I resorted to parody for a simple pragmatic reason. The targets of my critique have by now become aself-perpetuating academic subculture that typically ignores (or disdains) reasoned criticism from theoutside. In such a situation, a more direct demonstration of the subculture's intellectual standards wasrequired. But how can one show that the emperor has no clothes? Satire is by far the best weapon; and the blowthat can't be brushed off is the one that's self-inflicted. I offered the Social Text editors anopportunity to demonstrate their intellectual rigor. Did they meet the test? I don't think so. I say this notin glee but in sadness. After all, I'm a leftist too..." 

Sokal concludes: 

"Social Text’s acceptance of my article exemplifies the intellectual arroganceof Theory - meaning postmodernist literary theory - carried to its logical extreme." 

Sokal angered the whole liberal arts establishment because he had exposed its pretentiousness. But one ofhis supporters cynically remarked, "What passes for theory in academic circles is the intellectualequivalent of bubble gum, churned out solely in order to keep the otherwise useless at work." 

Alan Sokal played his remarkable hoax to illustrate the point that without better checks and balances inplace, patently false information and analysis is being disseminated and accepted as ‘true.’ His hoaxshows serious weaknesses in the peer-review process itself. These weaknesses are not restricted to journalssuch as Social Text. They are pervasive in the academy, and especially in the treatment and understanding ofIndia and its culture, as I will be arguing. 

Furthermore, the problem also exists in reverse: Many articles are not published even after they have beencritiqued (and even acclaimed) by the world's foremost authorities in some of the disciplines involved, simplybecause they undermine reputations of some academic icons.

This essay does not take any stand on either side of the universalism/relativism debate in philosophy thatSokal is involved in. My reason for starting it with the Sokal Hoax is merely to illustrate the fallibility ofthe peer-review system, in order to convince the reader not to dismiss my thesis simply because it raises thevery real possibility that many who pride themselves on having been vetted by "peer-review" are on shakyground. 

Errors despite peer-reviews 

Let me outline some of the major sources of errors in scholarship about India. Each source of error isseparate and distinct and, even if a given reader accepts only some of these arguments, it would puncture thelargely unquestioned credibility of scholarship in Indology and South Asian Studies.

Not scientifically verifiable or reproducible or universal: 

While I am primarily interested in criticizing the study of India and its culture, there are many overlapsbetween the issues concerning India Studies and those that apply to liberal arts and the humanities ingeneral. Peer-reviews in these disciplines simply cannot be as rigorous as those in science, because empiricalverification is unavailable. The conclusions they claim are not easily provable, especially as universalassertions. The liberal arts use a wide range of fashionable "theories" to reason and to reach scholarlyconsensus, but this process tends to be heavily political and deeply influenced by cultural biases as shownlater. 

What they produce should be seen as consensus and not truth. Like any consensus, it becomes in part amatter of who the players are in reaching the consensus, and what forces are at work, including funding andpolitics. The possibilities for blindness here are increased by this problem of method and verification. 

To illustrate the non-reproducible nature of the work using anthropology as an example, one must note thatthere is more glamour and recognition for an anthropologist to go to study an obscure community "where noone has gone before." A "good" anthropologist tends to spend years, possible decades, returning to thesame locality to become the Western academy’s expert on it. The result of this ultra-specialization is thatthis scholar’s work has to be taken at face value, as there is no other expert to contest any findingsconcerning that specific tribe or community. The native informants from within the community being studied aresimply unable to argue back, given the imbalance of power, and nor are they given a translated account of thescholar’s reports published in the West. So there is never any independent verification of the data orinterpretations of the scholar, because other experts work in different cultural contexts. One has to depend agreat deal on the "reputation" of the scholar, and this becomes a matter for the most part of politics andpersonality. 

This kind of scholarship is non-verifiable and non-reproducible. Western readers often fail tocontextualize the narrative as the perspective of an "outsider" that may be skewed in the following ways:(1) The native informant’s vested interests and intentions distort, just like any measurement perturbs thesystem being measured. (2) The scholar’s understanding, both literally and cognitively, is distorted by thescholar’s private framework. (3) The scholar has a propensity towards conclusions that support theparticular political, religious or other institutional frameworks s/he is operating in. (4) Thegeneralizations made lead to stereotypes, given the enormous diversity of the Indian experience. 

Arbitrary choice of theories: 

While anthropologists do acknowledge many of the problems in their discipline, less well known, though inmy view even more problematic is that the theories used in the research are entirely Western, privileging anembedded worldview. No such theory is value-free or neutral. 

"Being critical is being political..." says one popular introduction to the fashionable theories usedin the liberal arts. It goes on to say: "From Marxism onwards, critical theory has been very closely linkedto political positions." So how do these scholars claim objectivity?

Here is the issue: The selection of the theory or theories to be used in a given instance is entirelyarbitrary, and may be compared to picking ad hoc tools from a toolbox. The introductory guide makes this clearin its explanation to the newcomer: "The cultural analyst can pick or mix from the catalog of theories toput together synthetic models for whatever the task may happen to be." 

The proponents of liberal arts proudly claim that they no longer study literature, art or culture in and ofthemselves; rather, these are "objects" to be processed via specific "theories." This makes thelegitimization and promotion of particular "theories" a very serious business, indeed. Whoever controlsthe "theories" controls the discourse. Since Sanskrit-based literary theories and hermeneutics gotmarginalized a long time ago, among many other non-Western paradigms, to be a scholar today one must useWestern sanctioned theories. 

I want to advance analogies to business here, which, though they do not always apply, can illustrate whatis happening in this situation where there are so few serious controls or challenges to the dominantparadigms. In most industries, there is an overabundance of product choices, but very few can be distributedthrough the available channels, and this critical bottleneck of distribution often determines which productswin. If Wal-Mart gives your product shelf-space, that product will make money, and if it rejects your productthe chances are increased that it will fail. Critical bottlenecks are inherent in every distribution channel– media, catalogs, retail, sales force, etc., because the product availability vastly exceeds the limitedbandwidth of the channel. 

Therefore, the important question before us is: Who has the power, and by what authority, to decide whichamong the theories that are on the market shall belong to the catalog that is approved for scholarly usage? Towhat extent is popularity (by virtue of trendiness, money and powerful backing) the dominant criterion,analogous to the way internet search engines use the number of hits in their algorithms to rank web sites fora given search? Does this suggest a vicious cycle, whereby usage of the theory by intellectuals promotes thattheory to gain market share, a process of assigning value that is not commensurate with merit? 

To what extent is the power of funding the application of certain theories (via individual research, bookprojects, conference/seminar themes, "institutes" and "area studies") equivalent to web sites beingable to buy top spots in search engines, or PR agencies being able to get a new author on the Oprah Show, or apublisher being able to buy a prestigious display spot from Barnes & Noble? Why has the academy not wantedto inquire into such issues pertaining to the way market share is won for liberal arts theories?