Charles Plott's strategies for getting published

There are plenty of critiques of peer review, one of which is that it is incremental and leads to the most novel research failing to get published. If you are a researcher doing incredibly novel research, using methods that reviewers don't fully understand because those methods are not yet widely employed, then you could have real difficulty in getting your best work into top journals. That has been the case for many emerging fields. In economics, in (relatively) recent times (prior to the 1990s), that applies to behavioural economics and to experimental economics. The challenge, then, is how to get your work accepted if you are one of these researchers employing novel or misunderstood methods.

In a new article published in the journal Oxford Economic Papers (ungated version here), Andrej Svorencik (University of Mannheim) documents how one of the pioneers of experimental economics, Charles Plott, overcame the reluctance of editors and journal reviewers to accept the validity of laboratory experiments. As Svorencik writes:

Whereas there were no public detractors of experimentation in economics, the early and most prolific experimenters, such as Charles Plott and Vernon Smith, encountered skeptics and systematic rejections of their submitted papers. Getting them published required tenacity on the writers’ part to go through several rounds of often heated discussions with editors and referees. These iterations present a unique perspective on the arguments raised against experiments in economics and the specific strategies developed by experimental economists to counter them.

Svorencik uses the 'research corpus' of Plott, covering letters and responses to editors and reviewers dating from the mid-1970s to the mid-1990s, and establishes nine different strategies that Plott employed to disarm reviewers and convince editors that his publications using experimental economics should be published:

S1 Asking for knowledgeable referees because previous referees were ignorant of experimental economics;
S2 Claiming that results are interesting, relevant for theory, and have applications;
S3 Claiming that the experiments present real situations;
S4 Claiming that the theory applies to simple cases;
S5 Citing basic research;
S6 Conducting more experiments;
S7 Shifting of the burden of proof;
S8 Steering clear of a specialized journal;
S9 Claiming that field has been confused with method.

One particular example of strategy S4 struck me as particularly important. From a 1979 letter that Plott wrote to George Borts, the editor of the American Economic Review:

The laboratory processes are simple and very special markets... but they are nevertheless real markets which should be governed by the same principles that are supposed to govern all markets. The justification for studying them is the same as the justification for studying the simple special cases and special types of any complicated phenomenon.

In order to see why these markets are real, one need only apply directly the theory of derived demand. It works as follows. Let Ri(xi) be the revenue received by individual i from some source expressed as a function of the number of units (xi) he has to sell. Standard derived demand theory tells us that δRi/δxi is limit price (inverse demand) function for this individual. It is important to note that the theory places no restriction upon the source of the revenue so when the source is an experimenter the derived limit price function for this individual is just as real as when the source is a business. Furthermore, the theory places no restriction on what x is called (unless the individual gets consumption pleasures from it) so the theory applies equally as xi becomes baseball cards, shirts, food, or ‘commodities’ created especially for the purposes of an experiment. There are no ‘side payments’ or incidental sources of enjoyment so as long as the individual prefers more money to less we can be assured the preferences for units of x have been induced. The individual is indeed a ‘demander.’

The supply side of the market is handled similarly. Each supplier, j, faces an individualized cost function Cj(xj) which indicates what j must pay the experimenter as a function of units purchased for resale. Profits to j, which are j’s to keep, are simply the revenues received by j over costs Cj(xj). Clearly that δCj(xj)/δxj is a real marginal cost function. The fact that it was constructed by the experimenter makes the concept no less relevant because the concept is intended to apply universally.

We have then a valued and scarce resource. Almost any textbook will say that those conditions are sufficient for the existence of an economic problem. The laboratory markets are thus real markets and the principles of economics should apply to them as readily as they are supposed to apply to any other market.

Plott's responses to editors and reviewers was very forceful, and it appears that more often than not, he got his way. And generations of experimental economists have benefited from his efforts, as by the 1990s economics research using laboratory experiments had been broadly accepted and was regularly being published in top journals. Svorencik's article provides a key insight into how this process happened, and is a really interesting contribution to the history of economic thought.