If Research Is Good, Is More Research Better?
Column Editor: Bob Holley (Professor, Library & Information Science Program, Wayne State University, Detroit, MI 48202; Phone: 248-547-0306; Fax: 313-577-7563)
If research is good, is more research better? I’m asking this question after attending the debate between Rick Anderson and Jean-Claude Guedon on scholarly communication during the 2013 Charleston Conference. Anderson was countering the point from Guedon that spending $2,000 from grant funding was an effective way to provide open access. To him, this meant that $2,000 less research would be produced. The assumption behind this assertion was that more research was good, but this assumption isn’t self-evident even if we accept the proposition that research is good. What follows are my thoughts, however naïve, on this topic.
I’m not an expert in STEM (science, technology, engineering, and medicine) so that the first thing I did was ask the Wayne State University Library System science librarian and former student of mine, Jim Van Loon, to see if he could help me discover any published discussion of marginal return on research investment. He volunteered to look for me and found that, while there is significant interest in measuring research output, return on investment (ROI) in research funding has not been widely studied. This result didn’t entirely surprise me since I would expect researchers to avoid questions like this one. In the wrong hands, any answer that too much research could be counterproductive would be a dangerous weapon to cut funding.
If I were to use logic to answer this question, the law of diminishing returns would settle the issue. The Free Dictionary by Farlex states the following: “law of diminishing returns n. The tendency for a continuing application of effort or skill toward a particular project or goal to decline in effectiveness after a certain level of result has been achieved.” (http://www.thefreedictionary.com/law+of+diminishing+returns) I like this common sense definition because it is clear enough to explain the concept while avoiding the complexities of the economists’ definitions about units of production. To apply this law to research, increasing funding for research would be unproductive at some level, at least in the short run, because not enough trained researchers, lab space, and publishing outlets would be available to make efficient use of the increased funding. As was seen in past efforts such as ramping up research initiatives after Sputnik, ways are found to absorb the extra funding, though the argument might still be made about the utility of these heightened efforts. The counter argument to this point is that the United States is in a period of declining funding for research so that the STEM disciplines won’t face the problem of the law of diminishing returns anytime soon.
The issue during the Charleston debate most often revolved around funded STEM research, but research occurs in many other disciplines, some funded and some not. The issue of more research can then become time and expectations. In the Humanities, Social Sciences, and Fine Arts, university tenure and promotion committees are asking for more research because the competition for a limited number of tenured or tenure track positions allows them to increase research expectations. You also don’t have to be connected to higher education to create research. Independent scholars still publish their efforts, sometimes without any expectation of monetary gain but because they are passionate about their subject areas and wish to share what they have learned. With the increased ease of self-publishing, these researchers have ways to publish their research with relative ease and at a relatively low cost. The amateur naturalist or rock hound could even publish non-funded research in STEM disciplines. Is this increased amount of research good or bad? If no one looks at it, it’s perhaps irrelevant.
To continue my naïve view of research, I’m going to divide research into three categories that overlap. The first type is research that satisfies intellectual curiosity with few or no “practical” consequences. Whether or not Shakespeare wrote the plays attributed to him or whether a historical figure was a traitor or a loyalist may elicit great debate but has little impact on the “real” world. I would say the same for literary and fine arts criticism, though both can nurture the human spirit. Whether or not too much research exists in these areas may also be irrelevant since no one needs to pay much attention to it and outside funding is scant.
I would put much of social science research into the second category since it can influence public policy, determine whether someone makes money in the stock market, or has a harmonious relationship with co-workers. Much library and information science research falls into this category, though I’m not sure that the research has made libraries any more effective. The usefulness of this research depends upon its accuracy, its general applicability, and whether policy makers pay any attention to it. Even if well done, this research may be valid only for a certain place or a certain time and will need to be redone as circumstances change. Replication may increase the ability to generalize findings but does not necessarily prove the inaccuracy of earlier research. The practical implications of any such research are often highly debated and often ignored by those who don’t agree with them. To use my favorite example of its imperfections, the stock market may be the most researched topic in the world; but the results of this research seldom guarantee profit over the long run. I would also put much medical research in this second category because microbes and humans change to adapt to their environment. The medicine that worked against a disease a few years ago may now be ineffective against newly-evolved “superbugs.” Another difficulty in this area is the ethical imperative not to harm human research subjects. My biostatistician friend explained how hard it is to compensate for all the possible variables to come up with valid results.
My third area for research is what I call natural laws where the same experiment should come up with the same results. I’d extend this area to engineering principles where the probability of an event happening is similar if the exact physical conditions apply such as stress tests for bridges and other structures. In these areas, an exception to a generally accepted rule indicates that the rule needs to be rewritten to take into account the exception. In this area, replication should apply if conditions are the same. The assumption is that high energy particles should act the same whether the test occurs at CERN or the Budker Institute of Nuclear Physics. Unless the research methodology is flawed or the results are misinterpreted, the findings remain valid and won’t change because of changing conditions.
Whether more research is good or not may also depend upon the nature of the problem. Gregor Mendel and his peas (genetics), August Kekulé and his dream of snakes (benzene rings), and Darwin and the theory of evolution all came about from one person’s brilliant insight. I don’t know if many current discoveries are made in the same way in areas other than mathematics, where I’ve read that brilliance rather than systematic research is needed to solve well-known problems. Perhaps future discoveries are even possible from examining why certain medical conditions are rarer in some places, as was the case in discovering the link between fluoride and tooth decay from the lack of cavities in some Texans. The more common case is those areas where research is needed to test a multitude of possible hypotheses. In medicine, researchers would need to test multiple drugs and multiple procedures to discover which ones are most successful in achieving the desired results. More research is most likely good in that it eliminates the negatives and allows researchers to move forward with positive results.
The final issue is the effect of the scholarly communication system on research. If more good research is needed, does a vast quantity of poor or mediocre research hinder good researchers by forcing them to wade through less than stellar papers? The issue is then how to foster good research and reduce poor or mediocre research, a goal easier to formulate than reach. The cover article of a recent issue of The Economist (October 19-25, 2013), “How Science Goes Wrong,” partially blamed the scholarly communication system for the poor quality of scientific findings. According to the article, the prestigious publications seek to publish “headline” science rather than good science and thus encourage researchers to test novel hypotheses. The article goes on to say that replications of important findings are rarely funded and that articles with negative results are rarely published though these article would be more useful in advancing science.
As I conclude this short column, I wonder why questions such as whether more research is good don’t get asked. The obvious answer is that to do so might call into question the entire system of research funding and scholarly communication that supports researchers, universities, publishers, vendors, and ultimately libraries. A person might suffer damage to her career if the answer were not a call for increased research funding. Several commentators to The Economist article worried that questioning the accuracy of scientific research would “fuel the very problematic anti-science movement.” Perhaps like so many aspects of the modern world, tweaking the current system is more acceptable and productive than questioning its fundamentals. On the other hand, perhaps much is to be gained by asking such basic questions.