27 Sep 2004

We’ve all heard the phrase “publish or perish.” What we really mean is “publish positive or perish.” Scientists shy away from negative results. But are we doing ourselves a disservice? Are we missing out on a wealth of information? Paul Coleman, University of Rochester Medical Center, New York, agrees that we are, and as editor-in-chief of Neurobiology of Aging, has initiated a policy of accepting negative results for publication. This development will be highlighted soon on the Neurobiology of Aging website, currently being redesigned, and in an upcoming NBA editorial. In the meantime, Alzforum interviewed Prof. Coleman to learn more and get his perspective on the value of negative data. We welcome your comments. The journal welcomes your data.—Tom Fagan.

ARF: Why is it so hard to get negative results published? Are scientists not submitting, are journals not accepting, or is it both?
Paul Coleman: I think both, and there are a variety of reasons, but I would like to focus on two. First, something that does not work is generally considered to be far less interesting than something that does work. I’m reminded of a cartoon I have up in my office in which two scientists are looking at each other in the lab and one is saying to the other, “What is the opposite of Eureka?” Might it be Akerue? However, it is clear that from the point of view of the scientific community, that availability of information about what has not worked would at one level save resources that might otherwise be wasted duplicating what has already been done by someone else. At another level, negative results become part of the knowledge base of the scientific community. As an example, knowing that a drug effect is not modulated by a logically selected enzyme candidate should lead one to consider alternate pathways for the drug effect.

Another reason there has been a bias against publishing negative results has been the concern that if something did not work out, it was because there was a flaw in the study. This is a legitimate concern that can only be addressed by requiring that studies reporting negative results be impeccable. I believe the standard for publishing negative results needs to be exceptionally high. The issue addressed must be a real one and the rationale for the study sound. The study must be conducted according to the highest standards and with a sample large enough to generate confidence in the result. The conclusion(s) reached must be logically related to the data presented. And, the reader must be provided with enough information to allow him/her to reach an independent judgment as to how well these criteria have been satisfied.

ARF: What prompted you to initiate a negative results section in Neurobiology of Aging?
Paul Coleman: I think what prompted me to do it was a convergence of several matters. One was speaking to people working in research and mentioning it would be a good idea to do this experiment or that experiment and hearing from someone else, “Oh, we already tried that.” I would think, “Well, that’s the sort of thing people should know about.” Another was a recent conversation with a colleague who had studied why some corporate research is more successful. He, too, talked about the utility of knowing about what did not work. Basically, it seems to me that there are negative results/data out there that should be made available.

ARF: What’s the main benefit of having negative results published?
Paul Coleman: Well, to put it in its most general terms, the main purpose is to get additional valid data out there so that people are not wasting time, resources, etc. One of the purposes of positive data is to encourage replication, and one of the purposes, or effects, of negative data is going to be to discourage replication, so we have to make sure that the negative data that we present are really solid, that it is well done, that the “n” [number of samples or study subjects] is sufficient so that one can have reasonable confidence in the results, and so on. I think it is perhaps even more important, in the case of negative data, to make sure that the data are really solid, than may be in the case of positive data.

And, of course, as with all data, a negative result can raise further questions that need to be answered.

ARF: Should other journals be doing this?
Paul Coleman: I certainly think so. Absolutely, there’s no question that negative data are important. I think that we really have to be very careful about negative data so it doesn’t get tarnished.

ARF: What kind of submissions are you looking for?
Paul Coleman: Let me make a comment about mechanics. A negative result doesn’t need a lot of space. So essentially, one of the things we have decided is that the published material that appears in print should be one page. But, there needs to be enough information available so that interested readers can assess for themselves the validity of this negative result. So what I propose is that a great deal of the details will appear on the journal website.

ARF: Is there anything you don’t want submitted?
Paul Coleman: We have to be satisfied that the methods have been carried out meticulously, that the “n” is sufficiently large, and that the conclusions the authors are coming to are, in fact, soundly reflective of the data that they have.

ARF: So these manuscripts will be subjected to the same rigorous peer review process?
Paul Coleman: Absolutely, and I might even like to emphasize, the most rigorous possible peer review process.

ARF: How frequently will negative result manuscripts be published? Is there a set schedule?
Paul Coleman: No set concept about that. Because of the one-page limit, we don’t expect that it would cut seriously into our page budget for the types of things we’ve been doing all along.

ARF: How should manuscripts be submitted? Through the same channels as usual?
Paul Coleman: Correct. What authors would need to do is submit two parts. Part one should be what would appear as the one printed page, and part two would be the details that will appear online.

ARF: When is the launch?
Paul Coleman: We are ready any time. On October 15 we are going online for all submissions and review processes. Check with the Neurobiology of Aging website, which is currently being redesigned.

Comments

No Available Comments

Make a Comment

To make a comment you must login or register.

References

External Citations

1. Neurobiology of Aging website

Further Reading

No Available Further Reading