This idea is applicable to many fields, but in the era of the internet, this is a cheap idea that should gain broad acceptance.? Academics would benefit from the creation of a journal of failed research.? Rather, many journals of failed research, Chemistry, Economics, Biology, Sociology, Finance, etc.? There should be buoys in the harbor saying this way does not work; go another way.? There would be three salutary effects:
1) Researchers would learn of ideas that don’t work and would avoid them.
2) Researchers would conclude that your process does not work, but they have a better way to proceed and act on it.
3) Academics would get credit for doing honest research, and not twisting research through falsifying data or tweaking formulas in order to get significant coefficients.
It is almost as valuable to know that something doesn’t work, than to know that is does work.? How much time could be saved, and new avenues acted on, through journals that record failed research.? Who knows, but that it might improve honesty among scientists, if they get credit for publishing failed research that is honest, versus falsifying data or engaging in a specification search in order to tweak coefficients to make them significant.
This would be a big improvement for every academic discipline worth writing about, where data and fair results matter.? Let it happen then.? I am willing to set up online journals for failed research.? Let the submissions begin.
I agree. this would instigate a lot of interesting discussions which are currently only the fodder of arguments between grad students and their advisers, but really should be universal.
Not exactly the same thing, but some students from Rice University started something similar for publishing rejected math papers.
http://math.rejecta.org/
Actually the same point is made on the science blogs I read. They complain that it is much harder to get an article published in a journal if the article is disproving a hypothesis or even bringing sufficient evidence for doubt. There is only so much space, blah, blah, not as exciting, blah, blah. Never mind the fact that the few folks that actually read those journals sure are interested in the evidence, data, etc.
There is a video of Monish Pabrai discussing a similar idea, and transmuting it into a checklist to use as part of an investing process, much like the aviation indsutry does
http://www.morningstar.com/cover/videocenter.aspx?id=337119
Great idea David. Seems like it would be a formidable editorial challenge, but well worth doing.
The reason why such a thing does not already exist is because Academics don’t want to/can’t handle the criticism. If tearing-apart others’ work in a public manner happened even to a small degree they’d all fear for their egos and likely inflated reputation, if it were commonplace they’d be so afraid to put their work out there for fear of retribution to the point no one would publish anything at all! Just my $0.02
FYI, The Medical Hypotheses Journal editor Bruce Charlton was ousted by the PC police. Basically, it was not a peer reviewed journal, so the editor could allow dissenting views. This was not well tolerated by the ?scientific peers?. Anyway, those interested can read what Bruce Charlton has to say at his blog by the same name, Medical Hypotheses.
http://medicalhypotheses.blogspot.com/
I have a PhD (1979). Peer reviewed journals don’t like to publish research that fails to support or reject a hypothesis. Notice that I said it both ways. They don’t/won’t publish no results research. They should. I don’t know if its the editors or the referees – probably both. There’s this attitude that rejection or acceptance of hypotheses adds to knowledge, but that failure to reject or accept does not. Will anything be done about it? Nah. At least not in my lifetime. There’s no money in saying we don’t know. Who will buy a service that admits we don’t know how you can beat the market?
BTW I’ve beaten the market quite well over the years, not all years, but on average. And I have no wish to publish or sell my knowledge.