If:book has an interesting post about blog carnivals. Ben Vershbow calls them a “looser, less formal peer review” — a way to preserve some of the institutional authority we might find in journals without the long delays of a publication process.
As I pointed out in the comments, I’m not sure how much real “peer review” actually goes on in the world of blog carnivals. I’ve hosted Tangled Bank before, and it was a radical move for me to separate more actively political posts from more standard science posts. Other than that, I didn’t do a whole lot of rejecting. That said, generally the submissions I got were pretty high quality. Journal quality? Not really. After all, no one’s going to publish their original research on a blog. At their best, science blogs are really not much more than lit reviews. They might be inspiration for real research down the line, but they’re not real research.
I’ve spent less time looking at academic blogs from other disciplines, but as far as I can tell, it’s more of the same. People might toss around some ideas, but they’re never going to publish a fully formed, journal-quality argument. The best English blog posts I’ve seen are better than a lot of conference presentations — but that’s a pretty low bar to leap over.
Ben believes that eventually, scholars may look to blogs before they consult peer-reviewed journals. Why not? After all, your research gets out there much faster; science can proceed much more efficiently. It can take months — years — to publish in a journal, but blogs can be up the same day the result is found. What could be better than that?
The problem is, how to distinguish between really good research and the stuff that’s not much better than data noise. If blogs became the currency of academic research, how would we know when an article is Nature or Science quality? How would we distinguish between something publishable in PMLA versus, say, Bulwer Litton Notes? Just because a blog becomes popular doesn’t mean its author is always doing great work — just that they’re more likely to (and better connected).
One possibility would be to add a peer review “layer” to a group research blog. Posts would be private — only available to preselected board of experts — until a consensus was arrived at. Then, when a post was determined to be of acceptable quality, it would be posted publicly. But again, how would we distinguish between the different research blogs? Where would reputations be built? A journal might consider creating a separate research blog of short reports that builds on its good name, but why would it want to do that? The blog is a disruptive technology, and in this scenario, it threatens to subsume the journal itself. Why would a journal give away something it used to charge hundreds, even thousands of dollars for?
The other problem with such a system is that it threatens to get bogged down the same way journals do. Peer review is work. Finding good reviewers takes time. Why should active researchers review potential blog posts at all? I can imagine a system, similar to what many journals do, where a certain number of reviews are required before posting is allowed, but still, these journals spend a lot of money working to find reviewers and analyze their work.
A peer-reviewed blog might be doable, and it might even be better than the current journal-publication system. But it’s probably not going to be any easier.