Open peer review done right

There's been a lot of talk recently on the idea of open peer review, open source science, and so on. arXiv was sort of intended to serve this role, but at least in my subfield it's uncommon to put papers on there before they've been accepted by a "proper" conference/journal. In general we use it for open archival purposes, but not for initial peer review.

Image by Network Osaka

However, I recently saw a really nice use of open peer review. Vinay Deolalikar of HP Labs sent a manuscript to a few of his colleagues which was a proof of P ≠ NP. This is one of the most fundamental unsolved problems in CS, in fact it's such an important problem that The Clay Mathematics Institute offers a $1 million prize to anyone who can solve it. I won't burden you with the details of the problem or proof because that's not what this post is about, and will instead refer you to this excellent summary in New Scientist if you are interested.

So Deolalikar sent this proof to his colleagues, and Greg Baker (with permission) posted the manuscript on his blog. Somehow the blog post got Slashdotted and next thing you know everyone was talking about it. What I find really remarkable was how in many places there was a nice, civil discussion among the theory types about the strengths and weaknesses of the proof. (You can read about this in Richard Lipton's blog starting here).

What I especially loved was the peanut gallery. People were just thrilled to be watching the action. Even people with no background on the topic were offering support and encouragement. Here are some comments from Lipton's blog:

 Go Vinay!

I agree with the spirit of this posting. I am not an expert in complexity theory so I am unable to provide any feedback (not even elementary). However, I like it that there are researchers like Vinay who are not afraid of tackling the hardest problems. I am anxious to know what are the consequences of Vinay’s work.

-Anon@ 11:23pm

And these two really made me say, "awwww" (boldface is mine):

Professor Lipton,

Your accessible and well-written writings motivate me to continue studying mathematics. I develop software for a living; to stay sharp, I try to study ‘real’ proofs in my spare time. Complexity is an amazing field… I cannot wait to see what we’ll learn about computation (and its relation to physics, and to everything else).

Thanks for the inspiration!

-Robert W 

“P.S. I hope these discussions are helpful to the community at large.”

This member of the community at large can’t understand a word you say, but is nevertheless fascinated by every new post and comment. Seeing the review process unfold in public has rekindled my long-dormant interest in mathematics. I intend to register for a class this fall and (who knows?) perhaps pursue that Ph.D. after all these years.

Don’t infer from the paucity of experts who can contribute to the public conversation that you might as well confer entirely in private; on the contrary, public discussion is an immense service to the community. Thank you.

-Fritz Juhnke

I applaud Lipton for cultivating a positive exchange of ideas and place for civil discussion and making the review public for all to watch. Even though things got heated at times, in general the discourse was constructive and positive. Top complexity theoreticians came together to help Deolalikar out, for no reason other than they think it's a cool problem and they're nice people, and I think that's just fantastic.