NRC Computer Science Rankings Reprise

There's an article in CACM this month by Computer Scientists Andrew Bernat and Eric Grimson, Doctoral Program Rankings for U.S. Computing Programs: The National Research Council Strikes Out. It talks about the ways in which NRC rankings are broken for CS (we have heard this before), but it details ways in which it could be fixed, which we hadn't heard before, and I like.

Two suggestions I thought were good:

• "Explore making the rankings subdiscipline-dependent. It is clear that different departments have different strengths. Thus, enabling a finer-grained assessment would allow a department with strength in a sub-field, but perhaps not the same across-the-board strength, to gain appropriate visibility. This may be particularly valuable for students deciding where to apply."

• "Use data mining to generate scholarly productivity data to replace commercially collected citation data that is incomplete and expensive."

The first is a nice idea; for example, you might be interested in a top ranked department, but it turns out 19/20 faculty focus on Theory and you actually want to do Systems. Or there might be some school with three top faculty exactly in your subspecialty, but you don't see them because they're 93rd in the rankings. 

The second is nice as well; I think with Google Scholar Citations data available this turns out to be a trivially easy problem to solve. 

Maybe CRA can do their own rankings; they collect a lot of their own data anyway, and it avoids needing to rely on the NRC.