I was disappointed by this paper, which is to say that I was excited by the premise but felt let down by the execution. Here are some of the places where I think they went wrong:
This is fatal for Farrell and Shalizi's main arguments in favor of democracies because needless to say, it does no good for democracies to excel at combining information that they don't have.
Here's an example of the sort of thing I expected and would have liked. Consider the "who has the button?" problem. In this problem, there are n people. We will call out two of those people as person A and person B: person A has the button, and person B wants it. Only person A knows that A has the button, and the problem is deemed to be solved if person B ends up with the button.
Hierarchy will solve this problem only rarely. If we assume the hierarchy is a tree with d children at every interior node, then person A will on average have approximately log_d n "superiors". The problem gets solved only if person B is one of them, and (log_d n) / n is pretty small for even medium sized n. A more realistic model would also take into account the costs of A searching all of his "inferiors", which would happen on both successes and failures ...
Markets will solve this problem if the value of the button to A is greater than the value of the button to B plus transaction costs. This remains roughly true even in a networked setting without a central auctioneer (Robert Axtell, "The Complexity of Exchange", http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0297.2005.01001.x/abstract;jsessionid=C4944AF2AD52BE909CC4D6F0AD0A8FE5.d03t03 ; I got this reference from one of Cosma's previous posts to Crooked Timber which is why I was expecting "Cognitive Democracy" to be full of this type of juicy result.)
And when will democracy find the button? I don't honestly know. Maybe when person B can convince 1/2 n people to vote to give him enough money or power to obtain the button? The "or" in the previous sentence suggests that there are actually two different solutions, call them democracy+market and democracy+hierarchy, that succeed only when B can solve his voting problem plus when the market or hierarchy component can solve its part. To be fair, the market or hierarchy component can have different parameters than in the above analysis -- in the best case, B can get herself "root" access to the hierarchy or money equal to the social value of B getting the button.