Saturday, March 15, 2008

Experts

Many systems of governance/management/administration (although not all) rely on elections to select, or partly select leaders/administrators/adjudicators. The English Wikipedia is no different, in that it has a system of elections for administrators, for bureaucrats, and for arbitrators. Some may quibble whether these are pure elections, but there is an aspect of community popular voting to all of them, to be sure.

It is important not to fall into the trap of thinking that because one won a popular vote that one is somehow more qualified on various matters than those that did not. This is an easy trap to fall into, and unfortunately, sometimes, some members of our ArbCom fall into it. The Mantanmoreland case provides some instances of this, in my view.

Here, Uninvited Company opines that the statistical analysis used in the Mantanmoreland case, developed and vetted by many other community members, lacks rigor.

Here, and here, Jpgordon opines that statistical analysis cannot (yet?) be relied upon, but that textual analysis by arbitrators can, and well should, be.

Here, FT2 is giving us a bit of "I know it when I see it..." which should give us pause.

There are many more examples but these suffice. The arbitrators, at least some of them, at least some of the time, think they know more than others about some rather esoteric topics.

Well, I don't agree... not to give the arbitrators too much of a hard time, but no, not always, not by default.

As I said here ... there is absolutely no reason to believe that arbitrators start out knowing any more about anything in particular (just because they won a modified popularity contest) than anyone else about things like evaluating checkuser results, performing and validating statistical analysis, patterns of speech analysis, jurisprudence, or even arbitration, the process, itself.

I suspect I am not the only person that feels this way. For example Alanyst answers Univited Company's opining with some vigor here, saying, in part:

The point is, for those of us who tried to do this the right way -- methodical, objective, transparent, public, open to either conclusion, willing to go where the evidence led -- the message we got back is that such attributes are not desirable in a sockpuppet investigation, and such matters ought to be handled by those with the experience and judgment to make subjective evaluations of private evidence, discuss them in opaque deliberations, and report unattributed findings of inexplicable inconclusiveness. To get that message back from people we generally trust and admire -- the several skeptical arbitrators -- is what remains so difficult to understand.

That is quite strongly worded, if you ask me. But it points to a real problem, but one that's easily fixed if people keep in mind their own limitations. We are all of us human, after all. No one should take offense at being reminded of this.

No comments: