Fuzzy math involved in jurying your art?

I got this email from a very concerned Harriete Estel Berman. I’m sharing it here in hopes that those deeply interested in this topic will figure out the math involved!

She writes:

Using the jury system 1,2, 4, 5, is not more effective! In fact, it is more inaccurate than 1,2,3,4.   Artists just don’t understand the principles of math and often suggest 1,2,4,5 (omitting 3) as more effective. THIS IN NOT CORRECT! I am hoping you can inform your readers and disspell the myth of bad math!!!!

Below is an explanation. (Note from Alyson: Click here to download an entire PDF of the explanation, which goes into the various jury ranking systems. Harriet says the best system is 1,2,3,4,5,6,7.)

HERE IS AN EXPLANATION
1, 2, 4, 5
Sometimes it is suggested to use a system that removes the middle number from a 1 to 5 system (e.g. eliminating the “3”) and to use only 1, 2, 4, 5 to “force” a selection outside of average. However, using fewer score choices increases the number of possible ties (regardless of the number of jurors). Mathematically, there is no difference between 1, 2, 3, 4 and 1, 2, 4, 5 since the number of sums (outcomes) is identical. (see below for a more complete explanation of the problem with the 1, 2, 4, 5 ranking system.)

THE PROBLEM WITH THE 1, 2, 4, 5 RANKING SYSTEM
When using the 1, 2, 4, 5 ranking system (leaving out the score choice of “3”) the range appears to be larger because the highest value is “5.”   However, as a juror, you still only have four score choices. As a result, the number of possible outcomes is identical to any other system with only four score choices. This is a definite disadvantage.

Another problem with 1, 2, 4, 5, is excessive weighting to a dissenting juror. If one juror gives an unusual score using the 1, 2, 4, 5 system, it outweighs the judgment of the other jurors. An example is illustrated below.

Example 1)  ranking example:    4 + 4 + 4 = 12

Example 2)  ranking example:     5 + 5 + 2 = 12

In the first example, (4 + 4 + 4 = 12), these uniform scores demonstrate unanimous opinion that the quality of the work is “medium high.”

The second example, (5 + 5 + 2 = 12), demonstrates that if two jurors think the entry is of the highest rank and one juror gives it a “medium low” rank, then the work appears to be ranked equally to the  (4 + 4 + 4  = 12) example.

In this final example,

Example 3) ranking example:     5 + 5 + 1 = 11

In the third example, two jurors judged the entry at the highest rank but one juror ranked it the lowest. As a result, one juror’s vote skewed the score. This gives one juror‚s judgment too much weight, since two jurors cannot overrule one juror using the 1, 2, 4, 5 system. It is doubtful that this inequity was the intention of any jury.

Using a 1, 2, 3, 4 system is not an ideal system either because it only gives four choices but it is better than the 1, 2, 4, 5 system. Why? Because every juror’s opinion is equally weighted. Using 1, 2, 3, 4, 5 is better than a system with only four ranking choices because it uses finer increments for scoring. The jurors must make an effort to use the entire range of the scale. The 1, 2, 3, 4, 5, 6, 7 system is the recommended ranking system since it uses even finer increments without being unnecessarily burdensome.