As I write this there is a local magic competition tonight, which I would have participated in were it not for my recent hospital stay. It’s a magic competition I have had an interesting history with, as a participant, organiser and rule setter. My disappointment with its fairness in the early days also nearly got me ejected from the club.
The thing about magic competitions is that as a performing art, appreciation of any given entry is entirely subjective. Generally a small number of judges are appointed and the rules set out criteria to them judge the competitors on. These judges will have different ideas of what counts as suitable attire for a performer, what is entertaining, what is mystifying etc.
Skill should be an objective measure, as is seen in gymnastics competitions, except when Simone Biles does a backward double-twisting double tucked salto dismount (thanks Wikipedia), everyone sees it. If a magician does a multiple diagonal palm shift and everyone sees it it wasn’t very good. One year the organisers had to tell the judges that a particular routine was entirely self working, because having seen no sleights at all the judges were under the impression that they were in the presence of an absolute master of prestidigitation. This disclosure got back to the competitor and caused no end of problems as it was seen as the organisers introducing bias to the judges.
Is there a way to fix all this?
Well sure there is, but first we need to back up and explain how we got here.
I entered the closeup competition twice in its early days. Back then we ran the competition in a pub, and every table we performed to would have a lay judge and a magician judge, as well as a lay audience. The magician judges were told to judge performers on originality and skill as well as entertainment, presentability and overall quality of the magic. When all these scores were decided they would be taken aside where a winner was decided… and that’s literally all the rules said.
How the scores of six judges working from different criteria were boiled down to a single winner was never defined and rumour has it that behind the scenes a lot of executive decisions were made by competition organisers, overruling the judges, deciding that magician judges were more important than lay judges because they could give more points.
When I publicly mentioned how bizarre this wording was (along with my disappointment and musing upon the subjectivity of the decision and a lack of feedback) this was interpreted as sour grapes from a loser, by both the organisers and the winner, who still isn’t on good terms with me nearly a decade later. My facebook post was also found by another performer, who was definitely feeling sour grapes and attempted to use my comments to burn the club to the ground (metaphorically).
The next year no one wanted to organise a competition, lest they whipped up a similar ire. Not wanting the club to go without competitions, I offered to run the next competition – on the proviso that I could write the rules of how the winner was decided, ensuring fairness.
Here’s the thought process I went through:
Have you ever watched Come Dine With Me?
Come Dine With Me is a cookery challenge like no other.
In come dine with me five contestants take turns hosting a dinner party where the other 4 are guests and, ultimately, judges. What makes this so beautiful to watch is that occasionally a contestant will realise that as the scores are all just added together, if they can make up a reason to give the other four really slow scores, they can drag down their totals. Provided no one else is taking the same strategy, this means that your own score will average out higher if the meals are the same quality. Of course competitors cannot inflate their own score artificially, because they don’t score their own night.
But in a magic contest, a judge’s bias can disproportionately affect the outcome. If, for example, one of the lay judges was related to the performer, a system of totalling scores can be abused. Consider this:
If you have 3 judges and score out of 30 per judge (this is the situation with one contest I’ve participated in) you might get 2 judges who give everyone between 20 and 30 points to be nice to them, meaning the average score will be 25. A third, biased judge however may give their favourite 30, and everyone else less than 10. This would shift the average of their favourite to 27.5 whereas the average of the others drops to more like 20. What this means is that the winner is decided by one judge¹.
I have explicitly given an example with a biased judge, because while the competitions I’ve described above didn’t have biased judges to my knowledge, but another club I’ve competed with had one clearly biased judge, because the winner 3 contests in a row was a competition lay judge’s son.
When I ran my contest I knew I was going to be pulling in external judges, magicians from other clubs, lay judges from my non magician friends. None of them knew any of the competitors. But I knew that this wouldn’t be enough. You see the scoring patterns above are not just ‘fair’ and ‘biased’. There are, to my mind, 4 kinds of judges:
- Kind
- Harsh
- Fickle
- Biased
A kind judge wants everyone to do well and gives everyone high scores.
A harsh judge can find fault in everything and gives everyone low scores.
A fickle judge will score some people high and some low, based on the performance.
A biased judge will score some people high and some low based on their personal preference.
Getting rid of biased judges is not enough. A panel of kind of harsh judges is fine, because every contestant is run from the same baseline. A panel of entirely fickle judges is fine, because the contrast of the scores will average out. But if you have a panel of all 3, the kind and harsh judges are redundant, the winner is ultimately decided by the fickle². This is unfair to the contestants, who are relying upon the tastes of a single judge, and also unfair to the other judges who took time out of their day to participate in a process where ultimately their contribution was mostly ignored.
Fortunately, I have been in a lot of competitions, and I know that there are other ways of doing things.
I have participated in mass competitive RPG events where teams of 30 people turn up, each one participates in a single game with 5 other people from other teams two days in a row, and the judges each day give a gold, silver and bronze to 3 players of their game. At the end of the weekend these are converted back into points and a team is declared winner or loser based on a score of 5 points for a gold, 3 for silver, and 1 for bronze. The winning team hosts the next competition a year later, which led to the motto:
Having seen a similar system play out on a smaller scale however, one thing became obvious. It was possible for a contestant to win on silver medals alone³.
It was for this reason that I created the Trickle Down Medal system for determining the winner in a magic competition.
How does TDM work?
The only restriction for judging using TDM is that every judge has to have a clear 1st 2nd and 3rd place in their scores, so that you can issue a gold silver and bronze medal per judge.
NOTE: No physical medals are used. These are just terms to use on paper when explaining this…. that said, you could easily use real medals or tokens in the process.
Additionally every person in the audience writes the name of their favourite act on a piece of paper, and these are collected.
- Each judge awards a gold silver and bronze to their 1st 2nd and 3rd favourites.
- Each contestant is awarded an additional plastic medal for each audience member that favoured them.
- Each contestant has these totalled up.
- If one contestant has the most gold, they get first place. If two or more are tied, the tie is broken by silver. Failing that, it is broken by bronze. Failing that you turn to the plastic. If the plastic is tied, ask everyone to give an additional plastic medal vote for their second favourite. Every round of public voting makes a tie less likely.
- After a winner is picked, all other gold medals are given a value equivalent to 2 silver medals, and the most silver medals is used for 2nd place. If there is a tie*, use bronze. If still tied use plastic.
- After 2nd place is picked, all silvers are worth 2 bronze (and gold is worth 4 bronze due to the commutative property) and the most bronze is used to pick 3rd place. If there is a tie*, use the plastic.
* Not all contests need a definitive 2nd and 3rd place, and these can be awarded jointly. Some places only have a trophy for first, others have a trophy for second and even third, so ties must be broken here too.
I wrote these rules and I got them agreed upon by the committee, and the competition went great, except that that was the year half the contestants pulled out and of the remaining three, the one generally considered the ‘best performer’ by the club tried to do a really complex stack routine and messed up his stack twice. Meanwhile a competitor most of the club thought of as a joke did three solid self working effects and was favoured by the judges. The judges did not know that the guy who floundered and messed up 2 stacks was our ‘best performer’ because like I said, I got external judges who’s only impression of the contestants was the 10 minute set presented on the night.
This is what a totally fair competition looks like.
The rest of the club committee was furious.
At the next committee meeting many harsh words were spoken and the secretary vowed to revisit the competition rules, giving magician judges strict guidelines on awarding points according to demonstrated skill, with a list of sleights by their level of complexity offered as a scoring chart. As I pointed out, if the judges actually see these moves in action they haven’t been done very well, and so I wanted nothing to do with the next competition.
Fast forward and now the committee relies upon me to run the process I outlined above, and in my absence they wanted to know if I could make them a spreadsheet to automatically compute the winner. Instead of this (programmable spreadsheets are rarely interoperable across platforms) I decided to make a webpage to do all the work.
The competition calculator is basically open source and you can find it here:
Try CompCalc Now!
I’ve included examples of how these scenarios might play out as footnotes:
¹ One biased judge, 2 regular judges
| RJ | RJ | BJ | Tot | |
| C1 | 14 | 19 | 5 | 38 |
| C2 | 18 | 15 | 7 | 40 |
| C3 | 17 | 14 | 29 | 55 |
Competitor 3 wins due to one judge’s bias, even though the other two judges thought them worst on average.
² One kind judge, one harsh judge 1 fickle judge
| KJ | HJ | FJ | Tot | |
| C1 | 27 | 11 | 6 | 44 |
| C2 | 26 | 12 | 18 | 56 |
| C3 | 25 | 9 | 28 | 62 |
Competitor 3 wins entirely due to the fickle judge, even though the other two judges both thought they were the worst.
³ 4 judges, 4 competitors, each judge gives Gold worth 5, Silver worth 3, Bronze worth 1
| J1 | J2 | J3 | J4 | Tot | |
| C1 | G | G | 0 | 0 | 10 |
| C2 | 0 | 0 | G | G | 10 |
| C3 | B | B | B | B | 4 |
| C4 | S | S | S | S | 12 |
Competitor 4 wins by silver medals alone, whereas the 2 competitors with gold medals get relegated to joint 2nd. This scenario is mostly likely when you have more judges and each judge is seeing a different instance of the competitor, either because each judge is seeing a different member of a team⁴, or if competitors perform a number of sets and get judged for each individually.
⁴ Real talk though, this method may actually be better for evaluating teams where 100% disconnected overall performance is far more valuable than one or two stand out members. For individuals giving multiple performances, it doesn’t make sense to me that someone who was thought to be the best by multiple judges would lose out to someone who wasn’t considered the best by anyone.