≡Score for Score

Yesterday, in Part I of my mini-series on artistry in gymnastics, I gave an overview of what artistry deductions looked like at the Rio Olympic Games. Today, I’d like to delve a little deeper and examine a different question: do the scores suggest that the Code of Points - and the judges who apply it - have a coherent understanding of artistry?

If we have a coherent understanding of artistry, then judges should be able to take artistry deductions consistently and objectively regardless of their subjective taste. There are plenty of masterpieces out there that I can recognized as great works of art even though I do not find them pleasing or moving - Picasso’s Les Demoiselles d'Avignon and Stravinsky’s Rite of Spring come to mind. In other words, they have high level of artistry even though they are not to my taste.

Gymnastics judges are not supposed to answer the question, “Is this art to my taste?” Instead, they are simply supposed to determine, “Is this art?” Now, I’m certainly not going to venture a definition of art myself— in other words, I’m not a college junior sitting in some art history seminar. But what I am going to do is examine these artistry deductions for the sort of consistency we would expect if all the judges were working off a similar implicit understanding of what is and is not art in gymnastics.

Judges Seem to Agree About Artistry Deductions

If different judges given the same routine wildly different artistry deductions, then they clearly do not have a common understanding of each deduction. If they do have that common understanding, then we’d expect to see them all take about the same number of points.

To measure this, I took the standard deviation of all five judges’ scores for each floor routine. (I’m only going to look at floor, because floor routines tend to be more varied and controversial.) You can see the distribution of the standard deviations in the figure below.

So most of the time, the average judge takes deductions that are about half a tenth off from the average deduction across all judges. In other words, the judges pretty much agree on the total amount of artistry deductions. (Of course, the average gymnast gets two tenths off in artistry deductions, so half a tenth is not nothing - but the disagreement between judges is ultimately not really impacting her score.)

Interestingly, the most controversial routine - the one with the largest spread between the judges’ artistry scores - was that of Catalina Elena Escobar Gomez of Colombia, who got injured during her routine in Rio and had to leave the floor. One judge took a full 0.4, while the rest took basically nothing, presumably out of pity. I admire that one asshole judge’s commitment to accurate scoring over feelings.

So if judges seem to have a coherent understanding of artistry, can we get any closer to figuring out what that understanding is?

Is Artistry Just Execution?

One strong possibility is that judges fall prey to the common tendency to blur artistry and execution. You see this sometimes in the discourse around certain gymnasts; for example, some people seem to think that America hasn’t had a truly artistic gymnast since Nastia Liukin.

Below, I’ve plotted the artistry deductions versus the other execution deductions on beam and floor. As you can see, the two are very closely linked. The correlation between artistry deductions and execution deductions on beam is 0.71. Roughly speaking, this means that 71% of the variation in beam artistry deductions can be explained by the variation in beam execution deductions, which is very high. On floor, the correlation is 0.50.

Now, it’s totally possible that artistry and execution are two totally separate concepts and, for whatever reason, gymnasts who are good at one tend to be good at the other. When you listen to the way Eythora Thorsdottir talks about gymnastics, it’s clear that her love of beauty in sport drives her both to perfect her extension and toe point and to treat us to an expressive performance. While some expressive gymnasts can’t quite figure out their landings and can’t stay on the beam, I could not think of anyone with fantastic performance quality but a lot of built-in form deductions.

But I don’t think the two are inextricably linked. While no highly artistic gymnast has poor execution, plenty of gymnasts with flawless execution have very little artistry. I’m not going to name names, but you all know the type, and there were plenty in Rio.

If that’s the case, we should expect to see a lot of points in the bottom right corner of these graphs - high artistry deductions, but low execution deductions. I don’t see a single point there at all.

All this leads me to conclude that the shared common understanding of artistry promoted by the Code of Points is “consistently lovely execution that contributes to a nice overall look of the routine.”

I don’t know exactly what are we scoring when we assign points to artistry in gymnastics. And I don’t know exactly what we should be scoring, either. But I think most gymnastics fans agree that there’s a gap between what we have and what we want.

Tags: Fun with Score Data, Olympic Talk, On Artistry