≡Score for Score

Lately, I’ve been thinking about how to measure a gymnast’s consistency empirically.

When we think about whether a gymnast is consistent or not, we usually thing of one thing: how much does she fall? So one simple way to measure a gymnast’s consistency is to measure the percent of routines that she falls on.

However, we all know that there are plenty of mistakes that can add up to more deductions than a fall even if you manage to stay on the apparatus. Two big breaks on beam are as bad as a fall. A close catch on bars that puts your next handstand under by 45 degrees is as bad as a fall. I would still say that a gymnast is inconsistent if she rarely falls but frequently makes this sort of major mistake.

So, another way to measure a gymnast’s consistency is to look at the variation between her final scores. I would do this by taking the standard deviation of all her scores, or the average difference between each score and the average score.

However, I don’t think this applies straightforwardly to gymnastics. Sometimes a gymnast will chose to compete a lower level of difficulty at a less important competition, which would create variation in her scores. However, we would never want to say that Oksana Chusovitina is inconsistent just because she doesn’t throw a Rudi at every competition.

So instead, I’d propose ignoring the difficulty entirely and measuring the standard deviation of execution scores alone. This might not work so well for beam, where making every connection is a sign of consistency, but I think across the board it’s better looking at the standard deviation of the total score. Also, I’d probably include neutral deductions as a "deduction" in the execution scores so that they’re included in our measure. So, if a gymnast got an e-score of 8.5 and landed with both feet out of bounds, I'd be interested in her "score" of 8.5-0.3=8.2.

Finally, it’s hard to compare scores across different events – I wouldn’t want to classify Simone Biles as inconsistent just because there’s a lot variation between her vault scores and her beam scores. So my preferred metric of consistency is as follows:

1. Subtract the d-score from the total score, leaving the e-score minus any neutral deductions.


2. Calculate the standard deviation of each score on each event.


3. Take the mean of the standard deviation for each event.

Let’s take a look at this measure for a single gymnast. It would be most interesting to look at someone who 1) competes a lot, and 2) has a bit of a question mark around her consistency. There’s one obvious candidate: Angelina Melnikova.

At the start of Angelina Melnikova’s career, she looked like Russia’s Next Big Thing. She ran away with the junior European title in 2014. She unofficially won the Russian Cup all-around title in 2015 before she was old enough to be eligible. And in 2016 her first year as a senior, she won Russian Championships, the Russian Cup, and the unofficial all-around competition at the European Championships.

However, she had a disappointing showing in Rio, getting two-per-countried out of the all-around final and failing to make a single individual event final. Though she competed many, many times through the end of 2016 and throughout all of 2017, she struggled with consistency.

This year, things have felt different. She snagged her first-ever World Cup win in Birmingham. She won four gold medals and a silver at Russian Championships. She came away from the European Championships with three medals to her name.

Angelina Melnikova at the 2018 Birmingham World Cup

In other words, she seems to have shed the inconsistency that’s plagued her entire senior career. And with a very competitive level of difficulty – and possibly even a Cheng! – she’s looking like a real contender for Doha 2018.

Let’s look at the numbers! I would have loved to compare Melnikova’s junior and senior performances, but unfortunately I only have data for 2017 and 2018. So let’s see if the gymternet consensus holds up: has Melnikova been way more consistent this year than she was last year?

The answer is a resounding yes. The table above shows her consistency score on each event, as well as her overall score. Remember, a larger score means more inconsistency. Melnikova’s consistency score this year is half her score last year.

Intuitively, you can think of it this way: in 2017, Melnikova’s score on any given day was about six tenths above or below her average score – so her scores were pretty spread out. In 2018, however, her scores are much more closely clustered around the average.

Her improvement has been driven largely by a big improvement in her floor consistency – but she’s showed gains on every event except beam.

So, Melnikova is one example of this metric cohering with our general understanding of consistency in gymnastics. I’d love to hear your thoughts on this measure. If we can’t think of anything better, then I’ll do a longer post some time looking at consistency for a bunch of gymnasts.

Tags: Fun with Score Data