A More Accurate Way to Measure Player Performance   

Derived from the work of Rachel Marty

This blog was written based on research conducted by Rachel Marty, which is represented in her research paper, “High-Resolution Shot Capture Reveals Systematic Biases and an Improved Method for Shooter Evaluation.” This paper was submitted to the MIT Sloan Sports Analytics Conference Research Paper Competition and presented by Rachel at the 2018 MIT Sloan Sports Analytics Conference. Rachel’s research shows that multidimensional tracking is a much more effective way to evaluate shooters than shooting percentage alone.

One of the most crucial aspects of the game of basketball is analyzing, comparing and contrasting shooters. Player analysis factors into every aspect of the game – recruiting, player development, strategy and more.

It should come as a surprise, then, that the current way we evaluate and analyze players is largely inaccurate. Due to limited time and a lack of proper equipment, players are typically evaluated based on one single metric taken on a limited number of shots, typically measured in one setting. This exposes the collected shot data to high variation, sampling error and, as a result, hasty conclusions that may or may not be accurate.

For example, imagine you’re tracking performance and ranking three players, Player A, Player B and Player C. Each player has 25 shots to take, all of which are tracked for your use in ranking.

A player’s shooting performance in a 25-shot session tells you something, but it doesn’t tell you everything. What if Player A was having an unusually great session? What if player C was having an off day? Even the best shooters are inconsistent depending on the day, time and situation.

Noah Basketball has tracked over 22 million shots taken by over 500 NBA, college and high school players. Using data from Noah Basketball, Rachel Marty has proposed a better, more accurate and efficient way to track player shooting performance. As Rachel’s research shows, multidimensional tracking is a much more effective way to evaluate shooters than shooting percentage alone.

Rachel decided to test the ranking scenario mentioned previously. She took Player A, Player B and Player C and tracked their shots over 10 months. She took the shot data provided by Noah Basketball and pulled out sessions of over 500 shots for each player. Then, she divided this large session into smaller sessions. For example, if she took a session of 600 shots, she would separate the data into 24 25-shot sessions, 6 100-shot sessions and 1 500-shot session to track player performance during the session.

Rachel’s data showed that discrepancy between player performance over a 25-shot session was relatively low, whereas player performance over a 500-shot session was much more defined. This means that players can essentially “fake” their ability and consistency in a set of 25 shots, but they can’t fake it over time. A poor shooter can appear to be a decent one, whereas a great shooter who is having an off day can look subpar.

Using Noah’s technology, Rachel found that player performance is most accurately tracked in sessions of 1,000 shots, where patterns are shown and shooting “good days and bad days” are leveled out. With the Noah system, tracking 1,000 shots per player is easily possible. It is not, however, a time efficient or realistic substitute for the current method of player shooting analysis.

Rachel’s proposed solution is to use better tracking methods for these small sample sizes. Rather than tracking only makes and misses, Noah Basketball is able to track 9 measurements for each shot, including metrics like left-right position, depth and arc.

Using these additional measurements, players can be more accurately ranked after shooting sessions with less shots taken. This conclusion is based on Rachel’s comparison of high shot sessions without multidimensional tracking to smaller shot sessions with multidimensional tracking. The data shows that Noah Basketball’s multidimensional tracking is able to better predict shooting ability, even if a good player is having an off day, or vice versa.

So, say Player A is on fire during a 25-shot session, but is incredibly inconsistent and tends to miss the majority of shots tracked over a period of time. Tracking multiple aspects of the quality of players’ shots, Noah Basketball is able to determine whose form and shot style is most consistent with quality shooting.

Noah Basketball’s motto is help players make more shots and win more games. By helping teams, coaches, scouts and players better analyze performance, Noah Basketball is creating better shooters and better teams.

- - -