Introduction
Providing individuals the ability to have their surfing scored by WSL level judges in near real time for all their sessions with only a camera sounds like science fiction. However, with AI we can work towards making it a reality. Significant utility could be offered to elite surfers by facilitating immediate feedback to guide their training sessions. The casual surfer could also benefit from the opportunity to see how they would score against the world's best. In this blog we will discuss how a proof of concept can be built and a roadmap for generalization.
Background: How does the WSL judge surfers?
Surfing is a subjective sport, which makes this task more challenging, the WSL however provides the following criteria that we can use as a guide.
- Commitment and degree of difficulty
- Innovative and progressive maneuvers
- Combination of major maneuvers
- Variety of maneuvers
- Speed, power and flow
Judging Scale:
0.0 — 1.9: Poor
2.0 — 3.9: Fair
4.0 — 5.9: Average
6.0 — 7.9: Good
8.0 — 10.0: Excellent
The Concept
A surfer sets up a camera to record them at commencement of their session. A ridden wave is then analyzed and scored on a 10 point scale in near real time. The ridden wave and accompanying score is saved so that the surfer can review.
Solution Design
The first element of the concept is trivial and third should follow from the solution design of the ride analysis. Our time will therefore be dedicated to exploring how the ridden wave can be analyzed and scored.
This is not a trivial task and can be divided into several problems.
- Real time identification of the surfer and their movements throughout the wave
- Obtaining training data for the scoring model
- Developing the scoring model
- Generalising the scoring model
Problem 1: Real time identification
Identifying the surfer is a fundamental building block which can be used to provide parameters to the scoring model as well as standalone analysis. A proof of concept solution is discussed in this post and demonstrated below.
Problem 2: Training Data
Given the subjective nature of the WSL's judging criteria, a large amount of labelled training data to achieve sufficient pattern matching will be required. An additional issue is the ocean and different surf breaks are inherently variable. The WSL's judging criteria is scaled to these environmental conditions during each contest resulting in additional complexity.
There is however a specific scenario where we can avoid all of this intra day ocean variability, Wave Pools. In wave pools such as the Surf Ranch developed by kswaveco, the same wave rolls through each time providing us with the ideal base to test a proof of concept for our ai judge.
Additionally the WSL has now run multiple contests at the location providing a decent amount of scored examples. Wrangling all of this footage from the internet would however be a huge administrative effort, and if you tried to commercialize would surely bring the WSL knocking.
A better solution could be to partner with other wave pool manufacturers and perhaps government sporting organizations. For example in Australia, Surfing NSW has taken elite surfers to train at URBNSURF. By setting up recording infrastructure at the pool and having the surfers judged we could develop our training data.
The wave pools are also a prime implementation point for this type of technology, they could offer this feature as an upgrade on their existing admission fees. Further positioning themselves as an ideal location for training and many casual groups would likely be interested in competing for bragging rights across the group. Gamification could also be introduced, can you make the URBNSURF top rides leaderboard?
Problem 3: The Scoring Model
The objective of our scoring model can be described quite intuitively, namely it is a supervised pattern matching exercise whereby we want to train the model to predict the associated expert score.
The implementation will however likely be quite involved. Requiring an iterative approach based on the current SOTA in tangential fields and the nature of the data that we can collate.
Insights can likely be drawn from Action Quality Assessment ("AQA") literature and a candidate V1 approach could involve convolutional LSTMs. Below are four recent papers that can be used as reference.
Paritosh Parmar
Author
Xumin YuProblem 4: Generalising the Model
Generalizing our model for different breaks and conditions is a significant challenge given the aforementioned nuances in WSL judging criteria. Contrast this with an AI gymnastics judge where the environment and criteria is constant across locations.
The initial piece of work should not be overly concerned with this task, rather it can be viewed as an extension after successfully designing for a wave pool. It is possible Transfer Learning techniques could reduce the development time for new locations. Specifically the model that is built for the wave pool could be reused with addition of a comparatively smaller new break specific training set to customize the model for the new environment.
Summary
We presented a concept for an AI surfing judge and roadmap for it's development. The development was subdivided into four primary problems and the below recommendations were provided.
- The real time identification of the surfer and their movements can be implemented using instance segmentation
- Partnering with a wave pool is a mutually beneficial approach for obtaining the training data
- The SOTA in Action Quality Assessment presents a candidate for the scoring model
- Transfer learning could be used to generalise the judge to different locations
The AI surfing judge was motivated as both a training tool for elite surfers and novelty for the general public.
Continue the discussion
Feel free to send me an email!
📧 will@willbosler.com
