Kitman Labs

Player Assessments


We used a design thinking approach to develop a feature for backroom staff at sports teams to rate their players as part of a review process.

Step 1.

Understanding


The initial idea for the feature originated when we signed a partnership with Chelsea F.C. As part of their player development process they set player goals and review progress over time. As we develop a generalised product we wanted to be certain this feature would be useful for all our customers, so we reached out to a broad sample, specifically performance oriented staff, from different sports, in a number of semi structured interviews. What we quickly found was that this process of setting goals as a method for developing players is a common approach. Along with that, the process was similar, with only slight variations in the frequency on reviewing and the level of detail documented.

For example, a team like Chelsea F.C. (English football team), have a lot more time and staff available to work with each of their players, thus the goal setting process is very curated and progress is regularly reviewed every 6 weeks with a number of staff members. This is done based on three specific goals per player along with looking at a standard set of metrics, like Energy, Training Performance, each of which has definitive definition. A smaller sized team like Stellenbosch (South African rugby team) who have half the number of staff create a set of goals for players but they are selected from a predefined list, the goals are not particular to the players exact development needs, the regularity of review is extended, twice a year, and the rating is not indepth.

Another interesting finding pertained to the level of formalisation of the process at different teams. Generally, we found many teams were not as developed as Chelsea F.C. Nothing was operationalised at many of the teams we spoke with. For example at Saracens Rugby (English rugby team) and the Ducks (American hockey team), each staff member had a different approach to working with players and conducting the review process, even within the teams. At Chelsea F.C., though, things are rigorous, thought out and the process is standardised. They are also considered the benchmark for player development in sport. This highlighted an opportunity, to give guidance and reflect such a process in our software and offer it to customers.

Journey Map


Player Assessments Journey Map

Personas


Player Assessments Coach persona
Player Assessments Sports Science Persona

Step 2.

Definition


01

Schedule

A Sports Scientist needs a way to plan and schedule an assessment in advance so staff members can know what they must do and when.

02

Prepare

A Sports Scientist needs a way to prepare for an assessment so they can come to the meeting informed.

03*

Rate

A Sports Scientist needs a way to rate an athlete so they can document how well that athlete is developing.

04*

Discuss

A Sports Scientist needs a way to see other staff ratings so they can discuss those ratings.

05*

Visualise

A Sports Scientist needs a way to plot ratings using different visualisation types so they can analyse the development of an athlete.

06

Share

A Sports Scientist needs a way to share rating reports with staff, athletes, teachers and parents so they can keep them informed and engaged.


As part of the project we decided to concentrate on Needs 3, 4 and 5. We have a calendaring tool, so point 1 was covered. Point 2 we felt we also covered in the graphing area, all metrics and goals, if input could be visualised per player in this area, so we had covered that need. Finally, sharing the report was also a point we had already covered in our messaging feature and our download to pdf feature. The alternative would have been to build a full end to end flow independent of the current software system, but that would have added a whole new area in the system, which is not a scalable approach, it also leads to confusion for the user, as multiple similar locations would have lead to cognitive load and possibly paralysis, if even only briefly.

Step 3.

Ideation


Once the needs were identified we started moving into an ideation phase. A number of options were identified. Two concepts stood out, but neither was an obvious selection.

Concept 1 - Grouped

The idea behind Concept 1 was an ability for staff to enter multiple ratings against a number of different players and variables quickly. While also being able to reference other players.

Concept 2 - Focused

Concept 2 isolated the view to a specific player, which also incorporated a historical view, where staff could see previous assessments and could make reference to those previous scores.

Prototype link

Step 4.

Usability Testing


Prototype link

After running a set of usability tests, with a set script, we got clear signals that the Concept 2 - Focused was by far an ideal input mechanism and the historical aspect allowed for reference of previous ratings. What was interesting with regard Concept 1 - Grouped, was how useful this view could be at a general meeting when comparing players and reviewing averages for each player across a season or longer time frame. We decided to build the Focused concept and to later develop the table Grouped concept in the graphing area. A number of smaller interactions were also identified as needing to be updated.

Updates


1. Expand all function added

2. Add comment and staff member to a metric, remove talking points

3. Categorise metrics, sections added

4. Link back from reports section to the forms

High Fidelity


Step 5.

Development and Monitoring


As part of the build process, I started working on high fidelity designs. The engineering team had already seen the designs, I generally show the engineering team the designs before going to customers with any work, just to reduce any risk the designs are not feasible to build. This share also acts an opportunity to get engineering feedback. I like working closely with both the back end and front end teams. Once we formally started development we just worked via stand ups and two–week sprints with a demo at the end of each sprint. I also tend to QA, from a design perspective, as we go. Listing any issues in Github.

Through customer call sit ins, Intercom research, and analytics we saw that many different use cases started to appear. Medical staff are creating medical assessments for rehab programmes, Strength coaches are creating physical pre-season benchmarking assessments and Scouts are using it to review possible transfer targets.

Curiosity pushed us, later, to speak to the Premier League. What we learned was the Premier League was also trying to standardise the development process and had learned from Chelsea F.C. They had commissioned a sport software company, The Sport Office, to do this on behalf of the Premier League and push it out to every team in England. We subsequently bought The Sports Office.