ArbiterSports builds sports management software to help the NCAA and other sports associations and governing bodies to find, schedule, retain, and pay officials and other event workers.
This is the story of how I brought stakeholders into my process, and how, together, we created a multi-year strategic plan to combine 7 disparate products into 1 seamless user experience for our customers. We also designed and built the first phase of this plan, which reduced Eligibility set up time by 5,000% and was a major contributing factor in landing the 2nd biggest deal in company history.
Over nearly 30 years, Arbiter had built many products for various personas: governing body admins, referee assigners, officials, athletic directors, finance, etc. Each had their own product brand, and in some cases, completely unique domains.
These products, while intended to be complimentary based on persona, contain too much overlap between personas which resulted in a very disjointed user experience. The look & feel was also completely different. One challenging thing was that a single person often performed the role of multiple personas and was constantly switching from one product to another to get their work done.
The outdated technology these systems were based on also prevented simple usability enhancements that users have come to expect from the software of today.
In addition to usability & branding concerns there were 3 major areas of concern from the business:
PERFORMANCE PROBLEMS - some of these products had been in use for nearly 20 years. This had started to cause some major performance problems. Bugs were increasing, and the code was very fragile. We often found ourselves breaking production with new releases and developers were afraid to even touch some areas of the code.
CHURN - for the first time in company history, high school state sport associations who had been with the company for years (some for decades) were starting to look at other options. While we had enjoyed market dominance for years, user-focused disruptors were starting to come on the scene and we needed to up our game and join the Experience Economy.
SECURITY & ACCESSIBILITY - while security and accessibility standards had evolved quite extensively over the years, we had not kept up with them. Our ArbiterPay product was secure, but there were improvements that needed to be made. Accessibility had never been a concern and we wanted to start taking that more seriously as well.
I was hired & tasked with redesigning the ArbiterSports platform from the ground up. This was to be accomplished in a multi-phased approach and unify all functionality into a single platform.
There were 4 main goals or desired outcomes we set out to accomplish:
This was a pivotal time for the company. We needed to reduce churn and increase renewals. Churn rates were not crazy but it was the first time in many years that we had started to lose big customers. We wanted to nip the problem in the bud while we still could.
A large portion of our revenue comes from processing transactions as states & local assinger associations pay officials. We saw a big opportunity to increase both the number of transactions and expand into different types of payments.
During the "busy season" when governing bodies are setting up eligibility & registration requirements, officials are registering to officiate, and assigners are scheduling officials to games, we get a large amount of calls & support requests. By providing a simpler, more intuitive user experience we wanted to reduce the number of support calls our team was receiving. Nearly 50% of all support calls/emails were related directly or indirectly to our eligibility & registration system. These were calls we set out to eliminate as much as possible.
One of the main reasons for investing in building a UX team at ArbiterSports (hiring me) was that the executive team and board understood the need to take usability and customer feedback seriously. Arbiter historically had been a very customer-centric company, but up until I was hired in 2019 that customer obsession had been focused more on sales/support. Leadership recognized, however, that to continue to be the leader in the market, we needed to not only value but also understand our customers. That customer-centricity needed to be reflected in our products not just the way we sold and supported them.
When I was hired the initial thought was to just "redesign" & build each page in a modern technology stack. As I started diving in and becoming more familiar with our products, it quickly became apparent to me that a more foundational change was needed.
Let me share just 2 examples of usability problems that were baked far deeper than simply surface UI problems.
A "context switcher" was added to navigate from one product to another. It was broken out by Group (organization) and persona.
One of the difficulties was that you just had to know which role would take you to which product. Each selection also completely changed the UI.
It was a very jarring user experience that was baked into the entire system.
The first thing I did was spent time learning our various products. Some initial ways I approached that included:
I got a group of team members and stakeholders from across the company together for an Affinity Mapping brainstorming session. Participants included representatives from the executive team, product, engineering, implementation, customer support, and sales. Together, we captured our collective assumptions around users, pain points, core tasks, and key competitive advantages we didn't want to lose in the new platform.
From there I aggregated all the research together in a single spreadsheet that was easy to visualize and share across the company with stakeholders who weren't at the meeting. I got some feedback and a few people added to the document. I had everyone list problems using "How Might We..." statements so we could avoid the tendency to focus on solution implementation rather than problem identification.
With that knowledge in tow, my product management partner and one of our veteran sales reps and I embarked on a road show visiting many customers across all our core user types to interview them in the context of where they work to understand areas we had it right, and learn additional details that would influence the direction we took.
From there I aggregated our research findings into project personas that captured Jobs To Be Done, Core Tasks, Key Drivers, and current Pain Points.
One specific takeaway from our research was how much overlap there truly was between personas and how widely responsibilities varied from organization to organization. A single person often performed the duties of all our personas, while in other cases it was far more specialized. This led me to the hypothesis that a user-based rather than persona-based information architecture was needed.
At this point I had a pretty good grasp of the users and their pain points, but I needed to understand just what I was getting into. What were the building blocks I needed to arrange differently to provide a better experience? To make sure I had a complete understanding of our existing Information Architecture of our platforms I created a Content Inventory with the help of a couple senior support agents. In a spreadsheet we outlined all the pages, content, functionality and how it all fit together across all our products. This took time, but it was well worth it once it was done.
The task now became designing a new IA. I had a pretty good hypothesis and a few ideas floating around my head. But as this could be a large change for users, it was vital to test my ideas.
I mapped out common tasks by persona and job to be done, then got feedback from stakeholders 1 at a time to identify tasks I might be missing. This was similar to the brainstorming we did before our contextual inquiry except that it was informed by research and it was much more granular.
Next, I started imagining different ways we could structure and group those actions in a way that would be standard across personas so that we could follow a user-based architecture and get away from the persona-based architecture of the past. The three main grouping strategies I came up with were: by job (to be done), by entity, and by persona.
After talking with a few stakeholders, the team's consensus was that by job would be best. Howevever, it was vital that we test that assumption with actual users. So I set up a Hybrid Card Sort study. This would surface a list of tasks and participants had to sort the tasks into predetermined categories (my hypothesis). However, they could also create their own categories if they thought the tasks didn't fit neatly into one of the preset ones. I send invites to 1,000 users across our Assigner, Officials, and Admin personas.
I chose a hybrid card sort (as opposed to open or closed) because I was worried that:
Overall, the categories I had chosen made sense to the vast majority. The biggest area of confusion, was "settings". In our old system we had settings all over. I knew in the new system we had to do better.
One way I addressed this, was separating settings into 3 groups:
I started to explore how the navigation might work. My initial thinking and explorations were around a vertical nav. I wanted to get the nav out of the way as much as possible to provide more flexibility & room for content. I was worried a horizontal nav would take up too much vertical space. I wanted the platform to have more of a software feel than a marketing site.
However, this project would take 2-3 years to migrate all functionality into the new platform. APIs, Databases, & authetication would have to tweaked or rewritten completely in addition to the UI. In discussing with the team we were all worried that changing too much all at once would result in a jarring experience for customers and it could end up being perceived as a worse user experience.
We decided to take a more gradual approach.
STEP 1
We had already introduced new standaridized header/nav styling into our legacy products. We decided to go with a horizontal nav for the new platform so the experience between old and new would be much less jarring.
STEP 2
We would introduce a new main nav/sub nav across all legacy products following the new information architecture (without moving content on individual pages around). This would allow people to get used to the new IA.
STEP 3A
We would migrate functionality one main section at a time. So when they clicked "Eligibility", they would then be taken to the new eligibility pages, but the nav was already in place. This was one of the big reasons we decided to go with Jobs To Be Done-based IA instead of any of the other options we considered.
STEP 3B
As we migrated each section, we would introduce the user-focused model combining sub nav sections. For example, when we redesigned Eligibility & Registration, you would see the sub nav item for registrations you're configuring as a State Admin next to your personal registrations to officiate, all under the same main nav section of "Eligibility". This would reduce context switching while yet also making the change to a truly integrated platform more gradual.
This approach would take longer, but it would ensure a great user experience all along the way, not just at the end.
So we started down the path of Step 2 (Step 1 had been completed at this point). I started mapping how we would transition the old IA to the new. Green meant no change was necessary. Red represented a change between old/new.
In working with the executive team, product management, sales, and various other stakeholders, we chose Eligibility & Registration as the first chunk of functionality to migrate to this new information architecture. That decision was reached for a few reasons:
The experience of redesigning Eligibility & Registration was a fairly involved project of its own with its own pain points, user research, and design initiatives. I will write a separate case study going over the full process. For now, suffice it to say that process involved:
By far this product was the most successful launch in company history. Even though we took a pilot release approach (gradually releasing to a limited number of customers at a time, and knew that all customers wouldn't be able to move at first, we ended up being overwhelmed at how many customers were simplifying their internal processes (pricing structure for Registration for example) in order to be able to join sooner. It was a great example of how much better a user experience can get by doing the extra work up front to understand customers.
A couple of my favorite quotes we received for a customer during the release process:
So what about those results we set out to achieve? Here's just a few of the results we were able to achieve:
One of the biggest pain points that existed in the old system was having to create separate eligibility requirements & registrations for every level and fee structure per sport our clients offered. This often created upwards of 100 per year. Each one had to be configured and updated separately. We were able to reduce this number to 1-2.
Eliminating churn was one of the big focuses of this project. Some related results we were able to achieve included:
In addition, when one of our biggest partners (who was the de facto leader in sports-related insurance and testing content) decided to break ties & compete with us instead, we were able to win over the vast majority of their customers.
This is my personal favorite. One client in particular who was very skeptical when we first approach them to be a part of our user research during the design process, after seeing the solution we came up with, not only were interested in moving to the new system but simplified many of their processes and fees to be able to be a part of the initial pilot group.
There were many wins besides just the direct business outcomes and results. This was the first time a cross-functional, user-driven design process had been used at the company. Before this project developers and product managers rarely collaborated except to estimate user stories. Design had never really been a fundamental part of the process. The process being used previously was more like waterfall with a few scrum processes thrown in, than true collaborative agility.
During the course of the project the following cultural wins were achieved:
We still have a long way to go in both the product and culturally, but this project changed the course of the UX maturity of the company. It's now not a conversation of "if" (we should invest in UX) but "how many" and "when".