How test automation helped the FA achieve peak performance

In 2019, the English Football Association (FA) began its digital transformation journey to optimise the performance of its applications, which included the migration of many of its systems to the cloud.

The FA is the world’s largest governing football association, with thousands of stakeholders interacting with various systems - between 100 and 120 different applications supporting its operations - from members of grassroots teams, professional leagues, and the national team, to those responsible for managing Wembley Stadium and those participating in its education operation such as teaching potential coaches.

With an increasingly heavy load, the performance of the applications was beginning to suffer.

Poor performance results in negative feedback

Nick Sewell, Head of Application Development at the FA, explains the situation: “We received lots of negative feedback about the slow speed of our system’s response. This can prove challenging during the peak and busiest time of the year. For example, we have a player registration window in which millions of people playing football at the grassroots level need to be registered. So system response time is incredibly crucial.

“In addition, we frequently announce squads for the England men’s and women’s teams, which increases visitors to the website. Therefore we needed to make sure that we’re ready for those seasonal demands.”

“Historically, all our systems have been on-premises,” Sewell adds. “Now, we’re moving them to the cloud. Part of that journey is to improve our processes and tools when it comes to software development and testing. We needed a tool that allowed us to test performance to address the issues around speed and response.”

The FA had previously been introduced to Tricentis’s NeoLoad enterprise performance testing tool by its technology services partner Cognizant, but Roy Botley, Head of Quality Assurance (QA), extended its use in a more methodical, iterative manner as part of the overall transformational process. According to Sewell, “Roy looked at the process around performance testing and kind of ‘topped and tailed’ it. As a result, we made NeoLoad the standard tool to use with a standard process.”

Eliminating performance issues

During the busiest period, it could take up to 10 minutes to register a single player on the old system which, in Sewell’s words, was “just not an acceptable level.” On average, there are 15 steps in the registration process, which sometimes increases to 30, with around 300 checkpoints to measure. “Now we’ve reduced each step of the player registration to under three seconds,” says Sewell. “And we’re able to do bulk registrations. Instead of doing 100 registrations, one at a time, we can now speed up and do 100 in one go – in under 30 seconds.”

“Our biggest issue – system response time – has been eliminated,” he continues. “We’re now at a level where the business is confident in what we’re launching. Systems Performance is in a very good place. NeoLoad just reduces the risk – and provides peace of mind it for the business and the technical delivery team.”

Doing performance testing properly

Previously, a queuing tool was needed to throttle the volume of traffic, so the system could handle more than 400 people at a time. Now, though, the FA’s modern, iterative approach to performance testing means it can handle 10 times as many users accessing its website concurrently.

“We’ve tested up to 4,000 users,” says Sewell. “We’re confident in that number. The benefit of NeoLoad is that we don’t have to worry about performance issues; visitors to our website are doing what they intend to do at the speed they expect.”

“One of the key changes is that we’ve got the business involved [in performance testing],” says Botley. “We’d hear from people within the business that the performance was ‘below expectations’ – but what did that mean? We asked them to define the non-functional requirements in terms of how long something should take, and provide some critical workflow scenarios to enable us to run tests and measure the results against those SLAs.”

The FA takes the same iterative approach to infrastructure optimisation, too. The association is migrating its on-premise systems to the cloud. “We went through a process of experimentation on the different settings – CPU, memory, and so on,” say Sewell. “NeoLoad would test against these different settings, and we’d keep tuning and adjusting until it reached the point of being efficient and working in the way we needed it to.”

“This is the first place I’ve worked where we do performance testing properly,” comments Sewell. “NeoLoad allows us to do that for every step, for all non-functional requirements.”

Healthcare workforce platform used by 37,000 clinicians supports staffing for the U.K.’s National Health Service; Lookout Secure Cloud Access provides visibility into usage and access of all corporate data.
CipherTrust Data Security Platform as-a-Service allows businesses to rapidly deploy and scale key management and data security services without the need to manage hardware or fund upfront investments.
Cisco launches new security service edge (SSE) solution to enable superior hybrid work experiences and dramatically simplify access across any location, any device, and any application.
Purpose-built data lake helps customers and partners achieve a more complete, organization-wide understanding of their security posture and act faster to improve the protection of their workloads, applications, and data.
Contract provides a streamlined procurement process and accelerates the delivery of services to the State of Texas.
The industry’s 'most comprehensive' as-a-Service and multicloud portfolio helps companies better manage their data and applications wherever they live.
Kubernetes-native security cloud service supports both Red Hat OpenShift as well as non-Red Hat Kubernetes platforms, including Amazon EKS, Google GKE and Microsoft AKS.
Rackspace Technology has released Rackspace Object Storage service built on the Dell ECS storage systems.