How test automation helped the FA achieve peak performance

In 2019, the English Football Association (FA) began its digital transformation journey to optimise the performance of its applications, which included the migration of many of its systems to the cloud.

The FA is the world’s largest governing football association, with thousands of stakeholders interacting with various systems - between 100 and 120 different applications supporting its operations - from members of grassroots teams, professional leagues, and the national team, to those responsible for managing Wembley Stadium and those participating in its education operation such as teaching potential coaches.

With an increasingly heavy load, the performance of the applications was beginning to suffer.

Poor performance results in negative feedback

Nick Sewell, Head of Application Development at the FA, explains the situation: “We received lots of negative feedback about the slow speed of our system’s response. This can prove challenging during the peak and busiest time of the year. For example, we have a player registration window in which millions of people playing football at the grassroots level need to be registered. So system response time is incredibly crucial.

“In addition, we frequently announce squads for the England men’s and women’s teams, which increases visitors to the website. Therefore we needed to make sure that we’re ready for those seasonal demands.”

“Historically, all our systems have been on-premises,” Sewell adds. “Now, we’re moving them to the cloud. Part of that journey is to improve our processes and tools when it comes to software development and testing. We needed a tool that allowed us to test performance to address the issues around speed and response.”

The FA had previously been introduced to Tricentis’s NeoLoad enterprise performance testing tool by its technology services partner Cognizant, but Roy Botley, Head of Quality Assurance (QA), extended its use in a more methodical, iterative manner as part of the overall transformational process. According to Sewell, “Roy looked at the process around performance testing and kind of ‘topped and tailed’ it. As a result, we made NeoLoad the standard tool to use with a standard process.”

Eliminating performance issues

During the busiest period, it could take up to 10 minutes to register a single player on the old system which, in Sewell’s words, was “just not an acceptable level.” On average, there are 15 steps in the registration process, which sometimes increases to 30, with around 300 checkpoints to measure. “Now we’ve reduced each step of the player registration to under three seconds,” says Sewell. “And we’re able to do bulk registrations. Instead of doing 100 registrations, one at a time, we can now speed up and do 100 in one go – in under 30 seconds.”

“Our biggest issue – system response time – has been eliminated,” he continues. “We’re now at a level where the business is confident in what we’re launching. Systems Performance is in a very good place. NeoLoad just reduces the risk – and provides peace of mind it for the business and the technical delivery team.”

Doing performance testing properly

Previously, a queuing tool was needed to throttle the volume of traffic, so the system could handle more than 400 people at a time. Now, though, the FA’s modern, iterative approach to performance testing means it can handle 10 times as many users accessing its website concurrently.

“We’ve tested up to 4,000 users,” says Sewell. “We’re confident in that number. The benefit of NeoLoad is that we don’t have to worry about performance issues; visitors to our website are doing what they intend to do at the speed they expect.”

“One of the key changes is that we’ve got the business involved [in performance testing],” says Botley. “We’d hear from people within the business that the performance was ‘below expectations’ – but what did that mean? We asked them to define the non-functional requirements in terms of how long something should take, and provide some critical workflow scenarios to enable us to run tests and measure the results against those SLAs.”

The FA takes the same iterative approach to infrastructure optimisation, too. The association is migrating its on-premise systems to the cloud. “We went through a process of experimentation on the different settings – CPU, memory, and so on,” say Sewell. “NeoLoad would test against these different settings, and we’d keep tuning and adjusting until it reached the point of being efficient and working in the way we needed it to.”

“This is the first place I’ve worked where we do performance testing properly,” comments Sewell. “NeoLoad allows us to do that for every step, for all non-functional requirements.”

Node4 provides SD-WAN and SOC Solution.
Rackspace Technology says that supporting Ohio National Chile’s cloud transformation has resulted in accelerated creation of new products, cybersecurity, and cost reduction for the financial services firm. The Chile division of this financial services firm moved its entire core business to the cloud and instantly gained flexibility and the ability to create Virtual Machines (VMs).
The UK’s largest family-owned department store, Fenwick, has adopted Qlik Cloud®, to advance its analytics capabilities and empower more employees at the company to use data in their day-to-day roles.
NTT DATA helped support Three with cloud optimisation, ensuring its ecosystem was simplified, and allowing Three to better assist its customers and employees.
Available in US, UK and Canada, new private cloud service solution helps organisations address compliance and data sovereignty needs.
87% of EMEA respondents migrated storage from on-premises to public cloud in 2022, and 83% expect the amount of data they store in the cloud to increase in 2023.
Available in US, UK and Canada, new private cloud service solution helps organisations address compliance and data sovereignty needs.
Vodafone Qatar—which provides a comprehensive range of services including voice, messaging, data, and ICT managed services for consumers and businesses in Qatar—has selected both VMware Telco Cloud Platform-5G Edition and VMware Cloud Foundation with Tanzu as the cornerstone of its modern cloud platform.