How test automation helped the FA achieve peak performance

In 2019, the English Football Association (FA) began its digital transformation journey to optimise the performance of its applications, which included the migration of many of its systems to the cloud.

The FA is the world’s largest governing football association, with thousands of stakeholders interacting with various systems - between 100 and 120 different applications supporting its operations - from members of grassroots teams, professional leagues, and the national team, to those responsible for managing Wembley Stadium and those participating in its education operation such as teaching potential coaches.

With an increasingly heavy load, the performance of the applications was beginning to suffer.

Poor performance results in negative feedback

Nick Sewell, Head of Application Development at the FA, explains the situation: “We received lots of negative feedback about the slow speed of our system’s response. This can prove challenging during the peak and busiest time of the year. For example, we have a player registration window in which millions of people playing football at the grassroots level need to be registered. So system response time is incredibly crucial.

“In addition, we frequently announce squads for the England men’s and women’s teams, which increases visitors to the website. Therefore we needed to make sure that we’re ready for those seasonal demands.”

“Historically, all our systems have been on-premises,” Sewell adds. “Now, we’re moving them to the cloud. Part of that journey is to improve our processes and tools when it comes to software development and testing. We needed a tool that allowed us to test performance to address the issues around speed and response.”

The FA had previously been introduced to Tricentis’s NeoLoad enterprise performance testing tool by its technology services partner Cognizant, but Roy Botley, Head of Quality Assurance (QA), extended its use in a more methodical, iterative manner as part of the overall transformational process. According to Sewell, “Roy looked at the process around performance testing and kind of ‘topped and tailed’ it. As a result, we made NeoLoad the standard tool to use with a standard process.”

Eliminating performance issues

During the busiest period, it could take up to 10 minutes to register a single player on the old system which, in Sewell’s words, was “just not an acceptable level.” On average, there are 15 steps in the registration process, which sometimes increases to 30, with around 300 checkpoints to measure. “Now we’ve reduced each step of the player registration to under three seconds,” says Sewell. “And we’re able to do bulk registrations. Instead of doing 100 registrations, one at a time, we can now speed up and do 100 in one go – in under 30 seconds.”

“Our biggest issue – system response time – has been eliminated,” he continues. “We’re now at a level where the business is confident in what we’re launching. Systems Performance is in a very good place. NeoLoad just reduces the risk – and provides peace of mind it for the business and the technical delivery team.”

Doing performance testing properly

Previously, a queuing tool was needed to throttle the volume of traffic, so the system could handle more than 400 people at a time. Now, though, the FA’s modern, iterative approach to performance testing means it can handle 10 times as many users accessing its website concurrently.

“We’ve tested up to 4,000 users,” says Sewell. “We’re confident in that number. The benefit of NeoLoad is that we don’t have to worry about performance issues; visitors to our website are doing what they intend to do at the speed they expect.”

“One of the key changes is that we’ve got the business involved [in performance testing],” says Botley. “We’d hear from people within the business that the performance was ‘below expectations’ – but what did that mean? We asked them to define the non-functional requirements in terms of how long something should take, and provide some critical workflow scenarios to enable us to run tests and measure the results against those SLAs.”

The FA takes the same iterative approach to infrastructure optimisation, too. The association is migrating its on-premise systems to the cloud. “We went through a process of experimentation on the different settings – CPU, memory, and so on,” say Sewell. “NeoLoad would test against these different settings, and we’d keep tuning and adjusting until it reached the point of being efficient and working in the way we needed it to.”

“This is the first place I’ve worked where we do performance testing properly,” comments Sewell. “NeoLoad allows us to do that for every step, for all non-functional requirements.”

Streamlines the operation of its annual Christmas campaign to support the homeless.
Console Connect has launched the second edition of its Africa Interconnection Report, researched by analyst firm Balancing Act, to understand the drivers and barriers to cloud adoption in the region.
IT backup, recovery and data management expert, Jeniya Klimova, who has spent the past five years working at global software company, Veeam, has joined the team at cloud computing and disaster recovery specialists, virtualDCS, as a channel account executive.
Old Mutual, a premium pan-African financial services group headquartered in South Africa, has selected A Cloud Guru, a Pluralsight company, to support its cloud-computing skills development program. With A Cloud Guru, Old Mutual has accelerated its efforts to be fully migrated to the cloud, ensuring the company has the underlying technology needed to provide a superior customer experience for advisors and customers.
In the highly competitive world of the private hire business, innovative IT support services can make a significant difference. Cloud infrastructure and data specialist PeaSoup has partnered with Virtual Rank, a cloud platform, to deliver the secure infrastructure that is powering the spectacular expansion of one taxi firm - which has seen turnover grow six-fold.
The rapid increase in SaaS prices has meant that customers are spending 53% more on licensing than they were five years prior.
Redis to collaborate with AWS on go-to-market activities for Redis Enterprise Cloud on AWS, which is designed to accelerate cloud migration and application development.
Latest release leverages flash, cloud-native monitoring, easy Veeam and vCloud integration for increased performance, efficiency and adaptability.