Skills shortages top the list of the UK’s data challenges

Skills shortages in data streaming and engineering were cited as the number one data challenge by 30% of UK data professionals—ranking above all other concerns. The findings come from a poll of 110 UK data professionals conducted at Confluent’s Current event in London.

As more organisations invest in real-time data, a growing number of UK teams are distinguishing data streaming as a specialised skill set. Some are beginning to adopt the job title of “data streaming engineer” to reflect the increasing importance of streaming capabilities.

The second most cited challenge was slow decision-making due to delayed data, selected by 22% of respondents. This highlights the ongoing struggle many organisations face in accessing up-to-date information quickly enough to act effectively. In a business environment increasingly shaped by speed and responsiveness, delays in data access can lead to missed opportunities and reduced competitiveness.

The poll findings also come at a time when the UK is aiming to strengthen its AI capabilities. Real-time data is a critical component in powering intelligent systems, and skills shortages could slow progress.

The top five data challenges identified in the poll were:

Skills shortages in data engineering and streaming – 30%

Slow decision-making from delayed data – 22%

Siloed systems and disconnected teams – 20%

Data privacy and compliance pressures – 10%

Scaling infrastructure to meet demand – 10%

“If the UK wants to lead in AI, it must first lead in data,” said Richard Jones, VP at Confluent. “AI systems thrive on real-time insights, and that’s only possible with modern data streaming capabilities. We’re now seeing the emergence of dedicated data streaming engineers as organisations recognise that this is a distinct and essential role. This poll shows a clear and urgent need to address the talent gap in streaming and engineering to build the foundation for future innovation.”