Wednesday, 26th June 2019
Logo

Data workers struggle as complexity grows

Alteryx has revealed that approximately 54 million data workers around the world face common challenges associated with the complexity, diversity and scale of their organisations’ data. In an increasingly data-driven world, the term “data worker” spans beyond the 54 million identified in this study, but the findings are indicative of the challenges specific to those engaging in significant data work in their day-to-day jobs. The Alteryx-commissioned IDC Infobrief, The State of Data Science and Analytics, uncovered inefficiencies, ineffectiveness and wasted time as many organisations turn to data as the lifeblood of their digital transformation.

Eighty percent of organisations now leverage data across multiple organisational processes, but despite increases in innovation, data workers still waste 44 percent of their time each week because they are unsuccessful in their activities. Data workers spend more than 40 percent of their time searching for and preparing data instead of gleaning insights and, on average, use four to seven different tools to perform data activities, adding to the complexity of the data and analytics process. Among other key findings:

  • On average, data workers leverage more than six data sources, 40 million rows of data and seven different outputs along their analytic journey.
  • The top frustrations cited by data workers in the survey are indicative of root causes that are responsible for inefficiencies and ineffectiveness. For example, more than 30 percent of data workers say they spend too much time in data preparation, a task that can often be automated.
  • Eighty-eight percent of data workers, approximately 47 million people worldwide, use spreadsheets in their data activities. Spreadsheet functions are often used as a proxy for data preparation, analytics and data application development tools but are error-prone and expose the organisation to compliance and trust issues.

“Data is at the core of digital transformation, but until organisation leaders address these inefficiencies to improve effectiveness, their digital transformation initiatives can only get so far,” said Stewart Bond, director of data integration and integrity software research at IDC. “Consolidating platforms and looking for tools that address the needs of any data worker, whether a trained data scientist or an analyst in the line of business, can help reduce the friction that many organisations experience on their path to becoming data-driven.”

The survey found that data workers are unsuccessful for a variety of reasons, including lack of collaboration, knowledge gaps and resistance to change. Participants reported the lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the highest-ranked skills gaps responsible for productivity issues, indicative of the pervasive talent gap that exists between data scientists and data workers in the line of business. To overcome these issues and more, many organisations are hiring Chief Data and/or Analytics Officers to streamline analytic processes, build a culture of analytics and encourage data literacy across the enterprise as part of their broader digital transformation strategy.

“Collecting data alone won’t digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies. The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyse data effectively to drive real, business-changing results,” said Alan Jacobson, chief data and analytics officer (CDAO) of Alteryx. “As the data landscape becomes more complex, this survey exposes the tip of the iceberg when it comes to the sheer volume of workers needing to conduct analysis on a daily basis and the untapped potential for them to drive meaningful business impact.”

Alteryx predicts that England will lose to India and Australia will take home the trophy.
A new report from Dun & Bradstreet reveals businesses are missing revenue opportunities and losing c...
Following the success of Snowflake’s sold-out inaugural user conference, Snowflake Summit on June 3-...
In a joint pilot project, the Finnish Olympic Committee, Polar and Tieto are testing a new system fo...
GridGain Systems has launched the GridGain Data Lake Accelerator, an in-memory solution for digital...
WHISHWORKS reveals results of 2019 Big Data Survey with in-depth look at trends and challenges.
Chief data officers (CDOs) and their data and analytics (DA) teams are focusing on the right priorit...
Protecting and improving the nation’s vital infrastructure – such as energy, transport and digital c...