Fivetran introduces Metadata API – enables end-to-end data analysis and visibility

Enterprises can track entire data lineage in data catalogues to meet complex compliance requirements and empower global teams to access the right data securely.

Fivetran has introduced Fivetran’s Metadata API to enable the tracking of data “in-flight” from source to destination as it moves through Fivetran-managed pipelines. With this additional visibility, customers can integrate with governance and observability tools to give data teams more control over who has access to what data. Enabling automated data governance, the Fivetran Metadata API also provides data stewards, security teams and data engineering teams the needed visibility to answer where the data came from, who accessed it, and what changes have occurred in the pipeline.

“Every enterprise knows it must be data-driven, but traditional data governance has been a barrier with manual processes and reactive enforcement of policies. That's not a scalable approach, especially as data infrastructure grows to thousands of pipelines,” said Fraser Harris, VP of Product at Fivetran. “With Metadata API, our customers get out-of-the-box data governance automations and data quality workflows so they can proactively identify and take action on governance issues before they become a problem. Our automated in-flight approach enables data access at scale without increasing risk to the business.”

Complex legislation such as Europe’s GDPR, the California Consumer Privacy Act (CCPA), and other state laws impose fines on companies that do not comply with complex rules to safeguard data privacy. The Fivetran Metadata API helps enterprises address compliance requirements, easily integrating into their existing privacy and security strategies while bolstering the value of investments they’ve made in data catalogues and data quality solutions.

With the Fivetran Metadata API:

Data analysts are provided with a deep understanding of where data is coming from and are able to run impact analyses on it

Data stewards know end users have access to data that has been handled securely and is compliant with governance requirements

Security and legal teams can complete security audits and ensure the data moved is in compliance with organisational policies

Data architects and engineers will soon be able to understand upstream schema changes and ensure downstream processes are updated.

Creating an automated data governance experience – Fivetran partners with leading data catalogues

Fivetran is excited to launch better metadata management with four leading data catalogue vendors: Atlan, data.world, Alation and Collibra. Here are the combined benefits:

Information about all data can be consolidated into a single data catalogue, for a complete view and seamless user experience

End-to-end data lineage graphs are available for data, despite data passing through multiple systems and tools

By centralising governance in a single tool, data stewards can better ensure policies and processes are being applied to data as necessary

The ability to source trace data at a column level back to its origin helps confirm data quality and builds trust that the data is accurate and safe to use.

“Fivetran’s Metadata API solves a major gap in extracting data from operational systems into modern analytical systems by delivering the much-needed context. The availability of the Metadata API will accelerate development of reliable and secure data-intensive applications by exposing lineage, impact analysis, and security and privacy aspects,” said Sanjeev Mohan, Principal at SanjMo, a data and analytics expert and former Research Vice President at Gartner.

“Our customers are excited to move faster with more visibility, powered by metadata from Fivetran,” said Prukalpa Sankar, Co-Founder at Atlan. “No need to manually stitch together sources anymore. With the new Metadata API, customers can now bring metadata from Fivetran into Atlan for a truly automated data governance experience. This end-to-end lineage covers everything from upstream sources to the warehouse and BI layer, helping customers stay informed across complex data stacks with powerful root cause and impact analysis.”

“data.world's interface with Fivetran enables data producers and data consumers to understand and trust the data synchronisation and mapping along all databases, data sources and applications that matter to enterprises,” said Jon Loyens, CPO & Co-Founder at data.world. “Together with Fivetran, we provide granular data visibility, including metadata of source-destination column pairs, simplifying data discovery, data governance and actionable insights.”

VAST Data has formed a strategic partnership with Dremio to enable enterprises to get from data to insights faster with a hybrid, multi-cloud architecture for scalable analytics. Regardless of physical location – on-premises or in the public cloud – Dremio customers can now analyze their data anywhere by leveraging VAST’s massively parallel architecture for concurrent and near real-time data access at any scale.
Partnership programme expands to include global systems integrators and new markets across Europe, Latin America and Asia.
360-degree player insights and better data empower ECB’s approach to Women’s Cricket to drive performance improvement and efficiency.
Providing data and analytics teams with greater visibility and insight into their data pipelines.
Economic uncertainty raises need for faster access to reliable data to drive business impact.
UK Export Finance is modernising to the cloud with Informatica’s Intelligent Data Management Cloud (IDMC), freeing up team resources to better serve UK businesses of all sizes and sectors.
The Trust has selected BridgeHead’s HealthStore® Independent Clinical Archive to underpin its application decommissioning programme whilst ensuring all data remains accessible throughout its 25-year retention period.
Almost half surveyed say they are using hybrid cloud or local cloud service providers as an alternative to AWS, Azure, and Google Cloud to meet sovereignty requirements.