To print this article, all you need is to be registered or login on Mondaq.com.
Data fabric promotes an architectural change that allows organisations to leverage data innovation with minimal disruption.
The proliferation of the internet in early 2000s created the perfect environment for the mass adoption of cloud services. Driven by a need for flexible and scalable solutions, the introduction of cloud services into the mainstream triggered a surge in new companies and technologies, leading to data silos and architectural constraints.
Traditionally, organisations focused on integrating the technologies and data hosted in their on-premises data centres, as they had control. With the mass adoption of cloud services, many now adopt hybrid or full-cloud architecture, creating technical and architectural challenges:
Data fabric is an architecture approach that bridges on-premises and cloud environments, with the main goal of integrating data silos. It also provides a centralised platform that can be used to secure, curate, govern and share data across organisations. Data fabric improves the reliability and consistency of data by using metadata in conjunction with artificial intelligence (AI) and machine learning (ML) techniques to infer relationships and create an organisation-wide data catalogue.
The ability to adapt and leverage new technologies is key for organisations to keep their competitive advantage. In fact, 4 in 10 G20 business leaders claim to be under extreme pressure to integrate new technology according to the 2022 FTI Resilience Barometer. While many enterprises are quick to embrace new technologies, this adoption may come with a heavy price tag due to unfit data architecture.
The correct implementation of data fabric allows enterprises to:
By actively monitoring metadata across source systems, it is possible to automatically identify new data points and derive their context. For example, if organisations decide to change cloud providers, data fabric will integrate with the new cloud provider to identify or update the data catalogue.
The use of data fabric promotes the automation of repetitive tasks, such as the creation of pipelines, data profiling and data quality rating. By automating these activities, development teams can focus on key activities, improving their efficiency.
The ability to catalogue data across different architectures (on-premises, cloud, hybrid, multi-cloud) allows users to access data through a single interface. This allows end users to access the data they require without having to worry about the source location.
By actively analysing metadata from multiple data sources, it is possible to enforce data and security policies at a data-source level. In conjunction with automatic data classification, this helps reduce security risks and increase compliance. Organisations can also perform data quality checks across source systems from a central location, giving them the view of how healthy the organisation's data are.
One of the main challenges that organisations face is how to ensure their data remains inaccessible and indecipherable to unauthorised parties. Since data fabric relies on data being accessible across systems and locations, it is fundamental that robust security protocols and procedures are put in place.
To mitigate risks, it's critical to take a holistic view of the organisation's existing architecture. This is achieved by tying together the existing pieces of architecture instead of consolidating sources and creating single monolith solutions as a result.
It enables the centralisation of activities that were once individually managed and applied across data sources. For example, data governance policies can now be enforced and monitored from a central location.
Typically, data fabric architecture is supported by multiple technologies, such as data virtualisation, AI and ML. These technologies complement the existing vendor offerings and should be taken into consideration when assessing solutions.
The same way agile delivery promotes early feedback, it is also important for teams working on data fabric to collect early feedback from users. Teams can then ensure that the solution will be able to address the current business needs and be flexible to adjust to new ones.
/cloudfront-us-east-1.images.arcpublishing.com/tgam/5BNFJIP44VGB7C7T4OFSLKLDRA.jpg "Familiar tropes make a good heist book irresistible")
