Whether it is compliance, overall bank management, customer experience, the connection of innovative cloud applications or a data basis for strategic decision-making: How reliably data management works for financial institutions and insurance companies depends on the respective architecture. There are different approaches here - from data warehouse, data lake and data mesh to data fabric. But what is suitable for whom? Is there an optimal solution? Or is it rather an interplay of different architectures?
In order to be able to select the appropriate architecture in each case, the financial institution should define the current status quo and clarify what specifically is to be achieved with the data management solution. It is also important to decide whether the existing systems should be retained or whether a completely new IT environment is planned. However, the latter is much more costly and carries risks.
Data Lake
A data lake is a storage environment that stores petabytes of structured, semi-structured and unstructured data for business analytics, machine learning (ML) and other comprehensive applications. Just a few years ago, data lakes were all the rage. However, they often turned out to be "data swamps" in hindsight: Because the data had been injected into the architecture without strategy, it was difficult to convert it into information that could be used for customer acquisition or market analysis, for example.
Data warehouse
A data warehouse (DWH) is an optimized, central database that is used for all types of analyses. The DWH brings together data from several, usually heterogeneous sources. On the one hand, a data warehouse can be very powerful, but on the other hand, performance decreases with increasing data volume. In addition, it sometimes does not work with all sources and data types. For example, unstructured data cannot always be processed with it. Other disadvantages include often low flexibility and agility. Older on-premise environments may not integrate, and there is a risk of vendor lock-in.
Data Mesh
Data mesh is a decentralized data architecture in which data is organized by business unit - such as marketing, sales, customer service, asset management, etc. Because managers in each department are aware of the domain data, they can set policies for data management that address documentation, quality and access. However, this architectural approach consumes many resources. In addition, assigning data to domains encourages siloing.
Enterprise Data Fabric
The enterprise data fabric is a new architectural approach that accelerates and simplifies access to data assets across the enterprise. Data scattered across the enterprise can be networked and analyzed in real time. At the same time, the company can continue to use its existing applications and data. Data is accessed, integrated, harmonized and analyzed on demand to support various business initiatives. Valuable insights can be gained in the shortest possible time without having to rebuild data stores from scratch. Thanks to high scalability, this is also possible with large data volumes. A data layer between the business application and the data fabric helps automate processes such as data integration, data engineering and governance. This makes the data fabric more resource-efficient than the data mesh approach.
CONCLUSION/PRACTICE TIPS
The data lake is currently the common environment for data assets and is subordinate to all other architectures. Like the data warehouse, it serves in many places as a supplier to the newer approaches of data fabric and data mesh. For example, financial institutions that have already built and populated a data lake can implement a data fabric between the business side and the data lake. This makes it easier to transform data and associated metadata into information that can be used for business purposes.
A data fabric is particularly versatile and flexible. It can integrate different sources and use them in an application-specific way. Financial institutions benefit from a robust database that they can use for artificial intelligence and ML. Ad hoc requests from regulators, for example, can be answered more quickly to prevent compliance breaches. Timely risk and analytics data enables real-time responses to market developments. Decision-making in business departments is also facilitated, as up-to-date information is available for risk assessment. In addition, a data fabric enables seamless data access via user-specific rights management. Limitations in access to data are thus a thing of the past.
With a data fabric, companies can reduce the cost and complexity of data processing and gain insights that give them a competitive edge. Because existing technologies can continue to be used, investments are protected. And with the flexibility, scalability, and interoperability of a data fabric, the enterprise remains future-proof.
In order to be able to select the appropriate architecture in each case, the financial institution should define the current status quo and clarify what specifically is to be achieved with the data management solution. It is also important to decide whether the existing systems should be retained or whether a completely new IT environment is planned. However, the latter is much more costly and carries risks.
Data Lake
A data lake is a storage environment that stores petabytes of structured, semi-structured and unstructured data for business analytics, machine learning (ML) and other comprehensive applications. Just a few years ago, data lakes were all the rage. However, they often turned out to be "data swamps" in hindsight: Because the data had been injected into the architecture without strategy, it was difficult to convert it into information that could be used for customer acquisition or market analysis, for example.
Data warehouse
A data warehouse (DWH) is an optimized, central database that is used for all types of analyses. The DWH brings together data from several, usually heterogeneous sources. On the one hand, a data warehouse can be very powerful, but on the other hand, performance decreases with increasing data volume. In addition, it sometimes does not work with all sources and data types. For example, unstructured data cannot always be processed with it. Other disadvantages include often low flexibility and agility. Older on-premise environments may not integrate, and there is a risk of vendor lock-in.
Data Mesh
Data mesh is a decentralized data architecture in which data is organized by business unit - such as marketing, sales, customer service, asset management, etc. Because managers in each department are aware of the domain data, they can set policies for data management that address documentation, quality and access. However, this architectural approach consumes many resources. In addition, assigning data to domains encourages siloing.
Enterprise Data Fabric
The enterprise data fabric is a new architectural approach that accelerates and simplifies access to data assets across the enterprise. Data scattered across the enterprise can be networked and analyzed in real time. At the same time, the company can continue to use its existing applications and data. Data is accessed, integrated, harmonized and analyzed on demand to support various business initiatives. Valuable insights can be gained in the shortest possible time without having to rebuild data stores from scratch. Thanks to high scalability, this is also possible with large data volumes. A data layer between the business application and the data fabric helps automate processes such as data integration, data engineering and governance. This makes the data fabric more resource-efficient than the data mesh approach.
CONCLUSION/PRACTICE TIPS
The data lake is currently the common environment for data assets and is subordinate to all other architectures. Like the data warehouse, it serves in many places as a supplier to the newer approaches of data fabric and data mesh. For example, financial institutions that have already built and populated a data lake can implement a data fabric between the business side and the data lake. This makes it easier to transform data and associated metadata into information that can be used for business purposes.
A data fabric is particularly versatile and flexible. It can integrate different sources and use them in an application-specific way. Financial institutions benefit from a robust database that they can use for artificial intelligence and ML. Ad hoc requests from regulators, for example, can be answered more quickly to prevent compliance breaches. Timely risk and analytics data enables real-time responses to market developments. Decision-making in business departments is also facilitated, as up-to-date information is available for risk assessment. In addition, a data fabric enables seamless data access via user-specific rights management. Limitations in access to data are thus a thing of the past.
With a data fabric, companies can reduce the cost and complexity of data processing and gain insights that give them a competitive edge. Because existing technologies can continue to be used, investments are protected. And with the flexibility, scalability, and interoperability of a data fabric, the enterprise remains future-proof.