Most organizations are widely adopting data mesh to make data decentralized and build innovative and competitive products. Since Zhamak Dehghani coined the term, organizations began handling huge data through a distributed architecture. This data-decentralized architecture provides governance and ownership for complete data management. So, we breakdown the strategy to build data products that drive growth for organizations.

You can create your customized version of Data Mesh and implement it in your application systems. However, before beginning the process of transformation using data mesh, here’s what you must consider: 

  • Unique features in data product
  • Preparing the existing platform to support for the data mesh architecture 
  • Making data-driven decision-making culture by leveraging data products 

To find the answer to these questions, it is important to understand data products and how to convert existing data pipelines and data sets into data products.  

Characteristics of data products 

  • A data product should be owned by the domain team that works most closely with the data.  
  • It must be valuable and usable to them.  
  • Implementation of data products must be feasible in the current application platform.  
  • It should be discoverable – both semantically and syntactically. 
  • The owner of a data product must be able to measure parameters like completeness, lag, and statistical shape and be able to trust the data product.  
  • It should interoperable – meeting and governing global standards.  
  • Security – data product owner must have the choice to define the access control policy. 

How can data mesh handle datasets?  

  • Data pipelines and data sets exist. The control will be with the Data & Analytics team (D&A). 
  • The datasets become usable and valuable to the teams. Also, can be used by other teams when required. 
  • These datasets will have the feasibility to implement wherever needed to accelerate performance of the product. 
  • Datasets are discoverable once they are implemented in the organization for specific use cases. 
  • As pipeline team ensures quality management, they remain trustworthy. 
  • The datasets are most probably not governed by global standards and have the security layer on a case-by-case basis. 

Customized migration methodology from datasets to data products 

Global discovery  

This is the first phase of the strategy for building a data product. Analyze the datasets across the data warehouse and identify domain owners. The first party and second-party usage owners are determined. This task is simplified if the data catalog is structured. This phase establishes dataset domain ownership and allows cross-domain usability.  

Local discovery  

Here the domain owners conduct an analysis to identify datasets assigned to them. Once the data identities are known, purchased, or consumed, the tentative definition of data products is formed. For this to happen, data quality rules will have a trustworthiness and global access policy. A roadmap to convert data assets into data products is planned in this stage. After this research, feasible study, data product identification, review of existing technology, infrastructure, and its usefulness in the data mesh world will be the outcomes.  

Ownership handover  

There are two ways of shifting ownership in your organization. You can either completely convert to a domain-based ownership model or choose to create a team under the data and analytics division for specific sets of domains. The benefit of this step is admin rights, code responsibility, and management handovers are easily segregated among the teams and individuals.

Migration implementation  

Identify gaps in the data products. Create a project plan and fill these gaps in strategic ways that pave the way for the innovation of data products. This enhances discoverability and maintains data quality in the long run. 

Monitoring

Once the conversion is successfully completed, stringent monitoring of the data product for various SLAs is done. Measure and improve adoption based on the results. This will result in the seamless availability of data products in your organization.  

Thus, the data mesh architecture introduces two essential concepts – distributed domain-driven data ownership and the creation of data products. To establish the first bit, deep organizational changes are required which are difficult to adopt. However, data products are a relatively simple processto adopt and makes an organization datadriven. Migrating existing datasets into data products is the right step toward creating a secure and reliable data management system.