Data mesh is an exciting new approach to designing and developing data architectures. After my first series of articles and more recent materials on data products and data domains, I received queries on explaining what a practical implementation on Azure could look like. …

Data mesh in practice — All clients I work with are either interested in or planning their next generation of a modern data platform. In this transition, data mesh architecture is gaining a lot of traction. At the same time there are concerns about interpreting this concept: practical guidance is missing. About data mesh Data mesh advocates treating…

Strategic path towards designing an enterprise-scale data mesh — In all data mesh discussions I have with enterprises the subject of data domains draws most attention. Customers find domain-oriented data ownership the hardest part of data mesh. Often, there is a fundamental lack of understanding of Domain-Driven Design (DDD). Other data practitioners find the conceptual notions of DDD too…

Design considerations for building a data mesh architecture — All organizations I work with understand the importance of data and are either interested in or planning their next generation of a modern data platform. Their aim is to move away from tightly coupled data interfaces and varying data flows towards an architecture that allows eco-system connectivity: a cloud distributed…

Is data mesh only for analytical data? This is a common question that arises when discussing data mesh with my customers. While data mesh is often associated with only analytical data, many of its principles can also be perfectly applied to operational data. In this blog post, we will explore…

Over the last few years, decentralized architectures have emerged as the new paradigm for managing data at large. They are meant to scale distribution of data between teams, while aiming for higher value and a faster time to market. In this article, I would like to unpack how to implement…

Microsoft Purview is unified data governance service that enables organization to manage their data at scale. It has features like Glossaries, Lineage, Data Access Management, Business Metamodel and so on. For managing technical metadata, such as source system information, Purview uses Collections: structures used to manage metadata within a hierarchy…

Many of my clients employ a Medallion structure to logically arrange data in a Lakehouse. They process incoming data through various stages or layers. The most recognized layout, illustrated below, incorporates Bronze, Silver, and Gold layers, thus the term “Medallion architecture” is used. Although the 3-layered design is common…

There’s a lot of fuzz going around the data build tool (DBT). It gains a lot of traction from the data community. I’ve seen several customers and projects, where DBT is already chosen as the tool to be used. But what it is? Where can you position DBT in your…