Data Mesh Newsletter #018
Two editors are better than one issue
The quick bit…
Scott Taylor (The Data Whisperer) provides some comedic relief for all you weary data professionals out there as the year comes towards a close. Watch here before diving in to the articles in this issue. Titled ‘The Little Red Data Hen - A Cautionary Tale’. A Data Mesh nod inside.
For those of you who know Martin Chesbrough, please welcome him as a new contributing editor of this newsletter. Martin has been a valued contributing member of DML Slack since the early days. We look forward to his thoughtful contributions here.
Data Mesh Content Unzipped
GraphQL is Built for Federation, But Don’t Do It Wrong
Authors: Anant Jhingran and Dan Debrunner
Although there may be some confusion in the community where output ports are sometimes conflated with API endpoints, that doesn’t necessarily mean it isn’t an option. If this the case for you, then GraphQL may be considered to federate queries to your data products. The authors contend there is a right and wrong way to do this.
Data Management on a Decentralized Data Mesh
Author: Fernando Velez
This article starts by giving a nod to the future of Data Mesh while attempting to tackle the details of how consumers in a Data Mesh aggregate data from multiple domain products. The details are often glossed over in many articles and while Andrew thinks there are a few misconceptions here (output ports are always API endpoints and use of ‘data integration’ over ‘data aggregation’), it does well by initiating the conversation around the dirty details of aggregating multiple data products on the consumption side from the lens of the current art for doing the same.
Data as a Product: What We Can Learn from More Established Industries
Author: Jeremy Posner
The subject of data as a product and data products can sometimes be described in more esoteric terms that may not reach a broader audience. If the spirit of Data Mesh underscores anything, it is to eliminate the divide between data stakeholders from different disciplines. One way to start is to describe these terms in a way everyone can relate to and in fact from which these terms were inspired. The author does exactly that by tying back to common product concerns of physical products that give them the polish and familiarity we have all come to expect as consumers.
How a Data Mesh is Driving our Platform
Presenter: Trey Hicks (video)
The author describes the Data Mesh implementation at Gloo.us that leverages Confluent Kafka based suite of tools to build an event driven architecture through Data Mesh principles.
Presenter: Wannes Rosiers
Early Data Mesh implementer and DML Slack member Wannes Rosiers from DPG Media describes his team’s data evolution from enduring humble data silo challenges to defining and executing on an outcomes driven mission statement and working backward with the right balance of platform thinking and use case perspectives that are underlied by Data Mesh at its core.
Data Mesh: Should you try this at home?
Author: Tony Baer
Yet another data management history lesson followed up by a walkthrough of Data Mesh principles. Some nuance in each interpretation is always useful however. You decide.
Data Fabric vs. Data Mesh: What’s the Difference?
Author: Daniel Abadi
No doubt many of us are confused by these often used textile analogies. This article attempts to unravel the threads of confusion between the two.
The Next Big Challeng for Data is Organizational
Author: Bryan Offutt
This article does not explicitly mention Data Mesh but it does address the organisational issues around data engineering in a way that is sympathetic to a Data Mesh architecture. Specialisation, modularity, clarity and buy-in are (we think) principles that can be addressed through Data Mesh, therefore we see this article as complementary to Data Mesh thinking.
Our journey towards an open data platform
Ostensibly this is the story of how Yotpo built their data platform in a self-serve way due to resource constraints within data engineering. It serves as a good example of an organisation that is using a Data Mesh approach without having to over emphasize the Data Mesh principles. It is almost accidental that Yotpo realised they have used a Data Mesh approach - lovely story.
How to identify Data Products? Welcome “Data Product Flow” - Agile Lab
Author: Paolo Platter
This looks like a “must read” article for those interested in data products. It does assume a knowledge of Domain-driven Design and uses DDD concepts freely as it abstracts the operational product perspective into the data world. It is a longish read but does go through a worked process to identify a data product. This should be a useful article for anyone who wants to identify data products for their organisation
Upcoming Meetups / Zhamak Stuff
Check back next issue. Hopefully Zhamak will be able to get some rest as we approach the end of 2021 after a very busy year for Data Mesh!
Book Nook
If you have any suggestions please send my way.
/Newsletter
If you have questions/comments/concerns/suggestions for future newsletters, please let us know at datacequia@gmail.com.
Special thanks to Datacequia LLC for contributing time and effort to focus on this community and hopefully helping you learn more and give back.