Data Mesh Newsletter #05
Too Long, Didn’t Skim:
1) We did meetups with Intuit and DPG Media; more coming in June (just confirmed HelloFresh for June 24!) and beyond.
2) A list of all of Zhamak’s public data mesh content here.
3) A great interview with a company that deployed data mesh here.
4) Food for thought (get it?) article from us here.
Quick Musings
Something that came up from a Twitter conversation: Data mesh isn’t a “miracle pill”, something you can just take and you’re all better. It’s not something you can buy. It is more like a long-term diet and exercise regimen: it requires planning, discipline, measurement, adjustment, etc. It’s why we call them user journeys! 😎😋
Interesting tidbit from NewVantage Partners 2021 Survey (Zhamak references NVP surveys often in presentations): 92.2% of data leaders at large companies say people/process/culture is a bigger challenge than technology. - This tracks with feedback re data mesh. Trying to deploy “all the tech” before the company is ready is likely to lead to failure. Aligning the people and process changes will mean you can go fast and adapt quickly as you build out your data mesh architecture.
Zhamak has been BUSY; just this month she did a big training (4 days x 4 hrs each day) with 60 people AND Kafka Summit Keynote; KG Conf Keynote; QCon Plus (this week); a Datanova Roundtable; YOW! Data Presentation; a TDB Roundtable; a few podcasts; etc. Sure we missed at least a few there.
We put together a list of all (that we could find) the data mesh presentations, podcasts, and posts she has made or been a part of here. We will try to keep it up to date. Please let us know if we missed any. If there is interest, we might work on one for non Zhamak content; let us know (might need some vendor content screening…).
We’re working on a data mesh metaphor. Maybe something around food; domains are cuisines (new domains can be fusion of two cuisines too), data products are recipes/meals (whether ready made, prepared, raw ingredients, etc.), etc. What are your thoughts?
Interesting Data Mesh-Related Content:
Data mesh in practice: learnings from a customer journey
by Mathias Golombek (Exasol, a vendor); written interview
A written interview with a customer deploying data mesh talking about the agility and flexibility benefits of data mesh. Another good article for sharing with execs re driving buy-in to data mesh as a concept. The company was facing issues with rigidness of analytical data - analytical jobs were set up to answer specific questions; if there was a new question, they had to create a new process and analytical output - and the bottlenecks of a central team owning the data in a data warehouse and a data lake. (Exasol pitch is contained to the last question only)
Intuit's Data Mesh Journey - Past, Present, and Future
by Tristan Baker and Suresh Raman (hosted by DML); video (75min, no transcript)
Fantastic presentation by Intuit about their strategy for implementing data mesh. It covered a lot of topics including their 4 pronged strategy to keep the transition effort moving forward. It really is a journey (~4-5 year expectation to fully implement) and they are in year 1 so lots of exciting things ahead. Extended Q&A too.
Data Mesh at DPG Media: Increasing News Personalization Relevance
by Wannes Rosiers (hosted by DML); video (75min, no transcript)
Awesome presentation about how DPG transitioned from a centralized architecture and ownership to data mesh including some practical tips, some great ideas about ownership of a data definition (e.g. customer) versus a data product. The guidelines section was especially helpful. DPG has vastly increased their agility and responsiveness by moving to a data mesh setup.
Rethinking platform modernization using data mesh
by Tony Phoang / The Artificial Intelligence Podcast; podcast (11min; no transcript)
Short podcast; starting ~7:00, Tony gives a good (simplified!) example checklist roadmap for how you could decompose an existing analytics process to go from on-prem data warehouse/lake to a data product in the cloud (the concept is called cloud coexistence).
The Data Menu vs Data Creation Question
by DML (us!); short-ish blog post
Inspired by a conversation with Teresa Tung (and other recent conversations): should we only create data products from data we already generate? Or should we look to generate incremental data (data we aren’t currently collecting) to improve our data products? And what are the ethical issues? Possible food for thought as you move forward with data mesh 🤔 This whitepaper (pdf; non-gated) by Accenture has some interesting additions re data lifecycle management and other topics.
How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform (AWS Blog)
by Anu Jain (JPMC), Graham Person (JPMC), Paul Conroy (JPMC), and Nivas Shankar (AWS); blog post
AWS blog post so some marketing FYI - Deep dive into how JPMC is doing governance in their data mesh including the differences between copying data and in-place data consumption - in-place consumption has far higher restrictions. Also mentions using a data catalog to literally only track data product access usage/patterns and having a separate data lake for each data product. Interesting architectural setup.
How to Change an Enterprise’s Software and Culture
by Software Engineering Daily Podcast with Zhamak; podcast (58min; transcript)
Interesting interview with Zhamak from early 2018 where you can see the genesis of data mesh (she REALLY hasn’t been a fan of centralized for a while - see below from the transcript). Generally useful re data mesh as it has examples how to drive change and implement something like data mesh in a large organization.
Upcoming Data Mesh Meetups, Webinars, Etc.
DML Meetup: Data Mesh at HelloFresh (still finalizing agenda)
Jun 24, 2021; 6:30pm CEST / 12:30pm EDT / 9:30am PDT
(non DML) Meetup: Data mesh - what is 'inside'– Experiences, approaches, and reflections with Lorenzo Nicora (CMC Markets)
May 27, 2021; 5:30 CEST / 11:30am EDT / 8:30 am PDT
Zhamak-led Training with DDD Europe (we are working on getting a discount code)
Sep 9, 10, 16, 17, 2021; 6-10pm CEST / noon-4pm EDT / 9am-1pm PDT each day
/Newsletter
If you have questions/comments/concerns/suggestions for future newsletters, please let us know at community@datameshlearning.com.
Special thanks to DataStax for allowing me to focus on this community and hopefully helping you all learn more about data mesh 😅