Data Mesh Newsletter #012
The quick bit:
0) Scott Hirleman put together a list of some of his favorite data mesh related content here. ~30 pieces that should help anyone advance their thinking re data mesh.
1) Nick Heudecker has graciously agreed to do video interviews with vendors so they can share what they offer relative to data mesh on the DML YouTube/site (will be clearly labeled). As Nick has stated, “pitches get stitches”, so expect them to be an honest discussion instead of a pitch. To learn more and/or sign up, check here.
2) Juan Sequeda and a few others have graciously agreed to do some Q&A discussions and panels re data mesh. Please send your ideas for panels (topics and guests) to Scott Hirleman. Data governance and applying Domain-Driven Design to data are two likely near-term topics.
3) This is the last newsletter edition run by Scott Hirleman. The future newsletters will be run by a joint group led by Andrew Padilla of Datacequia (LinkedIn) (thanks for stepping up Andrew!); if you’d like to be part of running future newsletters, let Andrew know.
4) LOTS of great meetups coming up with data mesh implementers! Check them out and support your awesome fellow community members please!
5) Implementing a data mesh? Silicon Angle/theCUBE wants to do a video interview with you! Ping Scott Hirleman or see this post in the Slack for more info.
Interesting Data Mesh Related Content
Expert Panel: Data Discovery in Data Mesh (DML-hosted event)
Moderator: Paco Nathan (Derwen.ai); Panelists: Shinji Kim (Select Star), Sophie Watson (Red Hat), Mark Grover (Stemma), and Shirshanka Das (Acryl Data); video (62min) (co-hosted w/ OSD Podcast)
Paco led a great discussion about data discovery including where we have come from and where we are going and what are some best practices. Just a great start to the discussion around data discovery and how it applies to data mesh. You can see some of the questions that the audience asked here. Well worth the watch!
NOTE: There is also a post on LinkedIn on the topic and there are some great points but some are a little out there (e.g. adding more nodes to the mesh to scale it)
How to Make a Data Mesh; Not Data Mess
by Joe Caserta at MIT CDOIQ 2021; video (57min)
Very interesting video about data mesh though we don’t necessarily agree with some of it. Seems to really cover most of the data mesh concepts at a deep level but also missed a number of presentations talking about implementations. Overall, seems positive on what data mesh could be but focuses a bit much on what is missing to-date re specifications and the like; pretty fair about some of the challenges.
(Not in-line with data mesh) Why Data Mesh Needs Data Virtualization
by Alberto Pan (Denodo - data virtualization vendor); blog post
Per the article and a subsequent follow-up on Twitter, the author is arguing we should be using data virtualization to create data products but this goes against the tenets of data mesh. Zhamak has said, “If you think about using virtualization on top of your microservices database and expose them and call that a mesh, that is probably a bad idea.”
Making a Mesh Start
by Olivier Bauer (Vistaprint/Cympress); blog post
A good initial post on how and why Vistaprint is moving from a monolithic data warehouse to a data mesh. Very good points on communication and collaboration and finding the right balance between not enough governance and too much, especially re standards and data product interoperability. Very worth the time.
NOTE: Scott does not agree with one point from the five lessons section: Olivier recommends having your entire data domain space mapped before starting; but, especially for an initial PoC, this is something folks are getting bogged down on; you can move forward with a few domains bought in as you learn more on how to draw your domain boundaries and domains will change/evolve!
The best person to manage your organization’s data may be you
by Paul Gillin (Computerworld); article
Relatively quick read on why companies are evaluating/moving to data mesh. The last two paragraphs include a few interesting tidbits as well:
Announcing OpenMetadata
by Suresh Srinivas (open-metadata.org); Medium post
Interesting post about having easy collection and sharing of metadata in a centralized repository based on open standards. Seems early days but potentially a project to watch.
Upcoming Meetups / Zhamak Stuff
DML Meetups (10pm IST / 6:30pm CEST / 12:30pm EDT / 9:30am PDT):
Sept 9: AutoZone
Sept 16: AO
Sept 23: Overstock.com
Sept 30: Adevinta
Oct 7: Flexport
October speaking slots open! Contact Scott Hirleman if you want to share your journey!
Other Data Mesh Meetups/Webinars/Training
Zhamak-Led Training (4 days, 4hrs each day, Sept 20, 21, 27, 28); 15% discount link makes it 1,827.50 Euro (don’t understand how VAT works…)
Data Platform in a mesh architecture webinar; Zhamak and Emily Gorcenski (Thoughtworks); Oct 14 at 5:30pm CEST / 9:0pm IST / 11:30am EDT / 8:30am PDT
/Newsletter
If you have questions/comments/concerns/suggestions for future newsletters, please let us know at community@datameshlearning.com.
Special thanks to DataStax for allowing Scott to focus on this community and hopefully helping you all learn more about data mesh 😅 (and yes, this section will change w/ Andrew taking over)