Semantic Data Modeling (SDM) is a popular phrase in Analytics and AI circles these days, and part of it is mysterious. Today I’m going to introduce Semantic Models at a high level, and then talk about a few points of governance for living and working with this technology.
A Semantic Data Model is a data structure that hides the complicated realities of data storage behind a series of very flexible logical relationships, intended to drive Operational Analytics, Business Intelligence, and Retrieval Augmented Generation (RAG), such as Salesforce’s grounding for generative AI. Many popular data platforms offer Semantic Modeling, including Salesforce’s Data 360 and Tableau Next, Snowflake, and Sigma Analytics. These services also offer related features like standardized metric definition, join specification where needed, and critically for AI grounding, natural language descriptions everywhere.
This past September I was invited to join a roundtable discussion at Dreamforce on Semantic Data Modeling. I learned a lot from my fellow speakers and preparatory research to supplement my experience with this functionality. Where my fellow speakers focused on initial deployment and adoption, I was asked to talk about ongoing governance with three inspiration questions. I’m happy to share here those questions as well as my full thoughts.
I want to emphasize that this technology is very new. We - as users, developers, consultants, and decision makers - are still mapping the full breadth of the applications here. I am happy to be learning alongside my clients and learning from a variety of perspectives as we explore what’s possible.
Question 1: How do you coach your clients about what governance cadence should be for their semantic model?
We’ve got two aspects to this question, and in reality we need to apply both of them. Practically, our governance cadence is event-driven. Changes in data structure happen in response to business changes. In an Analytics context, this often means changing field definitions or formula computations, and propagating those changes to our SDM with potential impacts to downstream metric definitions. Each change of this kind requires governance review, although in many cases we can limit that review to the scope of the specific triggering change.
Ideally, we also engage ongoing review processes, both proactively and reactively. Proactively, we turn to automated testing with human reviews. Certain platforms, like Salesforce, provide us with environments made for testing Semantic Models. We’ll want to run these weekly after initial launch, and then slow down to quarterly if our drift and usage look good over time.
Reactively, we recommend a specially-trained person or team who view all end user feedback as valuable, as complaints don’t necessarily point to defects, but instead are valuable model evaluations. For example, user feedback about limits may verify that certain intentional guardrails are working as expected. This perspective is gold because it highlights where the abstraction of the Semantic Model meets the reality of the business world.
Question 2: How does governance differ for a 200-person company and a 10,000 person company?
The heart of the challenge of the difference between smaller and larger governance policies is complacency vs bureaucracy. Like a lot of real-world business challenges, our solutions look like dynamic balances.
A 200-person implementation has less need to compromise, and therefore theoretically can be simple or more elegant. They never are. We have a client who runs their entire - very successful - business from a single Salesforce object. From one perspective this is an elegant, minimalist solution, but it can make Analytics more complicated, and can limit the usefulness of a Semantic Data Model, which relies on relationships between objects. So while a single Semantic Data Model might be appropriate for a smaller implementation, we still want to apply best practices to the underlying structure to maximize our model utility. Similarly, we might be only need one person overseeing Model governance in a smaller business context, but that person still needs to think in longer timespans and stay engaged with their users to keep the model useful.
In contrast, a 10,000-person implementation knows up front that they’ll need to compromise, and so often have to commit to a data structure before they have all the usage info they need, and may be more susceptible to fallout from mis-aligned architecture. We once built combined Interaction and Activity analysis for a client, which is complicated because the Interaction/Activity to Parent Record relationship and the Attendee to Interaction/Activity relationship are both many-to-many. Initially we applied the best practices of clear, consistent Dataset grains, and relying heavily on lookup joins (which preserve both the data grain and the row count) like we had for other use cases. However, as usage expanded after deployment we saw that this traditional approach did not account for a number of their business applications and had some significant maintenance challenges that only appeared at a certain level of complexity. We ended up trading off our preferred single-grain Dataset strategy for a multi-grain Dataset strategy that gave us the flexibility we needed in both the data and visualization layers. The lesson here is that SDMs are new tech, and we’re thinking about our data in new ways, so we need to gather as much usage information as we can, and design data structures we can expand and maintain. For a larger implementation, this likely means we build multiple Semantic Models with more specialized focuses. By using shared metrics and definitions across those models, we can facilitate validation while still driving diverse Analytics across the whole breadth of the larger build. Governance at this scale requires at least a team if not layers of teams, and in this circumstance we have to be wary of overdoing it. If bureaucracy outpaces useful outcomes we refocus our governance layer and reconnect with our users to focus on utility.
Question 3: What breaks first, and how long after launch?
While it seems intuitive that something technical would break first after launch, from my perspective I think the first issue is going to be something political, and soon after deployment. Maybe first quarter close or first executive review after launch. We once built a predictive model for a client, and it indicated they could have better outcomes by choosing different partners to work with under certain circumstances. We showed the model to the executive team, and although the patterns rang true, they balked at deploying the model because the partner selection had subtle interpersonal and interorganizational factors that couldn’t be captured in the model. We plan for similar when deploying Semantic Models, and I think the solution is parallel as well: stay in touch with the end users and watch how the model is being applied in real life business situations.
Bonus Question: How do you know your semantic model is done enough to point an agent at it?
On this question I need to confess a bias. I studied Computer Science in college, so my first thought here was that “done” means the tests pass. However, to tie up the thread running through all three answers above, for a Semantic Data Model, the tests are our end users. We can test the sorts of questions we expect in an automated fashion, but as soon as we deploy a model with an agent surface, we’re going to start to see questions we didn’t expect and couldn’t test for. The utility of the model in those untested situations becomes our guiding star here.
In closing, there’s a lot we don’t know about how Semantic Data Models are going to change Analytics, but they’re some of the most interesting data structures I’ve seen in some time. My plan is to stay both curious and skeptical - to try new approaches while staying ruthlessly focused on what really helps our end users do their jobs.
