In this fourth post in my series, Observability: Why Did Your Agent Do That?, I look at the AI Agent observability problem.

(You can find the three earlier posts here:  Part 1: Autonomy with Brakes: Why Refusal Comes First ,  Part 2: Disobedient or Just Probabilistic , and Part 3: The Importance of the Context Layer.)

At Real Story Group, we’ve researched more than twenty Agent AI tools and have advised several enterprise clients on different Agent AI aspects. Over this period, one thing that has surprised me most is that these tools don’t explain themselves well.

What do I mean by that?

Let’s take a concrete example. When a marketing agent recommends pursuing email engagement over paid social, or allocates 35% of the budget to a channel you capped at 25%, there is no auditable trail that explains why. The agent just acted. The logs show that it acted. But the rationale behind the actual decision logic is not captured anywhere. At least, not in a human-friendly way.

Yet, as a MarTech leader, the time will come sooner rather than later when you will want to know the rationale.

And this is true across frameworks. CrewAI, LangGraph, AutoGen, and even vendor-embedded agents inside Salesforce or Adobe. None of them produces what a MarTech leader actually needs: a human-readable explanation of why the agent made the choices it made.

That gap is the topic of this post. And if you're a MarTech leader evaluating or piloting agentic tools, it should be near the top of your considerations.

What you get vs. what you need

You will find some degree of basic internal tracking. Agent frameworks that RSG evaluates, like CrewAI, LangGraph, and AutoGen, will produce activity logs. Sometimes verbose ones. You get token counts, latency, error rates, and maybe a raw chain-of-thought dump. Some vendors wrap this in dashboards. These are developer tools, useful for debugging during a build.

But what an enterprise martech leader needs is actual decision traceability. Let’s say your CMO asks why the agent recommended email over paid social. Compliance wants proof that the agent respected GDPR consent rules for the German segment. Finance asks why the agent blew through a channel cap. A chain-of-thought log that's 4,000 tokens long and half-hallucinated doesn't answer any of these questions.

At RSG, we distinguish between activity logging and decision traceability. Most frameworks give you the former. Enterprises need the latter. And in most cases, you have to build your way across that gap yourself.

What does Observability mean in real life?

Observability is a human-readable trace, right from input to final output. It clearly shows the end goal, the context the agent operated under, the tools it called, along with arguments/returns, the sources behind each claim, the policy checks that passed or failed, and the final status, including cost and time.

A case study

One enterprise we advised built an agent that generated weekly campaign recommendations. The agent pulled from multiple data sources, applied some budget logic, and produced a plan with channel allocations and messaging themes.

In week three, the agent recommended a large investment in a channel the brand had explicitly deprioritized. The team wanted to understand why. They did the obvious and went to the logs.

The logs showed the agent had called a market research tool, received data, called a budget allocation tool, and produced output. That's it. They could see what the agent did. They couldn't see why it chose that channel, which data source drove the recommendation, or whether the agent had access to the current channel policy or an outdated one. They couldn't determine if the context packet was the right version.

The team spent two days manually reconstructing the decision. By then, the plan had already gone to stakeholders. We see this pattern repeatedly. The agent works until someone asks "why," and nobody can answer.

What a production-grade run log actually requires

Building a “why” trail means capturing, for every agent run, not just what happened but why it happened. That means logging the goal and inputs, which version of the context packet was active, every tool call with its exact query and response, the specific source behind each claim in the output, reason codes for key decisions (why this channel, why this allocation), any refusal events and what was missing, and cost.

None of this is really new or difficult to build. It requires structured events data to be stored somewhere queryable. The difficult part comes when deciding what counts as a "reason" in your organization, and getting someone to own the schema.

Vendor observability vs. enterprise observability

This isn't a shortcoming of one tool. It's a gap across the whole category of agent platforms. We've evaluated observability features in LangSmith, CrewAI's logging, and several third-party tracing tools. All of them are oriented toward developers, not toward the stakeholders who will actually need to understand and trust the output: compliance, brand, finance, operations.

Platform vendors seem aware of this. Some are adding new capabilities here. But the gap between what ships today and what an enterprise compliance team, or CFO or CMO requires remains quite wide. And it applies equally to standalone agent frameworks as well as vendor-embedded agents, such as the Agentforces and Copilots that marketing teams may enable without fully appreciating what they can't see.

Observability without context is noise

This ties back to Part 3 in this series on Agents. A why trail that captures tool calls but not the context the agent was operating under will remain incomplete.

Consider the agent that recommends investing in Instagram influencers for a B2B enterprise. The log shows it called a social listening tool and received high-engagement data. Looks reasonable. Until you check whether the agent had access to the business model context that would have flagged Instagram as an off-target channel. If you can't tell which version of the context packet was active, which definitions were in play, and whether freshness checks passed, you're logging activity without meaning.

A maturity progression

Briefly: not every team needs full decision traceability on day one. We think about it in tiers.

At L1, you log what happened (inputs, outputs, tool calls, cost, refusals).

At L2, you log why (evidence binding, reason codes, context packet version, reproducibility anchors).

At L3, you track whether it's improving (reviewer edits with reason codes, edit-distance metrics, feedback loops).

Most of the enterprises we work with are L0 or L1. Getting to L2 is where trust gets built.

Why this matters for your agentic strategy

If you're a MarTech leader evaluating or piloting agentic tools, and if observability isn't on your checklist yet, it should be.

The current generation of agentic platforms ships without the decision traceability that your compliance, finance, and brand teams will require. That's true of standalone frameworks your team might build on. It's equally true of vendor-embedded agents; the ones that arrive pre-packaged inside your existing marketing suite and that someone on your team may enable with a few clicks, not realizing that no one will be able to explain what the agent did or why.

This isn't a reason to avoid agentic AI outright. Instead, go in with your eyes open. Budget for the “why” trail and staff it appropriately. Treat observability as a first-class requirement in your vendor evaluation, not an afterthought. And be sceptical when a vendor tells you their agent is “transparent” or “explainable.” Ask them to show you the run log for a specific decision and how you can audit specific rationales. If they can't, you know where you stand.

As I reflect on this series of posts– covering refusal, variance management, context, and now observability–a pattern keeps emerging. The technology works well enough in a demo, but much less effectively in production. The gap between demo and production is mostly organizational, yet still partly technical as well. It requires definitions, governance, logging, and human review. Closing the organizational gap is, paradoxically, human-intensive and therefore another cost element to add to the negative side of the ledger.

In short, the build-and-iterate effort required to make agents production-worthy is larger than most teams expect, and it keeps growing as we learn more.

But that's a topic for another post. Perhaps part 5. Stay tuned.

For More...

If your firm is an RSG corporate member, you have access to the complete case study and learnings, as well as a private review of your agentic strategy to date. For more practical support converting your pilots to productive solutions, contact us about consulting offerings.