When a Successful Call Is Still the Wrong Move
Picture this: a fighter's corner calls for a takedown. The referee signals it's clean, the execution is flawless—but the fighter wasn't supposed to shoot. They were only checking if the takedown was available, not committing to it. In the fight game, that kind of misread can cost a round. In AI systems, it's the same story, just with different stakes.
Take a customer service agent for a fight promotion company. A fan asks, "Can I still get a refund on this ticket?" The agent checks the order, reviews the refund policy, confirms the event hasn't happened, and then—without explicit permission—processes the refund. The tool call returns success. The log looks perfect: refund_order, order_id: 20260710001, result: success. But the fan only asked if they were eligible, not to actually refund. The action was wrong, even though the call succeeded.
This is the core problem Knowledge-driven Computing (KDC) tackles: just because a tool executes doesn't mean the judgment behind it was correct or authorized. For MMA, think of it like throwing a punch that lands—but it lands after the bell. The technique was fine, the timing was off, and the consequence is a penalty.
The Hidden Gap in Traditional Logs
In old-school software, a refund lives in a clear path: user clicks button, confirms, backend checks order and permissions, risk rules run, transaction executes. If something fails, there's retry or rollback. The why is baked into the code. You can trace it.
But with AI agents, the model decides on the fly. It interprets the user's goal, picks tools, and decides whether to act based on natural language. The same sentence "Can I refund?" could mean "Am I eligible?" or "Please refund now." That choice isn't in a code path—it's made at runtime. So when something goes wrong, you're left with a log that says what happened, but not why. It's like watching fight footage without knowing the game plan.
Reasoning Objects: Making Judgment Auditable
KDC's answer is the Reasoning Object. It's not a prompt, not a trace, and definitely not a full chain-of-thought dump. It's a structured, external record of a judgment—something you can audit and reference.
For a high-impact decision, a reasoning object should capture:
- Goal: What was the task really about?
- Context: User, order, time, session, system state.
- Knowledge references: Which rules or policies were used, and which versions?
- Memory references: Did past preferences or experiences influence this?
- Evidence: What facts or confirmations support the conclusion?
- Key judgments: What checkpoints led from goal to conclusion?
- Conclusion type: Is this a fact, a suggestion, a plan, or an action request?
- Uncertainty: Is the evidence solid, or do we need more info?
- Risk: What's the impact if this judgment is wrong?
- Action suggestion: Should we call a capability?
- Feedback: Did reality confirm or refute the judgment later?
In the ticket refund case, a solid reasoning object would separate two things: "The order is eligible for a refund" (a fact) and "The user explicitly requested a refund" (an authorization). They're not the same, but they often get merged. If the object marks the task as "eligibility inquiry" and flags "user confirmation" as a prerequisite, the system can stop the agent from jumping from advice to action.
From Judgment to Action: The Causal Chain
A governed AI action shouldn't just leave a tool call. It needs a chain: business goal → knowledge and evidence → reasoning object → capability suggestion → policy and permission check → capability call → execution result → real-world feedback.
The reasoning object sits between knowledge and action. It's the anchor that explains why this action happened. It also makes refusal a normal outcome. If evidence is weak, the goal is vague, or the user hasn't confirmed, the object can say "no action"—and the system doesn't have to wait for a tool to fail to know the premise was wrong.
Not every action needs the full treatment. Read-only queries can be light. But refunds, payments, permission changes, external notifications—those need at least the equivalent of a reasoning object. The bigger the impact, the more explicit the responsibility.
Reasoning Runtime: Managing Judgments, Not Just Prompts
The reasoning runtime is what creates, updates, and uses these objects. It's not the model itself. It classifies goals, assembles context, binds evidence, flags risks, suggests capabilities, and links feedback back to the original judgment. It's also not a prompt orchestrator—prompts organize a single input, but the runtime manages the whole judgment lifecycle.
You don't need a separate platform for this. Early on, an app module with structured logs and policy checks works. What matters is that someone owns the judgment lifecycle. The runtime proposes actions but doesn't execute them. That's where Tool, Skill, and Capability come in.
Tool, Skill, Capability: Three Layers of Execution
These three terms get thrown around, but they're different layers:
- Tool: How to execute a specific operation. Like
get_order_statusorrefund_order. It's the raw action. - Skill: How to organize multiple operations around a business goal. "Process a ticket refund" might include goal clarification, order lookup, policy check, eligibility judgment, user confirmation, refund execution, and follow-up. A skill defines the flow, preconditions, and failure handling.
- Capability: How to execute within governance boundaries. This is the semantic wrapper that adds permissions, risk level, owner, and audit requirements. A capability says "you can call refund, but only if the user confirmed and the risk is acceptable."
Without this separation, an action suggestion falls straight onto a tool, and all the governance gets lost. In MMA terms, a tool is a punch, a skill is a combination, and a capability is knowing when to throw it—and when to hold back.
Why Prompts Aren't Enough for High-Risk Actions
Teams often write in the system prompt: "You must get user confirmation before refunding." That's a good instruction, but it's not a safety net. Prompts can be ignored, misunderstood, or overridden by context. If the raw refund_order tool is open to the model, you're betting everything on a probabilistic guess.
Governance needs to be in the system, not the prompt. That means checks like: Does the current user have permission? Is the risk level high enough to require extra steps? Are all preconditions met? Is human-in-the-loop needed? Are there rate limits or circuit breakers? What happens if the action fails halfway? Who is accountable, and what's the audit trail?
In the ticket refund case, the control plane should treat "explicit user confirmation" as a hard requirement for the transactional refund capability. Even if the agent reasons wrong or the skill skips a step, the capability won't execute. Governance scales with impact—read-only checks are cheap, but irreversible actions get strict.
The Agent's Role: Coordinator, Not Container
If you treat the agent as a universal do-it-all, it ends up owning everything: understanding goals, holding context, forming judgments, picking tools, executing, handling failures, and generating audits. That sounds simple, but it's a mess when things go wrong. You get "the agent was wrong" with no clue which layer failed.
KDC puts the agent on top of objects, runtimes, and governance. The agent can interpret goals, organize context, coordinate reasoning, pick skills, suggest capabilities, and handle feedback. But high-impact judgments need reasoning objects, and high-risk actions need capability governance. The agent can suggest, but it doesn't get to execute just because it thought it through.
This isn't about turning the agent into a script. The agent still adapts dynamically—it's just that the critical responsibilities aren't hidden in a temporary context. The sequence diagram in the source shows it clearly: user goal → agent organizes context → reasoning runtime produces a reasoning object → agent requests a capability → control plane checks permissions and risk → executes or asks for confirmation.
Working With Existing Infrastructure
KDC doesn't reinvent the wheel. It builds on MCP for tool discovery, agent frameworks for orchestration, API gateways for traffic, IAM for identity, policy engines for rules, and workflow systems for approvals. What KDC adds is connecting them all to a single business causal chain.
For example: What business capability does an MCP tool represent? Why did the agent suggest calling it? What knowledge did the reasoning reference? What's the risk? What policy context did the engine use? And did the result actually change the real world? The capability control plane is a conceptual abstraction, not a product—you can implement it with existing infrastructure.
Practical Next Steps for Your MMA System
Don't start by designing a new platform. Pick a high-impact action in your system—like a refund, a payment, a permission change, or sending a notification. Take a real log entry and try to reconstruct the full chain: business goal, knowledge and memory basis, auditable judgment and risk, skill and capability choice, permission and policy check, user confirmation or human-in-the-loop, execution result, and real-world feedback.
Then look for four gaps:
- Causal gap: You see the tool call but can't trace the goal or rationale.
- Semantic gap: The tool has parameters but no business meaning, risk, or owner.
- Governance gap: Critical boundaries exist only in prompts, not system policies.
- Feedback gap: The API returned success, but you never verified the real-world outcome.
If you can only piece this together from chat logs and scattered traces, your action audit chain isn't stable. For enterprise AI—and for MMA—the key isn't being able to call tools. It's knowing when not to.
The Real Test: Knowing When to Stop
Getting a model to call a tool is easy now. The hard part is putting runtime judgment, business risk, and execution permission into one governable chain. Reasoning objects let the system explain why it judged as it did. Skills let you reuse goal-level flows while keeping flexibility. Capabilities add semantics, permissions, risk, owner, and lifecycle. The control plane makes sure critical boundaries don't rely on the model's self-discipline.
So remember: a successful call doesn't mean the judgment was correct. A correct judgment doesn't mean you're authorized to act. And an API success doesn't mean the real-world goal was achieved. Enterprise AI—just like a fighter—needs to act when it should, stop when unsure or out of bounds, and make every major move explainable, auditable, and accountable.
But even with perfect judgment and execution, you're not done. Reality shifts, knowledge expires, users correct you, and capabilities fail. The next piece of the KDC series will tackle how history and feedback become memory—and how to avoid compounding your mistakes. That's the next round.
Comments (0)
Please sign in to post a comment.
Don't have an account? Create one
No comments yet. Be the first to comment!