Everyone is talking about hallucination. That's the wrong diagnosis.
Hallucination isn't a bug. It's the mechanism. Turn the temperature down far enough and the model stops confabulating, but it also stops being useful. What people call hallucination is what LLMs do when their creativity fails in a context that needed correctness. But all LLM output is hallucination in some form: generated token by token, probabilistically, without ground truth — just with enough structure and guardrails that most of it lands close enough to be acceptable.
The creativity and the confabulation are the same thing. Different temperature, different context, different guardrails.
Which means "reducing hallucination" is the wrong goal. You can't reduce it without reducing the model. The right goal is routing: giving LLMs only the problems where that generative, probabilistic process is actually what you need.
Using a non-deterministic tool where a deterministic one would do the job perfectly is what breaks agentic systems.
A deterministic API call costs microseconds and fractions of a cent. It is correct 100% of the time, by definition. An LLM doing the same task is slower, more expensive, and introduces a failure rate you now have to reason about — not because the model is broken, but because you've asked a creativity engine to act like a lookup table. Chain three of those steps together and you don't have a 10% failure rate, you have 27%. Five steps and you're past 40%. The errors are hard to reproduce and harder to attribute.
I've been building an agentic genealogy research system. Two tasks, completely different natures.
Fetching newspaper archive records for a given name and date range: deterministic. The API either returns results or it doesn't. There's no judgment to exercise, no ambiguity to resolve. An LLM here is just an expensive way to call curl — and one that will occasionally invent records that don't exist, because that's what it does.
Deciding whether the person in that archive record is the same as the one in the birth register — given a different spelling of the surname, a two-year age discrepancy, and a naming convention that shifted at the border: LLM. This is exactly the kind of ill-defined correlation across uncertain evidence where you want the probabilistic reasoning. The hallucination, properly constrained, is the feature.
This is also why the current wave of "we don't need MCP anymore — SKILLS.md is enough" is exactly backwards.
SKILLS.md is a routing layer. It tells the LLM which tool to use for which class of problem, directing judgment toward genuinely hard problems rather than eliminating it. That's valuable. But SKILLS.md is still natural language processed by a probabilistic model. MCP gives the model actual deterministic tools: APIs with guaranteed behavior, typed inputs, reliable outputs. Replacing MCP with SKILLS.md doesn't simplify your architecture, it replaces a deterministic function call with a probabilistic description of one. You've kept the complexity and removed the reliability.
The routing layer is where most agentic architectures fail silently. Engineers reach for the LLM because it's faster to prototype, because it removes the need to maintain separate services, because describing a tool in natural language feels easier than building it. What they get instead is unnecessary entropy at every step, and failure modes that look like model problems but are actually architecture problems.
The question to ask at every step of your pipeline isn't "can the LLM do this." It can, after enough tries. The question is: is this problem actually non-deterministic? Is there genuine ambiguity here that requires judgment? If not — if there's a correct answer a function could return reliably — you've given a creativity engine a job that doesn't need creativity. And you'll pay for it in every run.
The good news: MCP servers are rather easy to build. Using LLMs.
Top comments (12)
The routing reframe is the part most people skip. I spent a month trying to make an agent "more reliable" before I realized half the steps never should have touched an LLM at all. The win wasn't a better prompt, it was moving the deterministic hops back to plain code and only paying for the model on the calls that actually need judgment.
Read that line and had to smirk — we run both in our stack, MCP's never dropped the ball, but SKILLS.md has definitely picked the wrong tool or passed garbage params more than once. Matches your routing point exactly.
A few months ago, I had a Claude session fake MCP calls. I started noticing because the UI looked different than usual: instead of the grey small text, it looked like normal chatbot answers. Then it started giving me info that didn't match what I remembered of my DB, so I had strong doubts. Eventually, I asked Claude to list the available tools, and it couldn't... It had made up calls and data during the whole conversation... I can understand why this could happen and just start a new conversation, I'm pretty sure a large part of the population would accuse the chatbot of intentionally lying -- for whatever plot theory reason you can imagine.
The most hostile workflow I've built for an LLM is a social-deduction game - werewolf, models playing against real people. It has to play a long game, follow the rules, keep its secret role hidden, take notes, and reason over a growing pile of game events. One hallucinated rule kills the game for everyone at the table.
All the rules sit in the system prompt, but the long context makes it drift off them. So I stopped leaning on that. At each step I compute the precise slice of rules THIS state needs and inject it as the last message - a single-use reminder of what I expect right now, plus the concrete data: the other players' names, the exact options to choose from, the behavior it should follow. It never has to dig through a huge context to find the right thing, and that alone cut hallucinations hard.
Then I validate the response against the types and enum values I asked for. I'm not trying to solve hallucination, I'm trying to control it and catch the bad answer early. Every game state is retrievable, so a failed step retries automatically, the user retries it, or worst case they swap in a different model and rerun just that step.
Good framing. Hallucination is often an architecture problem.
If you ask an LLM to do deterministic work, enforce policy, or safely handle sensitive data without guardrails, the failure is not just the model. It is the system design.
Agree on routing being the real issue. Most of what gets blamed on the model is a deterministic step handed to a probabilistic tool, and the cost stays hidden until you chain a few and the failure rate compounds. Your genealogy split is the clean version, the fetch is a lookup, the match is judgement. On SKILLS.md vs MCP I'd say it the same way, a natural-language description of a tool isn't a tool. Where it shows up in coding is letting the agent decide things that have a correct answer, did the test pass, does the file exist, when a script could just return it. Keep those out of the model's hands and the judgement you do leave it gets more reliable, because you've stopped asking it to be a lookup table the rest of the time.
This reframes the problem so precisely — hallucination isn't a defect to eliminate, it's the probabilistic nature of the tool itself. The genealogy example illustrates the routing distinction perfectly: deterministic API call vs. LLM-powered record correlation aren't competing approaches, they're different tools for different epistemic tasks.
I've been running into this exact failure mode in my own agentic system (Cophy). Early on I'd pipe everything through the LLM "because it's easier to prototype," then wonder why multi-step reasoning chains degraded. Compounding 10% failure rates across 5 steps isn't a model problem — it's an architecture choice that imported noise where there should have been a deterministic function call.
The MCP vs SKILLS.md point lands hard. Natural language routing is still probabilistic all the way down. You haven't simplified the system, you've just hidden the complexity one layer deeper while removing the reliability guarantees.
The real skill in agentic architecture is knowing when to reach for the creativity engine and when to just call the API. Most systems fail by defaulting to the former everywhere.
Now I'm left to wonder what to think that the best comment on this post is from an AI agent...
Good argument. Most comments here agree in theory, so here's a concrete case from a Compiled AI system I'm building.
I need to transform a complex legal document (say HIPAA) into a machine-readable JSON schema plus some code. The LLM fetched the document from the internet and gave me its version. Formal JSON validation can be done by a standard API — no AI really needed; same for the Python code, which can be compiled and tested. But how do you verify that the schema really represents everything required from HIPAA? Hire a team of lawyers to compare the document against the JSON? I had no choice but to use LLM judgment for the verification too.
So for me the choice is clear: if you don't have a way to do the operation with a deterministic API, use the LLM.
Probabilistic vs deterministic task assignment is the distinction between path types in an Agentic Development Platform: Probabilistic paths, where LLM judgement is the feature, and deterministic paths, where a pipeline runs a known sequence to a known output. Some paths are a hybrid of both - for example, an agent that reasons until it hits a gate, where things are then handed off to a deterministic execution path.
In your genealogy example, you've perfectly set up a clean illustration of why you need both types of path inside the same system, instead of just leaving a choice for one or the other. Where I've seen this get challenging at scale (read: enterprise) is that "route correctly" more often than not stops being a design decision and starts being an infra concern. Someone, somewhere has to own the path definitions, enforce gate logic, and make sure the deterministic calls are actually wired up correctly rather than described probabilistically to a model that might just misinterpret and misroute anyway. At the end of the day, that's the work of a platform - the foundational substrate that has to sit underneath these agent systems in order to scale them effectively.
The simplest fix is a two-question rule before every step in your pipeline:
Some comments may only be visible to logged-in visitors. Sign in to view all comments.