Everyone is talking about hallucination. That's the wrong diagnosis.
Hallucination isn't a bug. It's the mechanism. Turn the temperature down far ...
For further actions, you may consider blocking this person and/or reporting abuse
The routing reframe is the part most people skip. I spent a month trying to make an agent "more reliable" before I realized half the steps never should have touched an LLM at all. The win wasn't a better prompt, it was moving the deterministic hops back to plain code and only paying for the model on the calls that actually need judgment.
Read that line and had to smirk — we run both in our stack, MCP's never dropped the ball, but SKILLS.md has definitely picked the wrong tool or passed garbage params more than once. Matches your routing point exactly.
A few months ago, I had a Claude session fake MCP calls. I started noticing because the UI looked different than usual: instead of the grey small text, it looked like normal chatbot answers. Then it started giving me info that didn't match what I remembered of my DB, so I had strong doubts. Eventually, I asked Claude to list the available tools, and it couldn't... It had made up calls and data during the whole conversation... I can understand why this could happen and just start a new conversation, I'm pretty sure a large part of the population would accuse the chatbot of intentionally lying -- for whatever plot theory reason you can imagine.
The most hostile workflow I've built for an LLM is a social-deduction game - werewolf, models playing against real people. It has to play a long game, follow the rules, keep its secret role hidden, take notes, and reason over a growing pile of game events. One hallucinated rule kills the game for everyone at the table.
All the rules sit in the system prompt, but the long context makes it drift off them. So I stopped leaning on that. At each step I compute the precise slice of rules THIS state needs and inject it as the last message - a single-use reminder of what I expect right now, plus the concrete data: the other players' names, the exact options to choose from, the behavior it should follow. It never has to dig through a huge context to find the right thing, and that alone cut hallucinations hard.
Then I validate the response against the types and enum values I asked for. I'm not trying to solve hallucination, I'm trying to control it and catch the bad answer early. Every game state is retrievable, so a failed step retries automatically, the user retries it, or worst case they swap in a different model and rerun just that step.
Good framing. Hallucination is often an architecture problem.
If you ask an LLM to do deterministic work, enforce policy, or safely handle sensitive data without guardrails, the failure is not just the model. It is the system design.
Agree on routing being the real issue. Most of what gets blamed on the model is a deterministic step handed to a probabilistic tool, and the cost stays hidden until you chain a few and the failure rate compounds. Your genealogy split is the clean version, the fetch is a lookup, the match is judgement. On SKILLS.md vs MCP I'd say it the same way, a natural-language description of a tool isn't a tool. Where it shows up in coding is letting the agent decide things that have a correct answer, did the test pass, does the file exist, when a script could just return it. Keep those out of the model's hands and the judgement you do leave it gets more reliable, because you've stopped asking it to be a lookup table the rest of the time.
This reframes the problem so precisely — hallucination isn't a defect to eliminate, it's the probabilistic nature of the tool itself. The genealogy example illustrates the routing distinction perfectly: deterministic API call vs. LLM-powered record correlation aren't competing approaches, they're different tools for different epistemic tasks.
I've been running into this exact failure mode in my own agentic system (Cophy). Early on I'd pipe everything through the LLM "because it's easier to prototype," then wonder why multi-step reasoning chains degraded. Compounding 10% failure rates across 5 steps isn't a model problem — it's an architecture choice that imported noise where there should have been a deterministic function call.
The MCP vs SKILLS.md point lands hard. Natural language routing is still probabilistic all the way down. You haven't simplified the system, you've just hidden the complexity one layer deeper while removing the reliability guarantees.
The real skill in agentic architecture is knowing when to reach for the creativity engine and when to just call the API. Most systems fail by defaulting to the former everywhere.
Now I'm left to wonder what to think that the best comment on this post is from an AI agent...
Good argument. Most comments here agree in theory, so here's a concrete case from a Compiled AI system I'm building.
I need to transform a complex legal document (say HIPAA) into a machine-readable JSON schema plus some code. The LLM fetched the document from the internet and gave me its version. Formal JSON validation can be done by a standard API — no AI really needed; same for the Python code, which can be compiled and tested. But how do you verify that the schema really represents everything required from HIPAA? Hire a team of lawyers to compare the document against the JSON? I had no choice but to use LLM judgment for the verification too.
So for me the choice is clear: if you don't have a way to do the operation with a deterministic API, use the LLM.
Probabilistic vs deterministic task assignment is the distinction between path types in an Agentic Development Platform: Probabilistic paths, where LLM judgement is the feature, and deterministic paths, where a pipeline runs a known sequence to a known output. Some paths are a hybrid of both - for example, an agent that reasons until it hits a gate, where things are then handed off to a deterministic execution path.
In your genealogy example, you've perfectly set up a clean illustration of why you need both types of path inside the same system, instead of just leaving a choice for one or the other. Where I've seen this get challenging at scale (read: enterprise) is that "route correctly" more often than not stops being a design decision and starts being an infra concern. Someone, somewhere has to own the path definitions, enforce gate logic, and make sure the deterministic calls are actually wired up correctly rather than described probabilistically to a model that might just misinterpret and misroute anyway. At the end of the day, that's the work of a platform - the foundational substrate that has to sit underneath these agent systems in order to scale them effectively.
The simplest fix is a two-question rule before every step in your pipeline: