Every impressive AI agent demo runs against a clean, modern API. Every real enterprise runs on something else: a 20-year-old ERP, a mainframe that only speaks a fixed-width file format, a system whose vendor went out of business a decade ago and whose "integration layer" is a nightly CSV dropped on an FTP server. The gap between the demo and the deployment is almost always this gap.
We do a lot of this work - putting an agent in front of systems that were never designed to be talked to by software, let alone by a probabilistic language model. Here is what we have learned about making it work without breaking the thing the business actually runs on.
Wrap, do not replace
The first temptation is to "modernize" the legacy system so the agent has something nice to call. Resist it. That ERP processes payroll, or inventory, or claims. Touching its internals is a multi-year, high-risk project, and the agent will be obsolete before you finish. Instead, build a thin integration layer around it: a small service that exposes a few clean, well-defined endpoints and translates them into whatever the legacy system understands - a SOAP call, a stored procedure, a screen-scrape, a file drop. The agent talks to your clean layer. Your layer absorbs the ugliness.
The agent should never touch the system of record directly
This is the single most important rule. An agent will, eventually, do something you did not predict - call the same action twice, pass a malformed value, retry a request that already succeeded. If that lands directly on the system that holds the truth about money or stock or patient records, you have a real incident. So we always insert a boundary: the agent proposes an action, the integration layer validates it, and only validated, well-formed, idempotent operations reach the core. The legacy system gets clean input or no input.
Make every write idempotent
Old systems rarely forgive a double-submit. Run the same "create invoice" twice and you get two invoices. Because agents retry - on timeouts, on ambiguous responses, on a step that looked like it failed but did not - you must assume duplicate calls will happen. We give every operation an idempotency key: a deterministic identifier the layer checks before it acts. Seen this key already? Return the previous result instead of doing it again. This one pattern prevents a whole category of expensive, embarrassing errors.
Translate the data model, do not leak it
Legacy systems carry decades of accumulated quirks: status codes like "07" that mean "approved-pending-review," date formats that assume a two-digit year, a customer record split across three tables for reasons lost to history. Do not make the agent reason about any of that. Your integration layer should translate the raw legacy data into clean, self-describing concepts - "status: approved, review_required: true" - before the agent ever sees it. The model is good at language and intent. It is bad at remembering that "07" is special. Keep the tribal knowledge in code, not in the prompt.
Plan for the system being slow, flaky, or asleep
That nightly batch window. The maintenance reboot. The query that takes 40 seconds because it scans a table with no useful index. Modern API assumptions - fast, always-on, returns JSON - do not hold here. Build for it: generous timeouts, retries with backoff, a queue for operations that cannot complete in real time, and a clear, honest response to the user when the core system is simply unavailable. An agent that says "I have queued your request and the system will process it tonight" is far better than one that hangs or hallucinates a confirmation.
A grounded example
A manufacturing client wanted an agent that could answer "when will order 4471 ship?" The data lived in an ERP from the early 2000s, reachable only through a clunky internal API that returned a wall of coded fields. We did not touch the ERP. We built a read-only service that called it, translated the coded fields into plain status concepts, cached results for a few minutes to spare the old system, and exposed one clean endpoint. The agent queried that. Total footprint on the ERP: minimal. Risk to production: near zero. Value to the floor staff who used to phone the planning office: immediate.
The real skill is boundaries, not models
Connecting an agent to a legacy system is rarely an AI problem. It is a systems-engineering problem - idempotency, validation, translation, graceful degradation - with an agent on one end. The teams that succeed treat the legacy system with respect, wrap it carefully, and never let the unpredictable part of the stack touch the irreplaceable part. That discipline is the whole game.
About Shanti Infosoft: Shanti Infosoft is a CMMI Level 5 AI development company that has delivered 700+ projects across 16+ industries. We help teams move from AI ideas to dependable, production-grade software - shantiinfosoft.com | AI integration services.
If you need an AI agent to work with a legacy ERP or CRM rather than around it, we can map the safest integration path for your stack. Talk to our team.
Related reading: The Build-vs-Buy Decision for AI Agents: Templates, Managed APIs, or Custom
Rishabh Jain is a Director at Shanti Infosoft, where the team builds AI agents and automation for real business operations.
Top comments (0)