How to Add AI to Your Existing App Without a Rewrite

Most teams that ask me about AI assume the first step is a rebuild: new stack, new data layer, "AI-native" from scratch. It almost never is. The app you already have holds the users, the data and the workflows that make an AI feature useful. The job is to attach a well-bounded AI capability to it, prove it earns its keep, and grow from there.

This is the approach I use on client projects, in the order I use it.

Why you should not rewrite your app to add AI

A rewrite delays the AI feature by the length of the rewrite and adds risk that has nothing to do with AI. Martin Fowler's writing on the strangler fig pattern makes the general point well: in his experience, plans to replace a serious system in one go have "gone down in flames most of the time", while building new capability alongside the old one lets investment and returns arrive gradually and visibly.

AI features fit that pattern neatly. A model call is just another dependency: something your app sends a request to and gets a response from. You do not need to change your framework, your database or your hosting to make one. What you do need is a clear seam where the AI part plugs in, so it can be tested, monitored and switched off without touching the rest of the product.

Step 1: pick one feature users already need

The first AI feature should remove effort from a task people already do in your product, not invent a new one. Good first candidates share three traits: the input already exists in your system, a wrong answer is cheap to spot and fix, and you can tell whether it worked.

Notice the pattern: in every safe option a person sees the output before it changes anything. Autonomous actions can come later, once you have evidence the model behaves.

Step 2: put the AI behind one internal API

Your existing backend should call one internal endpoint, such as "summarise this ticket", and never talk to a model provider directly. That AI service owns the prompts, the model choice, retries and output parsing. The rest of your app only knows the contract: what goes in, what comes out, and what an error looks like.

This gives you three practical benefits:

• you can change models or providers without touching product code,
• you can add caching, rate limits and logging in one place,
• you can turn the feature off with a flag if something goes wrong.

The service can live in your current codebase as a module or run as a small separate deployment. If your app is a monolith in a language with weak AI tooling, a small TypeScript or Python service next to it is usually the cleanest choice. It is the same "beside, not inside" idea at the deployment level.

Between the AI service and the provider, consider a gateway. Vercel's AI Gateway, for example, records latency, token usage and cost per request, supports provider and model fallbacks and budgets, and can be called from any infrastructure, not only apps hosted on Vercel. Whether you use a managed gateway or a thin wrapper of your own, centralise keys, logging and spend limits rather than scattering them across the codebase.

Step 3: give the AI read-only access to the right data

Most of the value comes from your data, and most of the risk does too. The AI service should read through a narrow data access layer that returns only the records the current user is allowed to see, already shaped for the prompt.

• Reuse your existing authorisation checks. The AI layer should never see more than the user could see in the UI.
• Start read-only. Writing back to your database is a separate, later decision.
• Keep personal data out of prompts unless the feature truly needs it, and check your provider's data retention terms.

If the feature needs to answer questions over documents, you will need retrieval. I compare the options in RAG vs fine-tuning; for most existing apps, retrieval over your current data is the right starting point, and fine-tuning is rarely needed for a first feature.

Step 4: guard the edges in code

Treat the model as an untrusted input, because that is how security guidance treats it. OWASP's entry on prompt injection recommends specifying expected output formats and using "deterministic code to validate adherence to these formats", restricting the model's privileges "to the minimum necessary", and adding human-in-the-loop controls for privileged operations.

In practice that means:

• ask for structured output (JSON with a schema) and validate it before your app uses it,
• give tools and data access only what the feature needs,
• keep a human approval step for anything that sends, deletes, pays or publishes,
• mark content from users, emails or the web as untrusted when you put it in a prompt,
• log inputs and outputs so you can investigate problems.

When you are ready to let the AI take actions through tools, my guide to building an AI agent with MCP and Claude covers how to expose those tools in a controlled way.

Step 5: measure quality and running cost before you expand

An AI feature is not done when it demos well; it is done when you know how often it is right and what it costs per use. Before launch I build a small test set of real examples with expected results, run every prompt or model change against it, and keep tracking after launch with user feedback such as thumbs up or down, edits and retries. It is the same discipline I describe in the evals that keep AI agent projects from failing.

Running cost is mostly tokens, and there are well-documented levers for it. With Claude, for example:

Prices change, so check the current pricing pages when you plan. The structure matters more than any one figure: a long, stable system prompt plus a short user input is the shape caching rewards, and overnight jobs such as tagging or summarising backlogs are a natural fit for batches.

What a sensible first rollout looks like

Ship to a small group behind a feature flag, watch the logs and the evaluation scores, and widen access in steps. A typical sequence on my projects:

• Internal users only, with logging of every request.
• A small share of customers, with a visible way to give feedback.
• Everyone, with budgets and alerts in place.
• The next feature, reusing the same AI service, data layer and guardrails.

The fourth step is where the approach pays off. The second and third features are much cheaper than the first, because the seam, the gateway, the evaluation set and the guardrails already exist.

Work with me

I add AI features to existing products: scoping the first feature, building the AI service beside your app, and setting up the evaluation and cost controls that keep it reliable. See AI integration and full-stack development, or browse my work and book a scoping call.