API and UI integration: embedding AI in an existing product or internal tool

The hard part of adding AI to a product is not the model call; it is everything around it — the endpoint contract, the timeout, the screen that shows the result and the button that takes it back. This is for teams that already have a product or an internal tool and want an AI feature in it that survives real users and a provider outage.

What API and UI integration actually covers

A language model is a service dependency with unusual properties: slow, nondeterministic, billed per token and occasionally unavailable. Integration means designing three things around it so the product does not inherit those quirks: the API contract, the interface that puts results in front of a person, and the tools through which the model reads or changes data.

The model is usually the smallest part of the work. In a typical Hilluter AI integration project the prompt is a few hundred lines; the endpoint, UI states, fallback path, logging and evaluation set are the rest.

API design for a model-backed endpoint

1. Structured outputs

The endpoint returns JSON validated against a schema, never free text the client parses. Use the model's structured-output mode, validate server-side, and treat a schema violation as a model error, not a client problem. A field with five allowed values is an enum; a date is a date. This is also the first guardrail layer: nothing malformed reaches the UI.

2. Idempotent endpoints

Model calls time out, clients retry, users click twice. Every request carries an idempotency key (a ticket id plus a version, or a client-generated UUID), and the same key returns the stored result instead of spending tokens on a second, slightly different answer. Without it, retries turn one bad draft into three different drafts in the same thread.

3. Timeouts and the fallback path

Set an explicit budget: 5–8 seconds for anything interactive, minutes for batch jobs. When the budget is exceeded or the provider returns errors, the endpoint returns a defined "unavailable" state and the product does what it did before the AI feature existed — shows the template, leaves the field empty. The UI never spins forever. Design this path first.

4. Streaming vs. batch

Stream when a person is waiting and the output is prose: a drafted reply, a summary. Perceived latency drops from eight seconds to under one. Do not stream JSON that must be validated before use; the client cannot act on half an object. Batch when nobody is waiting: 500 documents overnight go through a queue, a worker and a status endpoint.

UI patterns that earn trust

The interface decides whether the feature is used or quietly turned off.

Suggestion mode is the default. Autopilot — acting without a person looking — is earned later, per action type, when the evaluation harness and production monitoring show the accuracy is there. The fifth pattern is feedback capture: accept, edit and reject are logged with the input, the output and the prompt version. Those records are labeled data for the harness.

Tool interfaces, MCP servers and versioned prompts

When the model needs to read or change something — order status, a ticket field — it gets a tool, not database access. We expose tools through MCP servers, one per system: a read-only Postgres server, a Notion server scoped to one workspace, a Microsoft 365 server that reads a mailbox but cannot send. Each tool has a narrow schema, and the permission lives in the server, not in the prompt. A tool that does not exist cannot be called, which is the cheapest guardrail there is.

Prompts are code: they live in the repository with a version identifier and a changelog, and the model is pinned to an exact version string. A prompt change goes through a pull request, runs the evaluation harness, and ships with a version number logged on every request, so any bad output can be traced to the prompt and model that produced it. Rollout and logging are covered in deployment and monitoring.

A worked example: reply drafts inside an internal CRM

A B2B software company with a 35-person support team runs its own internal CRM and wants AI-drafted replies in the ticket view. The decisions, in order:

• Endpoint. POST /api/drafts takes a ticket id and an idempotency key and returns {draft, tone, cited_lines, confidence_bucket}. The draft streams; the metadata arrives as one validated object at the end.
• Budget and fallback. Six seconds. On timeout or provider error the panel shows the three existing reply templates and a "draft unavailable" notice.
• UI. Side panel, cited ticket lines highlighted. The agent edits and sends; the system never sends. Accept, edit and discard are logged with prompt version and model id.
• Tools. One MCP server with two read-only lookups: order status and plan tier.
• Prompt versioning. Versions 1.0 to 1.3 in the first month, each run against 280 historical tickets before merging.

After four weeks: 58% of drafts sent with no or minor edits, median time to first reply down from 11 minutes to 4, and one provider outage nobody outside engineering noticed, because the fallback was already there.

The two mistakes we see most

The bolted-on chat box. A chat window in the corner with no access to the record on screen, no tools and no measurement. Users try it twice and stop. The feature that works is embedded in the workflow: the draft in the ticket, the extracted fields in the form.

No fallback when the model is down. The AI feature and the ordinary feature it wraps fail together. If the "unavailable" path is not designed and tested, the model is a single point of failure for something that did not need one.

Checklist before the first release

• The endpoint returns schema-validated JSON and rejects anything else.
• Every request carries an idempotency key; repeats return the stored result.
• A timeout budget exists, and the fallback path has been tested.
• Default mode is suggestion; any autopilot action has a named owner and a measured accuracy.
• Evidence and a confidence category sit next to every output; every applied change can be undone in one step.
• Accept, edit and reject are stored with input, output, prompt version and model id.
• Tools go through MCP servers with narrow schemas and server-side permissions.
• Prompts are versioned in the repository; every change runs the evaluation set.

If you have a screen where AI should go, send us a screenshot of it and we will sketch the endpoint contract and the UI states with you.