Every AI feature built on a third-party model API starts the same way: pick a provider, call the endpoint, ship it. That decision is easy to make and expensive to unmake. Eighteen months later the pricing changes, a better model appears elsewhere, or the provider deprecates the exact endpoint your product depends on, and the team discovers that “switching providers” actually means rewriting half the AI layer.
This isn’t an argument against using third-party model APIs. For most teams, calling a hosted model is still the right call: nobody should be running their own GPU cluster to summarise support tickets. It’s an argument for building the surrounding code so that a provider swap is a configuration change and a re-run of your evaluation suite, not a rewrite. We’ve made these decisions across several of our own AI products, and the pattern that causes the most pain is always the same: coupling that nobody noticed until they needed to remove it.
Where the coupling actually happens
Lock-in with AI APIs rarely looks like a contract clause. It builds up in the code, in four places:
- SDK coupling. Provider client libraries get imported directly into business logic, so their request and response shapes, retry behaviour and error types leak into every function that touches the model.
- Behavioural coupling. Prompts get tuned, sometimes for months, against one model’s specific quirks: how it handles system instructions, how literally it follows formatting requests, how it behaves near its context limit. Move that prompt to a different model and the output quality changes even though the wording didn’t.
- Data coupling. Embeddings generated by one provider’s embedding model aren’t compatible with another’s. If your vector store is full of one vendor’s vectors, changing providers means re-embedding your entire corpus, not just swapping a config value.
- Commercial coupling. Enterprise agreements, committed spend, or fine-tuned models hosted only on one platform, structural reasons unrelated to code that make leaving expensive regardless of how clean the architecture is.
The first two are fully within an engineering team’s control. The third is manageable with the right storage choices. The fourth is a procurement decision, not a technical one, and worth having explicitly rather than backing into.
Design decisions that keep the door open
Put a thin boundary between your product and the provider
The standard fix is an adapter: one internal interface for “generate a completion” or “embed this text”, with a provider-specific implementation behind it. This is old advice for a reason, it works, but it’s easy to get wrong in AI code specifically. Providers don’t just differ in request format, they differ in what they’re good at: function calling reliability, streaming behaviour, structured output support, context window size. An adapter that flattens all of that down to the lowest common denominator produces a worse product than not abstracting at all.
The version that works keeps the interface narrow and honest about capability:
interface CompletionProvider {
complete(prompt: PromptSpec, opts: CompletionOptions): Promise<CompletionResult>
capabilities(): { structuredOutput: boolean; maxContextTokens: number; streaming: boolean }
}Callers check capabilities() rather than assuming every provider behind the interface can do everything the current one can. That one method is what stops the abstraction from quietly becoming a lie.
Version prompts like code, and test them against more than one model
Prompts that live as string literals scattered through the codebase are the single biggest source of behavioural coupling. Keeping them as versioned, named assets with their own test cases does two things: it makes prompt changes reviewable, and it gives you a fast way to find out how much a provider swap actually changes output quality, because you can run the same suite against the new model before committing to anything. Teams that skip this step find out their prompts were overfitted to one model only after the switch has already shipped and users start noticing.
Treat embeddings and fine-tuned models as data, not configuration
If a product relies on semantic search or retrieval, decide up front whether re-embedding the entire corpus on a provider change is an acceptable cost. For a small document set it’s a non-issue. For a large one, it’s worth storing enough of the source text and metadata to regenerate embeddings from scratch rather than only storing the vectors, and worth knowing roughly what a full re-embed would cost in time and API spend before you need the answer under pressure.
When self-hosting an open model is the right call
Self-hosting isn’t automatically the answer to lock-in, it trades one set of constraints for another: you now own GPU provisioning, model updates, and the operational load of keeping an inference service available. It tends to make sense in a narrow set of situations: sustained, predictable volume high enough that per-token API costs exceed infrastructure costs; strict data residency or confidentiality requirements that rule out sending data to a third party at all; or a task narrow enough that a smaller open model performs as well as a general-purpose one, at a fraction of the cost.
Outside those cases, hosted APIs are usually still cheaper once engineering and operational time is counted honestly. The mistake is deciding this once and never revisiting it. Usage patterns change, and a decision that was correct at launch can be wrong two years later purely on cost.
What this costs you
None of this is free. An adapter layer, a prompt versioning system, and a multi-model evaluation suite are extra code to write and maintain, and they add a small amount of friction to shipping the first version of a feature. If a product genuinely depends on a capability only one provider offers, real-time voice with specific latency characteristics, for instance, building an abstraction on the promise of a future switch that will probably never happen is wasted effort. The judgement call is about how likely a switch is and how painful it would be if forced, not building portability as a rule for its own sake.
The question worth asking before the first API call isn’t “which provider is best today”, it’s “if this provider changes its pricing or deprecates this model in a year, what does our exit look like”.
A checklist before you commit to a provider
- Is every provider call routed through a narrow interface, or are SDK types and request shapes used directly in business logic?
- Are prompts stored, versioned and testable independently of the code that calls the model?
- If embeddings are involved, can the corpus be re-embedded from stored source data, or only from the vectors you currently have?
- Do you know, roughly, what re-running your evaluation suite against a second provider would cost and how long it would take?
- Is any part of the commercial relationship, committed spend, an enterprise agreement, a fine-tuned model hosted only on their platform, creating lock-in that no amount of clean code will remove?
- Have you actually revisited the build-vs-buy-vs-self-host decision recently, or is it running on the assumption made at launch?
If most of those have clear answers, the architecture is sound regardless of which provider sits behind it today. If they don’t, that’s the gap worth closing before the next model release makes the decision for you.