--- title: "Your AI vendor keeps the part you actually built" url: "https://xubris.com/blog/your-ai-vendor-keeps-the-part-you-actually-built" description: "Instructions, test sets, corrections and logs are where an AI system's value builds up. They are also the parts most contracts forget to let you take with you." --- # Your AI vendor keeps the part you actually built October 8, 2026·4 min read·Editorial team A stack of black magneto-optical discs sealed in a smoked acrylic box beside a dark monitor, a magenta light on the clasp A support team spends a year teaching an AI assistant how their business works. They write and rewrite its instructions. They collect a few hundred real questions with the answers they want, and use them to check every change. Agents correct the assistant’s drafts every day, and those corrections shape how it behaves. Then the platform changes its pricing, or is acquired, or retires the model the team tuned everything against. The team goes to export and finds the conversation transcripts and little else. The instructions live in a configuration screen. The test set lives in the vendor’s evaluation dashboard. The corrections were used to improve “the service”, in a form nobody can download. The model was never theirs. A year of work turns out to have been a deposit in someone else’s product. ## The real cause With conventional software, the valuable thing a customer creates is data, and most buyers now know to ask for it. With AI systems the valuable things are different and less obvious: the instructions that make a general model useful to one organisation, the examples that define what good looks like, and the record of what went wrong and how it was fixed. They are expensive to produce, specific to the customer, and invisible to a standard data-export clause because they do not look like records. They are also what a vendor most wants to keep, because they are what makes switching painful. A competitor can match the model. It cannot match a year of your corrections. ## Where the value gets stuck **Instructions live in the interface.** System prompts, tool definitions and routing rules are edited in a web screen with no version history and no export. Rebuilding them means reconstructing intent from memory. **The evaluation set belongs to the tool.** The questions and expected answers used to judge quality sit in the vendor’s dashboard. Without them you cannot even tell whether a replacement is as good. **Feedback is pooled.** Ratings, corrections and edits tune a shared system. They improve the product for everyone and leave nothing that belongs to you. **Logs are kept on the vendor’s terms.** Transcripts may be retained for a fixed period, sampled, or available only on request. That is also the history you would need to show an auditor what the system actually did. **Behaviour is tied to one model version.** Everything was tuned against a model the vendor can retire. Even with every file in hand, moving means re-testing against something new — which is only possible if you kept the tests. ## What to do instead **Keep the instructions in your own repository.** Treat prompts and tool definitions like code: versioned, reviewed, and deployed to the vendor rather than authored inside it. **Own the evaluation set.** A plain file of inputs, expected behaviour and scoring rules, held by you. It is the most portable asset an AI system has, because it lets you compare any two providers on your own terms. **Ask where feedback goes, in writing.** Whether corrections train a shared model, whether you can export them, and in what format. If they cannot be exported, capture them yourself before they go in. **Set log retention yourself.** Send transcripts and decisions to storage you control, kept for as long as you decide. EU rules on artificial intelligence increasingly expect systems used for consequential decisions to leave a traceable record, so in some cases keeping logs stops being a matter of preference. What applies depends on the sector and the use; it is worth checking with someone who knows your sector. **Prefer systems that can run on another model.** An architecture that can swap the underlying model, including an open-weight one you could host yourself, keeps the vendor honest even if you never switch. This is the argument we make for any system — that [customers should be able to leave](https://xubris.com/blog/we-only-back-companies-you-could-leave) — applied where it is currently least respected. The AI products we would want to build, or back, keep a customer’s instructions, tests and history in formats the customer can take with them: in the words of [the homepage](https://xubris.com/), their systems, their data and their way out. ## A test you can run today Take the AI tool your team relies on most and try to rebuild it somewhere else, on paper. List: 1. Every instruction and rule it runs on, and where each one is stored. 2. The examples you would use to check that a replacement works. 3. The corrections your team made last month, and whether you have a copy. 4. How long transcripts are kept, and who decides. Anything you cannot find, or can only see inside the vendor’s screens, is the part of the system you are renting rather than owning, and the list is a fair agenda for your next conversation with the vendor. Whether the parts you can find would actually load anywhere else is the second half of the test, described in [an export you have never restored is not an exit](https://xubris.com/blog/an-export-you-have-never-restored-is-not-an-exit). - AI - data portability - evaluation - lock-in ## Keep reading - [An export you have never restored is not an exit](https://xubris.com/blog/an-export-you-have-never-restored-is-not-an-exit) - [Build the exit before your first customer asks for it](https://xubris.com/blog/build-the-exit-before-your-first-customer-asks)