AI vendor due diligence: 12 questions to ask before you sign

Short answer: before you sign with an AI vendor, get written answers to 12 questions: what result it will change, whether your data trains their models, where your data goes, which model sits underneath, how you’ll hear about incidents, whether you can test it on your own data, what stops wrong answers, what you’ll need to explain it, what it costs at scale, who owns the outputs, and how you leave. Then put the answers in the contract.

Most AI products are bought on the strength of a demo. The demo is real, but it answers the vendor’s question, which is whether the product can do something impressive. Your question is different: whether it’s safe to put your data into it, whether it will keep working, and whether it will pay for itself.

These are the questions I’d want answered before a mid-market business signs. If you want to compare vendors side by side, my free AI vendor due diligence scorecard turns them into a form and a report. This article is about why each question matters, and what a weak answer sounds like.

This is general information, not legal advice.

Why buying AI isn’t like buying other software

Most procurement checklists were written for ordinary software. AI adds four risks they weren’t built for:

• Your data can become their product. Some AI services use what you type in, and what they produce, to improve their models unless you opt out.
• There’s usually another company underneath. Many AI products are built on a model from a large provider, so your data may pass to a third party you never signed with.
• The product changes after you buy it. A model update can change how it behaves without a single line of your configuration changing.
• It can be confidently wrong. AI output that reaches a customer or a decision without a check is a risk your old software never carried.

Australian regulators expect you to deal with these before you buy. The OAIC’s guidance on privacy and commercially available AI products says organisations should do due diligence to make sure a product is suitable, including whether it has been tested and proven for the intended use, and warns against a “set and forget” approach. The National AI Centre’s Guidance for AI Adoption starts with deciding who is accountable, and that includes the AI you buy, not only the AI you build.

The 12 questions to ask an AI vendor before you sign

1. What business result will this change, and how will we measure it?

A vendor that can’t name the measure, such as hours per task, cost per transaction or error rate, is selling activity. Agree the baseline and the target before you sign, so the pilot has a pass mark. I’ve written about why usage figures make a poor measure in Tokenmaxxing.

Weak answer: “Customers see huge productivity gains.”

2. Who uses it in production, at a company like ours?

Pilots are easy. Ask for reference customers of a similar size and industry, ideally in Australia, who have run it in production for at least six months. Then call them, and ask what surprised them.

Weak answer: a logo slide and no names you can call.

3. Will our data, prompts or outputs be used to train or improve any model?

This is the first question I ask. The OAIC suggests checking the service terms to see whether the developer can access your inputs or outputs, including for further training. You want “no” by default, in writing, covering the vendor and any model provider it uses.

Weak answer: “We anonymise it first,” or a link to a privacy policy the vendor can change at any time.

4. Where is our data stored and processed, and who else receives it?

Ask for the regions and a current list of sub-processors, including the model provider. If personal information goes offshore, you remain responsible for how it’s handled under Australian Privacy Principle 8 (cross-border disclosure). The OAIC also suggests checking whether features that send data to third parties, such as web search, can be switched off.

Weak answer: “It’s all in the cloud.”

5. Which AI model is underneath, and will you tell us before it changes?

The Australian Government’s own AI contract template (Word, from the Digital Transformation Agency) requires the seller to use only an approved underlying model, to state its country of origin, ownership and where data sits, and to get the buyer’s written approval before a change that breaks those terms. A private business can ask for the same. At the least, you want notice of material model changes and a chance to re-test first.

Weak answer: “We use the best model available at any time.”

6. How fast will you tell us about a security or AI incident?

Under the Notifiable Data Breaches scheme, once you suspect an eligible data breach you must take all reasonable steps to assess it within 30 days, and the OAIC treats that as a maximum. You can’t meet it if the vendor tells you late. That’s not a remote risk: the OAIC has reported that most breaches affecting more than one organisation result from a breach of a cloud or software provider, and it recommends dealing with privacy risk in the contract. Ask for a notification period in hours, and current independent security assurance, such as an ISO/IEC 27001 certificate or a SOC 2 Type II report.

Weak answer: “We’d let you know as soon as practicable.”

7. Can we test it on our own data before we commit?

A demo on the vendor’s data proves the vendor’s data. Ask for a time-boxed pilot on a sample of your real cases, including the awkward ones, with success criteria agreed up front. The OAIC specifically recommends testing customer-facing tools, such as chatbots, before you deploy them.

Weak answer: a long contract with a pilot “included” in the first year.

8. What stops a wrong answer reaching a customer, and can we switch it off?

Every AI system gets things wrong. What matters is whether a wrong answer is caught: confidence thresholds, links back to the source, a person reviewing anything high-stakes. The government template goes further and requires a “circuit breaker” that lets the buyer stop the system immediately. In the AI translation program I led, people reviewed and refined the AI’s work rather than translating from scratch. The review step was the control.

Weak answer: “The model is very accurate.”

9. Will you give us what we need to explain it?

From 10 December 2026, organisations covered by the Privacy Act must say in their privacy policy when they use computer programs, with personal information, to make or substantially inform decisions that could significantly affect people. If the AI you’re buying shapes decisions about customers or staff, you’ll need the vendor to explain what it does, what data it uses and where it shouldn’t be used, in terms your board would accept. You’ll also need that for your AI register. My AI governance checklist covers what the register should hold.

Weak answer: “That’s proprietary.”

10. What will it cost at two to three times today’s usage?

AI pricing is often per user, per message or per token, and it rarely stays where the proposal put it. Ask the vendor to price your expected volume, then double and triple it. Ask for usage alerts or caps.

Weak answer: a price per seat with “fair use” limits nobody will define.

11. Who owns the outputs, prompts and configuration?

You should own your data, the outputs and anything configured or tuned for you, and the contract should say so. For generated content you’ll publish, ask whether the vendor gives an IP indemnity, and whether the liability cap reflects the real exposure.

Weak answer: ownership is “shared”, or isn’t mentioned.

12. How do we leave?

Ask how you’d export your data and configuration, in what format, and how the vendor confirms deletion at the end. Check for automatic renewals and minimum terms. A vendor that’s easy to leave usually doesn’t need to rely on making leaving hard.

Weak answer: “Customers don’t leave.”

What a good answer looks like

Question	The answer you want	Red flag
1. Business result	A named measure with a baseline and a target	Productivity claims with no measure
2. Proof in production	Australian reference customers you can call	Logos only
3. Training on your data	No by default, in the contract	An opt-out buried in settings
4. Data location	Named regions and sub-processors	“In the cloud”
5. Underlying model	Named, with notice before material changes	Swapped without notice
6. Incidents	Notice within hours, plus ISO 27001 or SOC 2	“As soon as practicable”
7. Testing	A pilot on your own data with agreed criteria	Demo data only
8. Wrong answers	Human review for high-stakes outputs, and an off switch	“It’s very accurate”
9. Explainability	Plain-English documentation of what it does and its limits	“Proprietary”
10. Cost at scale	Unit pricing modelled at two to three times today	Undefined fair-use limits
11. Ownership	You own data, outputs and configuration	Silence
12. Exit	Export, confirmed deletion, no auto-renewal	Long minimum terms

Questions 3, 4, 6, 8 and 11 are the ones I’d treat as deal-breakers: a “no” on any of them is a red flag, however well the vendor scores elsewhere. The scorecard marks them as critical for that reason.

How much due diligence does a mid-market business need?

Scale it to the risk. An AI writing assistant that never sees customer data needs written answers to questions 3, 4 and 10 and a short trial. A tool that answers customers, screens job applicants or decides who gets a discount needs all 12, a pilot, a privacy impact assessment and a lawyer’s eye on the contract.

Three habits make it work, whatever the size of the purchase:

1. Get every answer in writing. A sales call is not a commitment. Send the questions as a document and ask for written replies.
2. Put the answers in the contract. If the vendor’s standard terms contradict what you were told, the terms win. The government template is a useful model of what to ask for, even for a private business.
3. Ask again when something changes. An existing supplier switching on an AI feature, or a vendor changing its model provider, is a new purchase for these purposes.

The bottom line

The demo shows you what an AI product can do. These 12 questions tell you whether it’s safe to buy, whether it will keep working and whether it will pay for itself. Ask them in writing, put the answers in the contract, and walk away from a vendor that won’t commit on your data, your exit or a way to catch its mistakes.

If you’re comparing vendors now, use the free AI vendor due diligence scorecard to score them side by side. I’m platform-neutral, and helping leadership teams choose and govern AI is part of my work as a fractional Chief AI Officer. If you’d like an independent view before you sign, book a 30-minute call.