Three quarters of companies now buy AI instead of building it. Most are running that purchase through a process designed for software licenses, and the thing you evaluate in the demo is not the thing you will operate next year.

Ask a vendor for a completed model card before the first technical demo. Not after the pilot, not during security review. Before the demo.

The ones who send it back inside two days have a mature engineering culture and documentation they actually maintain. The ones who ask what a model card is, or promise it later, or send a marketing one-pager with the word “responsible” in the headline, have told you something useful about what the next eighteen months will look like.

That one request separates more vendors than a forty-question scorecard, and it costs nothing.

The buy side got bigger, fast

In 2026, 76% of enterprises are purchasing AI solutions rather than building them internally, up from 53% in 2024. That shift happened in about two years, and procurement did not move with it. Most organizations are evaluating AI systems using a framework built for perpetual software licenses and annual SaaS renewals.

We build custom software for a living, so understand the position we are taking here. Buying is frequently the right call. A mid-market company with three developers and a roadmap should not build a document classification engine from scratch. The problem is not the decision to buy. The problem is buying it the way you bought your CRM.

Here is the structural difference, and it is the whole article in one sentence. Traditional procurement evaluates performance at the moment of purchase. AI systems change after purchase, because models get retrained, capabilities get extended, and vendor terms get revised.

You are not acquiring a fixed artifact. You are entering a dependency on something that will be different in six months, controlled by someone else.

The spend is up and the value is not

Deloitte’s 2026 survey of 3,235 enterprise leaders found 84% planning to increase AI spending. In the same population, only 25% said generative AI was transforming their business, and 37% reported running AI with little change to their underlying processes.

Read those three numbers as one finding. Money is moving faster than method. A large share of that spend is going into tools bolted onto processes nobody redesigned, purchased through a diligence process that asked about uptime and pricing tiers and never asked who controls the training data.

Four questions standard diligence never asks

These sit outside the scope of normal commercial review, which is exactly why they get missed.

Who controls the training data, and what happens to ours? Specifically, does the vendor retain your prompts and outputs, and are they used to retrain a model other customers touch? This is the single most consequential question in AI procurement and it rarely appears on a standard security questionnaire.

What happens when the supplier gets acquired? AI vendors are consolidating. The terms you negotiated, the data handling commitments, and the roadmap all belong to whoever owns the company, and your contract should say what survives a change of control.

How does liability work when the system produces a harmful output? Not a security breach. A confidently wrong answer that someone acted on. Most contracts are silent here, and silence favors the vendor.

What is the deprecation policy for the model underneath? Vendors retire models. Anyone who has built on a foundation model already knows this. You need notice periods, a migration path, and a written answer about what happens to behavior you tuned against a version that no longer exists.

Red flags are overrides, not inputs

Scoring models fail in a specific way. A team builds a weighted scorecard, a vendor fails badly on one category and scores well on eight others, and the average carries them through. The finding gets logged and forgotten.

Treat certain findings as disqualifying rather than as numbers in a mean. A vendor that stores your prompts to retrain a shared model cannot be redeemed by excellent documentation. That finding has exactly three valid outcomes: resolved in the contract, accepted as a written and named risk, or the vendor drops out.

There is a fair counter-argument worth respecting. Scorecards become theater. Procurement collects answers nobody reads, and the technical team picks the vendor it already wanted in week one. The fix is not abandoning the scorecard. The fix is giving it teeth: one named owner per category, required evidence for every claim rather than a self-attestation, and a written escalation path for when a red flag fires.

Design the pilot against your worst data

Most AI pilots are run against a clean slice, and that is why most AI pilots pass and most AI deployments disappoint.

The demo dataset is tidy because the vendor built it, or because your team pulled a well maintained subset to get a fast result. Production is not tidy. Production has duplicate customer records, fields that went stale in 2023, partial inputs where someone skipped a required box, scanned documents at three different quality levels, and the one legacy naming convention that predates your current system.

So build the evaluation set from the mess on purpose. Pull records you know are duplicated. Include the documents your own team complains about. Add a handful of edge cases that a human would need to ask a question about, and see what the system does when the correct answer is “I do not know.”

Two things come out of this. You learn the real accuracy rate rather than the demo accuracy rate, and you learn how the system fails. A tool that declines to answer when it lacks context is operationally different from one that produces a confident guess, and no scorecard category captures that difference. You have to watch it happen.

Insist on running the pilot against your data, not a sanitized extract, and insist on seeing the failures rather than a summary accuracy figure.

What this looks like in Wisconsin

The state’s employment base shifted this year in a way that matters for who is buying. Forward Analytics reported in July 2026 that health care and social assistance overtook manufacturing as Wisconsin’s largest employer, by 4,933 jobs, making Wisconsin the 47th state to complete that transition. Manufacturing remains substantial, with nearly 460,000 workers, average pay of $75,115, and roughly 15.6% of state GDP.

Both of those sectors are buying AI right now. Both run procurement functions built for capital equipment and software licenses. A health system evaluating an AI scribe and a Manitowoc manufacturer evaluating a vision inspection system are facing the same gap: a purchasing process that grades a demo, applied to a product that changes after the contract is signed.

For the many Wisconsin companies without a deep internal engineering bench, buying is not just reasonable, it is the default. Which makes the quality of the buying process the main lever they have.

The unglamorous framework

The pattern across companies that get real returns is not sophisticated. It is disciplined.

Fund it from the operating budget rather than an innovation line, because operating budgets come with owners and scrutiny. Assign one named person who is accountable for the outcome, not a committee. Define the commercial result before signing, in a sentence a CFO would recognize. And attach a proof clock: a date by which the thing either demonstrates the result or gets turned off.

Then treat the vendor relationship as ongoing rather than settled. Review AI vendors on a cadence tied to their risk tier, not only at renewal, because the system you approved is being modified continuously by someone who does not report to you.

The honest tradeoff

Ninety days of real diligence is fast by procurement standards and painfully slow to a project sponsor who wanted to go live last quarter. That tension is genuine and we are not going to pretend otherwise.

Our position: the model card request and the four questions above take under two weeks and catch most of what matters. Run those before the demo. Spend the remaining diligence budget on the one or two vendors that survive, rather than spreading ninety days across five.

Sometimes the answer that comes out of this process is that nobody should be paid yet, because the workflow is not ready for the tool. That is a good outcome. It is cheaper than the alternative, and we would rather tell you that than sell you a build.

Setting a 2027 AI budget right now and want a second set of eyes on the vendors in front of you? That is a conversation we have most weeks.