AI proof of concept: what it should prove before you fund the build.

01Why most proofs of concept prove the wrong thing

A typical AI proof of concept takes a clean sample, a carefully written prompt and a week of work. It ends with a demo that answers well in a meeting. Everyone is pleased, a budget is approved, and three months later the project is “almost ready”.

What went wrong is that the PoC proved the model could do the task on good data. It did not prove the system could do the task in your business: with your real records, inside your tools, at a cost that makes sense, for people who have their own way of working. Those are different questions, and the second one is the one you are paying to answer.

02What an AI proof of concept should prove

1. Quality on real data. Run it on a few hundred real examples, messy ones included, with known right answers. Report accuracy as a number, and look hard at the failures.
2. The integrations work. Read from and write to the real systems, even if only in a test environment. Most of the risk lives here.
3. The unit economics. Cost per task, compared with what it costs today. If the AI is accurate but costs more than the person, you have a research result, not a business case.
4. People will use it. Put it in front of two or three of the people who will actually use it, inside their workflow. Watch what they do, not what they say.
5. Failure is safe. Show what happens when it is wrong: who notices, how it is undone, and when it hands over to a person.

03An AI readiness checklist

Before a PoC, answer these. If several are “not sure”, that is your first piece of work, not a reason to stop.

• Outcome: Can you name one measurable result, such as time to first reply, documents per hour or cost per ticket?
• Data: Can you export a few hundred real, representative examples? Do you know how messy they are?
• Systems: Do the tools involved have APIs or exports? Who controls access?
• Process: Could someone write down how the task is done well today?
• Owner: Is one person accountable for the result, with authority to make decisions?
• Risk: Which mistakes are acceptable, and which must never happen without a human check?

04A two-week PoC plan

Week one: test it on reality

1. Agree the outcome and how it will be measured.
2. Pull real samples and look at their quality: gaps, duplicates, free text used as data.
3. Map every system the AI must read from or act in, and how we connect to each.
4. Build a small working prototype on that data, in the real workflow where possible.

Week two: turn it into a decision

1. Build an evaluation set from real cases and measure the prototype against it.
2. Estimate cost per task at your real volumes.
3. Draft the architecture: models, data flow, guardrails, hand-off, monitoring.
4. Write up risks, a dated plan for the first release, and a fixed price for it.

This is exactly how our discovery sprint runs. It is a fixed fee, starting from $1,500, and the written spec is yours to take to any team.

05Go, change or stop

A PoC should end with one of three answers, said plainly.

• Go: quality, cost and adoption all clear the bar. Build it.
• Change: it works, but only with a narrower scope, better data or a different model. Adjust and re-test.
• Stop: the data can't support it, a simple rule does the job, or it will never beat the current cost. A clear no in two weeks is far cheaper than a pilot that drifts for six months.

06From proof of concept to production

Passing the PoC is the start. The work that makes AI dependable is the part demos leave out: an evaluation set run on every change, guardrails and approvals, a log of every action, cost controls, monitoring of quality and uptime, and integration where the work actually happens. Plan for it from the first day, and the first release typically lands 2 to 4 weeks after kickoff.

The model is the engine. The other 80% is the car around it.

07When not to build it in-house

Build in-house if you have engineers with time, experience shipping AI to production, and someone to own it after launch. Bring in a partner when the team is busy with the core product, when the work needs skills you won't need again soon, or when you need it live in weeks rather than quarters. Either way, insist on owning the code, the data and the evaluation set. That is what keeps you free to change partners or models later.

08Questions people ask