Insights
How to evaluate an AI use case before you buy the tool
Toni Atunrase
How to evaluate an AI use case before you buy the tool
A simple pre-purchase scorecard for fit, data readiness, risk, and adoption so the technology serves the process instead of the other way around.
Most AI purchases fail quietly. Not with a dramatic breakdown — with a slow fade. The tool gets bought, onboarded, used enthusiastically for six weeks, and then it becomes the thing three people still use, one person avoids, and nobody remembers to cancel.
The failure rarely happens at the tool. It happens before the purchase, in the five minutes someone spent deciding it was a good idea. A confident demo and a reasonable price point are not evaluation. They're marketing working as intended.
I run every use case a client is considering through four checks before we talk about vendors at all. If it doesn't clear these, the tool doesn't matter — nothing you buy will fix a use case that wasn't ready.
1. Fit: is this actually the right kind of problem for AI?
Not everything that's annoying is an AI problem. Before evaluating any tool, get specific about what kind of problem you're solving:
- Pattern recognition across volume — good fit. Categorizing tickets, flagging anomalies, summarizing calls.
- Generating a first draft of something a human will finish — good fit. Emails, proposals, content, code.
- A one-time decision with high stakes and low repetition — poor fit. You don't need AI to decide whether to sign a five-year lease.
- A process that doesn't exist yet in any consistent form — poor fit, at least for now. AI needs something to learn or follow. If your team does the task five different ways, the tool won't create the consistency for you.
Write the use case as a single sentence: "AI will do X, so that a person doesn't have to do Y." If you can't complete that sentence cleanly, the fit isn't there yet, regardless of how good the tool looks in a demo.
2. Data readiness: does the tool actually have what it needs?
This is the check most businesses skip, and it's the one that kills adoption fastest. AI tools are only as good as what they can see. Before buying:
- Where does the data currently live? If the answer is "three systems and a spreadsheet someone maintains by hand," that's your real project — not the AI tool.
- Is it clean enough to trust? Duplicate records, inconsistent formatting, and missing fields don't get fixed by adding AI on top. They get amplified.
- Do you have enough historical volume for the tool to learn from, if it's a use case that depends on that? A tool trained on thirty examples behaves differently than one trained on thirty thousand.
- Who owns getting it connected? If the answer is "IT will figure it out," get a name and a date before you sign anything.
If the data isn't ready, that's not a reason to walk away from the use case — it's a reason to sequence the work. Clean the data, define the source of truth, then evaluate tools. Buying first just means you're now paying a monthly fee to remind yourself the data isn't ready.
3. Risk: what happens when it's wrong?
Every AI tool will be wrong sometimes. The question that matters isn't whether — it's what the cost of that looks like, and who catches it.
Score the use case on two things:
- Reversibility. Can a wrong output be caught and corrected before it reaches a client, a regulator, or a bank account? Drafting a follow-up email is low risk. Auto-approving a refund or auto-sending a legal document is not.
- Visibility. Will you actually know when it's wrong? Some failures are loud — a broken workflow, an obvious error. Others are quiet — a slightly-off answer that sounds confident and goes unnoticed for weeks. Quiet failure modes need a human checkpoint built in, not just good intentions.
High-risk, low-visibility use cases aren't disqualified — they just need a human in the loop before they go live, not after something goes wrong.
4. Adoption: will your team actually use it?
This is the check that gets skipped because it's the least technical, and it's the one most likely to sink the investment anyway. A tool with strong fit, clean data, and low risk can still fail for one reason: nobody changes how they work.
Before buying, be honest about:
- Does this remove a step, or add one? Tools that require your team to do their old process and feed the new tool don't get adopted — they get resented.
- Who's the champion? Not the person who chose the tool — the person who will actually answer questions and model using it in week three, when the novelty is gone.
- What does the team lose? Sometimes it's a task people liked. Sometimes it's a sense of control. Naming this upfront doesn't prevent resistance, but it does mean you're not surprised by it.
A scorecard doesn't need to be complicated to be useful. Rate each of the four areas low, medium, or high before you take a single vendor call. A use case that's high on fit and risk exposure but low on data readiness isn't a "no" — it's a "not yet, here's what needs to happen first."
The actual point of the scorecard
None of this is about slowing you down for the sake of caution. It's about making sure the tool serves a process you've already made sound, instead of asking the tool to compensate for a process you haven't. The businesses that get real value from AI aren't the ones who moved fastest to purchase — they're the ones who did this fifteen minutes of thinking first, and bought the right tool once instead of the wrong tool twice.
If you're evaluating a use case right now and can't confidently answer all four, that's useful information. It just means you're not ready to buy yet — you're ready to build the readiness that makes the purchase worth it.