The AI services market right now looks like the dot-com bubble: everyone slaps "AI-powered" on the homepage, and buyers are picking firms based on demos instead of results. The failure rate that follows is not bad luck — it's selection. So before you sign, here are the red flags that predict a burned budget, and the questions that flush them out.
The red flags
They lead with the demo, not the failure modes. A polished demo proves the happy path works on three inputs the vendor chose. The entire job is everything else — the adversarial input, the load, the long weird tail. A firm that only shows you the demo is hiding the part that matters. (If it works that well, ask why they're selling it instead of running it.)
They won't show you an eval. Ask: "How will we know it's good?" If the answer is "you'll see in the demo" rather than a dataset of real questions, real answers, and explicit thresholds, there is no objective definition of done — and a build with no eval is the one that dies in week two.
They hide their pricing. Opaque pricing isn't sophistication; it's a qualification game. A firm confident in its value will tell you the range before the call. If you have to sit through a pitch to learn the number, the pitch is the product.
Bait-and-switch staffing. The senior people who pitch are not the people who'll write the code; the work gets handed to juniors learning on your budget. Ask: "Will the people in this room be the ones building it?"
They can't explain a past failure. Ask them to walk your technical team through a project that went wrong and what they did. A real firm has scar tissue and will show it. A firm that has only ever succeeded has either never shipped to production or isn't telling you the truth.
They won't give you the keys. If IP ownership, code handover, and the model/API keys aren't yours from day one, you're not buying a system — you're renting a dependency. Read the data-usage terms before you sign; broad, vendor-favorable rights are the default in more contracts than you'd expect.
They promise a guaranteed outcome. Anything stated with the confidence of a law of physics — "guaranteed X% accuracy / leads / savings" — is marketing, not engineering. Real systems have failure modes and the honest firm names them.
The questions that flush them out
Ask these four, and the answers will sort the field fast:
- "Show me the eval that decides whether a change ships."
- "Open a real failed request and trace what happened."
- "Who operates this after handover when it breaks on a holiday weekend, and with what runbook?"
- "What's the price range, and do I own the IP and keys?"
A firm that ships production systems answers all four without flinching. A demo shop pivots to the roadmap on the first one.
What good looks like
The inverse of every red flag: leads with the failure modes, shows the eval, publishes the price, staffs the same senior people from scope to handover, has scar tissue it'll share, and hands you the keys. That's not a long list — it's just rare, because most of the market is optimized to win the pitch, not to survive the first operating week.
You're right to be skeptical — most AI projects fail, and most failures trace back to the vendor selection, not the model. Use the four questions. The firm that answers them plainly is the one worth a second call.
If you want to ask us those four, bring the problem, the owner, the budget, and the date — that's the call we like.