Most write-ups on a consultancy’s site are wins. This one is about a project we shut down, because how it was shut down says more about how we work than any success story would.
The idea
The project was called Sundial: a lead-generation tool for shade and pergola contractors. Draw a boundary around a neighbourhood and get back a list of homes with uncovered, sun-baked patios, each one scored by how many hours of direct June sun that patio actually gets, computed from Google’s solar data.
The feature the whole pitch hung on was the mailer. Not a generic flyer: a photorealistic mockup of the homeowner’s own house with a pergola on it. That image is what would make a contractor’s mail piece impossible to ignore.
Test the thing that can kill it first
Before building any pipeline, we spent a few days on that one feature, because if the mockup did not work, nothing else mattered.
It did not work. The Google API that could have produced the imagery turned out to cover the US only, and the fallback approach produced convincing renders only on cherry-picked houses on streets with little tree cover. On a typical leafy Ontario street, it fell apart.
Notice what did not happen: weeks of engineering before finding this out. The riskiest assumption got tested first, while the total spend was still a rounding error.
The pivot, and the honest measurement
We pivoted to a plainer product: the lead list without the picture. A working version came together quickly. One command turned a neighbourhood boundary into a ranked list: open address data filtered to the area, a satellite image of each property classified by an inexpensive AI model at roughly half a cent per home, and the costlier solar analysis run only on the addresses that passed the filter. Every run reported its own cost and hit rate.
Then came the step most AI projects skip: checking whether the classifier was actually right. We hand-labelled a set of real properties and scored the system against them. The results were mediocre. Precision on hard cases was around 72 percent, a stronger and more expensive model did no better, and the marker for where the patio sits landed on the actual patio only about a third of the time. That last one matters, because the sunlight measurement is taken at that marker.
The call
All of it was fixable with more tuning rounds. But the honest ceiling of the product, without the mockup that made the pitch special, was a mailing list somewhat better than “every detached home in a nice postal code,” sold to small contractors, a tough and budget-conscious buyer. The numbers said the effort would cost more than the product could ever be worth. So we shelved it.
Not everything was lost, and that is part of the point. The reusable machinery survived: the address ingestion, the cheap wide-net image classification, the pattern of spending on expensive APIs only for pre-qualified hits, and the evaluation harness that produced the numbers above. Those pieces will show up again in other projects. Only the patio business died.
Why this matters for your project
This is the exact process we bring to client work, and it is worth spelling out what it buys you.
- The riskiest assumption gets tested first, while it is cheap, not after months of build.
- The AI’s accuracy gets measured against labelled reality, not vibes from a demo.
- Go or no-go decisions come from numbers, and no-go is a real option.
The most expensive AI project is not the one that fails fast. It is the one that limps along for a year because nobody ever checked whether it worked. If someone building their own product is willing to kill it in week two, you can trust them to tell you the truth about yours.