What was compared
Classical computer vision, SAM2 and MobileSAM, an open-source segmentation model with Apache-2.0 weights. SAM2 was disqualified: about 115 seconds an image on a CPU, and it segmented roads as readily as roofs.
Roof detection CV
A roof outline from a photo, in seconds
Custom GPU-trained computer-vision models that detect roofs from aerial imagery and automate solar system design, trained across entire cities. Turns raw imagery into roof outlines, including internal edges and geometry. It runs inside the company’s in-house solar design and quoting tool, where it draws the roof when no other roof data exists for an address, and staff check and refine the outline it draws.
~4 sper roof outline on two vCPUs, down from 30–60 s
11/12real Australian scenes outlined correctly
22.7°median roof pitch, measured on 150 Australian roofs
The problem
Every solar design starts from a roof outline
No outline, no design. Panels are laid out on roof faces, and production, price and payback all follow from that layout. The design tool’s automatic mode relied on a third-party solar API for its roof faces, and that API has no coverage for many newer estates and regional addresses. For those, automatic design simply stopped.
The address is not the roof. A geocoded address often lands on the street or the kerb rather than the house, as I found on real Australian addresses. A detector that trusts the point outlines the road, so this one searches the neighbourhood around it for the dwelling.
The goal. Get a usable outline from the aerial photo alone, fast enough to sit in the middle of a quote, and honest about what a single photo cannot tell you.
Choosing the model
A bake-off on real Australian roofs
Three approaches went head to head on six real Australian addresses, each photographed by two imagery sources: twelve scene pairs. The bar was decision-level: the right dwelling found, with a tight outline, starting from the geocoded address point.
The recipe around the model
The model alone does not know which shape is a roof. The winning recipe probes in rings around the address point, scores each candidate for roof-likeness from its colour and shape, grows it facet by facet, refines the prompt to an interior point and traces the result to a clean polygon.
How it scored
Ten of twelve scene pairs gave the correct dwelling outline in the bake-off, including prompts geocoded onto the street. Re-run later on newly fetched imagery, the same detector reached eleven of twelve.
The failure it found
Both early misses were one scene: an address geocoded onto a commercial roundabout, where red-painted pavement looks exactly like terracotta tiles from above. So the tool always tells staff to verify the outline, and the outline stays editable.
How it runs
A small Rust sidecar, no Python on the hot path
The sidecar
- The whole detection pipeline is ported to Rust, with the model run through ONNX Runtime and statically linked into one small binary.
- About four seconds a detection, against 30–60 seconds for the Python reference it replaced.
- One JSON request in, one JSON answer out, no state kept between calls: the same convention as the battery optimiser sidecar in RenewCompare.
- The model weights are baked into the container image, with their licence checked against the upstream project, so a deploy depends on no outside model registry.
Inside the design tool
- The Python API supervises the sidecar. A crash or a malformed answer reads as “unavailable” rather than crashing the request.
- Automatic mode falls back to the detector when the third-party solar data has no coverage, and returns exactly the same shape of answer, tagged as computer vision, so it is priced like any other design.
- In manual mode, a “detect roof” button suggests the outline for staff to refine.
- The outline feeds the tool’s existing roof builder, which draws ridges and hips from it with a straight skeleton, for staff to adjust.
Checked, not assumed. A parity harness runs the Rust binary over every scene pair and compares its decisions with the Python reference, and the API’s tests cover the sidecar contract with a stand-in binary plus an end-to-end run of the real one. The test imagery is re-fetched when needed rather than committed to the repository, because of the imagery licences.
Training
Custom models, trained on GPUs, from whole cities of 3D roofs
The custom models are trained to find roof faces, the way each one faces and its pitch, from open, government-published 3D building models paired with 10 cm aerial photography, covering an entire German state in one-kilometre tiles.
Training data, rendered
A render job streams each tile pair, rasterises every roof plane with its compass direction and pitch worked out from the 3D geometry, cuts 512-pixel training crops and then deletes the source. Roughly 900 GB of imagery is never stored. Ten workers run in parallel and share the work by claiming tiles, since the platform gives a worker no index of its own.
Labels that know north
Every pixel carries one of sixteen compass directions, or “flat”, or “background”, plus a pitch and a flag for the boundary between two faces, which is how the model learns the internal edges. Flip or rotate a training image and the compass labels have to rotate with it, so the augmentation tables are derived from the angle maths rather than written by hand.
GPU training jobs
A SegFormer-B0 encoder with separate heads for roof segments and for superstructures such as dormers, windows and existing solar panels, warmed up on an open roof dataset. Training runs as a serverless A100 GPU job on Azure that starts, trains, saves its best checkpoint and its metrics, and stops.
A gate before it ships
A trained model replaces nothing until it beats the MobileSAM baseline on the same twelve Australian scenes, under the same decision rules. Every training source’s licence was checked by download, and non-commercial or unlicensed datasets were rejected outright.
It learns from the people who use it. Whenever staff save a design with corrected roof faces, the corrected outline and the details needed to fetch the same imagery again are kept as an append-only training label, behind a switch. Ordinary quoting builds an Australian fine-tuning set as a side effect.
What the measurements showed
Pitch is the hard part, so I measured it
A single overhead photo cannot measure a roof’s slope, so detection assumes a typical pitch. To find out how good that assumption is, I measured 150 roofs across five Australian capital cities by fitting planes to a third-party elevation dataset, used for evaluation only, inside my own outlines. The plane fits have residuals of 0.03–0.35 m.
Australian roofs, in numbers
Median pitch 22.7°, with the middle half between 18.4° and 25.4°: the trade’s rule of thumb of 22.5° holds up. 9.3% of roofs are flat, under 10°, and that minority is where all the error lives. Assuming 22° everywhere is out by 6.08° on average, and by 16.9° on flat roofs.
A rule from the trade
You cannot lay tiles on a flat roof. One threshold on colour saturation catches 79% of flat roofs at a 7% false alarm rate and cuts the error to 5.28° on held-out roofs, beating the constant and a learned pitch head, which sat near 8.4° on its own validation set.
The building code agrees
The National Construction Code sets a 15° floor for tiled roofs, and only 1% of roofs the rule calls tile fall below it. Steel sheet can go down to 1°, and metal roofs split into two populations, a median of 17.3° across a range of 2–33°. The remaining problem is one question: is this metal roof flat or pitched?
Four ideas that did not survive measurement
Reading pitch from tile-texture foreshortening, inferring each face’s direction from the outline’s straight skeleton (right on four of eight faces, a coin toss), snapping pitch to standard truss angles, and reading cladding profile from rib spacing. Each was measured and written down with why it failed, so none is built on by mistake.
The lesson. Everything that tried to infer roof geometry from a photograph failed or was a coin flip; what held up either measured it or came from how roofs are actually built. Before building on a geometric idea, I now run the cheapest measurement that would kill it. Elevation data also covered every metropolitan test site and none of the regional towns tested, which is why the photo alone has to work.
What comes next
Planned, in order
Pitch estimation
A pitch head trained on the true slopes in the 3D city models, aimed squarely at the flat-or-pitched metal roof question and always labelled as an estimate. It has to beat 5.28°, the bar the one-line rule set.
Obstructions and existing solar
Chimneys, dormers, skylights and existing panels as extra classes in the same model, so panels are never placed on an obstruction and a roof that already has solar is recognised.
Trees
Canopy mapping from open LiDAR height data and vegetation segmentation, feeding the design tool’s shading model with real canopy shapes in place of hand-placed tree circles.
Sunlight where the roof is
Per-location solar exposure from public Bureau of Meteorology and NASA datasets, modelled hour by hour, in place of the yield rule of thumb the tool uses today.
Computer vision • Rust • ONNX Runtime • MobileSAM • PyTorch • SegFormer • Python • FastAPI • GPU training • Azure Container Apps