Case study · Edge ML

Video analytics on ARM: what breaks when you leave the demo

A TensorFlow Lite and OpenCV pipeline on cheap ARM boards guarding a residential settlement. The model was never the constraint. Frame-time spent outside inference was, and then the physical world was.

Client: A suburban residential cooperative in Russia
Domain: Edge ML
My role: Whole pipeline, from board to alert
Period: 2025 – present
Result: 3–5 fps to a sustained 15–20 on the same hardware

Context

Perimeter and street cameras for a suburban residential cooperative: several dozen Raspberry Pi 4 units on poles and gateposts, each classifying what moves past it as a person or a vehicle. No faces, no plates. The system answers a narrower question: is anyone there, and did they arrive on foot or by car.

Inference had to happen on the device, for four reasons that all pointed the same way. The settlement shares one consumer uplink, which will not carry a few dozen video streams. Central GPU capacity for the whole fleet costs more than the fleet. Continuous video of residents leaving the property is a liability nobody wanted to own. And the link drops, often, so a device that stops detecting when the internet goes down is worth nothing at the exact moment it matters.

The hardware was chosen for unit price, not for compute.

The problem

The model worked. It worked on a workstation, on clean footage, at a frame rate nobody had questioned. On the Pi, in a sealed enclosure, on a pole, the pipeline delivered 3 to 5 frames per second against a requirement of 15 to 25. That is not a tuning gap. A factor of five means the shape of the pipeline is wrong, and no amount of shaving milliseconds off inference will close it.

Constraints

The hardware was fixed and cheap. A Raspberry Pi 4 with 4 GB of RAM, one per camera, several dozen of them. The budget came from residents’ dues, so “buy a Jetson for every gate” was never going to be funded, and I knew that before writing a line of code.

Four CPU cores, and the model is not the only tenant. With 4 GB the memory ceiling turned out to be comfortable, which surprised me. The scarce resource was CPU time, shared between video decode, colour conversion, resize, inference and the OS. Everything below follows from that one fact.

Thermal throttling was the cheap problem. It is the first thing everyone warns you about on a Pi in a sealed box, and a passive heatsink closed it for about three dollars a unit. Worth saying plainly, because the expected constraint absorbed a lot of attention early and turned out to cost the least to fix.

Nobody visits the devices. There is no on-site engineer at a residential cooperative. Updates ship as containers over the air with a rollback to the previous image, because a bad push that requires driving out to thirty poles costs more than the entire fleet.

Real footage is nothing like the training set. Two things actually hurt. Night, where headlights blow out half the frame and a person walking behind those headlights simply is not there for the model. And slow lens contamination: dirt, dust, and spiders, which build webs directly across a warm housing and are patient about it.

What I did

Measured on the target device from the first week. Every performance number that matters comes from the real board, in an enclosure, at temperature, on real footage. Desktop numbers are not a proxy for anything.

Quantised and profiled, in that order, repeatedly. SSD MobileNet in int8 ran two to three times faster than the float model, and the accuracy loss was small enough that it never showed up in field complaints. Then the profile said something more useful: a large share of frame time was never inference. It went on decode, colour conversion and resizing in the OpenCV pre-processing path. Speeding up the model further would have bought very little.

Cut work before optimising work. Two changes did most of the job. Motion gating means an empty frame costs a cheap difference check instead of a full inference, and at a residential gate the frame is empty for most of the day and almost all of the night. A region of interest removes sky, trees and a neighbour’s plot from the picture before anything else touches it, which shrinks both the pre-processing cost and the number of things the model can be wrong about. The cheapest inference is the one you never run.

Designed for degradation rather than failure. When the box is momentarily too slow, the pipeline drops frames and stays current. The alternative is a queue that grows, eats memory, and eventually reports an intruder who left twenty minutes ago.

Made the fleet observable. Every unit reports sustained frame rate, SoC temperature, free memory and uptime. That is a short list on purpose: it is cheap over a weak uplink, and it is enough to tell a device that is dying from a device that is merely busy. Without per-device telemetry a fleet of remote units is a black box, and you learn about problems from the customer.

What went wrong

Night buried the system in false alarms. The performance work landed, the frame rate held, and a few weeks later the thing was effectively dead anyway. Hundreds of alerts a night, nearly all of them wrong, so residents stopped reading them. Headlights sweeping across a gate produce moving bright shapes that a detector trained on daytime footage is happy to call a person. The system was technically running and practically ignored, which is a worse state than being down: nobody files a ticket for it.

The fix was retraining on night footage collected from those same cameras, in those same positions. Public datasets did not help. What mattered was this gate, this lamp, this angle of approach, and cars whose headlights hit the lens at the exact height where the fence stops.

Spiders. They build webs across a warm housing overnight, and a web in front of the lens degrades detection slowly enough that no single day looks broken. This is an argument for image quality telemetry rather than for a smarter model, and it is the sort of failure no amount of validation-set accuracy will warn you about.

The over-the-air update path, for its part, never bit us. That is worth stating because it was the risk I spent the most design effort on, and the money would have been better spent on night data.

Result

15 to 20 frames per second sustained on a Raspberry Pi 4, on the same hardware that delivered 3 to 5 at the start.
Several dozen devices in production since 2025, running unattended.
Inference stays on the device, so the settlement’s single uplink carries events and thumbnails instead of video, and no resident’s footage leaves the property.
Hundreds of false alarms a night, down to a handful, after retraining on footage from these cameras at night. That is the number I care about most, because it decides whether anyone still opens the notifications.

The engineering that made this work was almost entirely outside the model. The model was a fixed cost; everything that made the system viable was about not paying that cost more often than necessary.

Stack: TensorFlow Lite · OpenCV · Python · ARM · Linux