OpenCV and the Wild Kingdom
Deploying a thermal camera and homebrew motion detection to catch a thief or three.
The code for this post can be found here.
If you don’t care about computer vision and just want to see animals stealing food jump to An Ongoing Mystery.
I purchased a FLIR Boson on Ebay earlier in the year and after pointing it at everything in and around my house I decided to do something useful with it. The useful thing I landed on was pointing it at the sky and looking for cool things, as one does. There’s no shortage of planes, birds, and bugs that can be seen in LWIR that might go unnoticed in the visible band, but staring at a computer waiting to see something isn’t how I like to spend my time. To that end I started working on a simple motion detection tool that would do a few things
- Trigger when the scene changed
- Record only frames with motion in them
- Timestamp the imagery
- Optionally also record a live feed
OpenCV would let me do all of this so I started writing a prototype in Python and quickly got distracted playing with color maps.
LOL Python Slow
I found that recording the raw frames would give me videos that played back at odd multiples of the actual time I was recording, and after too many calls to ffmpeg to get the playback corrected I figured out that Python was to blame. Despite being a low-effort meme, Python actually is slow and my script was dropping frames. This manifests as odd playback speeds because the encoder expects N frames every second when in reality the script is supplying < N and the video plays back faster than real time.
Detecting Motion
There is no end to the off-the-shelf solutions to object tracking and motion detection these days, and the problem isn’t exactly new, either. Motion is a great example of an “old school” (old enough to drive!) solution meaning it doesn’t use a neural network à la Yolo. However, simple motion detection is a fun problem that can be solved in a day or two and wasn’t something I had written from scratch before. Motion detection really just means quantifying “did something in the scene change?” and describing that in terms a computer can understand. In order to achieve this I store an array of frames, average them, and then subtract the current fraame from that average. Any changes show up as non-zero pixels in the resulting image, and if we see any “hot” pixels then we know something is different and can decide what to do from there.
The left side is the raw video, and right is the subtracted image. My hand is clearly visible on the right hand side because the background has been subtracted, leaving only my hand.
All Roads Lead to C++
With the basic motion detection working it was time to port the Python prototype to C++. I won’t bore you with the details; no one wants a line-by-line breakdown. However, there are a few improvements that were added as I did more testing:
- The Boson applies what’s called a Flat Field Correction at a configurable rate. When it does this it sticks a bright green square in the top right corner of the frame. I ignore any movement in that area of the image to stop false-positives.
- Images are noisy and the subtracted image might have 20 hot pixels that belong to two objects. I call a simple dilate function to expand those pixels and hopefully make continuous blobs.
- Only 3 things are tracked at once
- A bounding box of fixed size is placed around the center of the blobs that are detected.
An Ongoing Mystery
We have an outdoor cat named Rory that “lives” in the carport, and by that I mean she has a closet in the carport with her food and a bed that I don’t think has ever been used. She spends her days relaxing on the front porch in her favorite cardboard box or curled up in the sun in the front yard. The closet has a cat door and inside is a dry food feeder. The feeder would mysteriously empty itself over just a couple days and while the cat is healthy, she isn’t that healthy. The cycle of refill-empty happened enough that I got curious about it and had considered throwing an old IP security camera in the closet to see who or what was stealing the food. Instead I decided to deploy the thermal camera and my motion tracking tool.
The Culprits
TLDR: The full video can be found here: https://streamable.com/kzelmb
10:13 PM: The camera was placed up on a shelf almost at the top of the closet, looking down on the food and door. There’s a bug in the frame below and if anyone knows what kind of bug might show up as hot, please reach out. That little guy showed up all night and never stopped moving.
12:28 AM: There are a few neighborhood cats around and ours seems to be friends with all of them. Some kids across the street said their cat was our cat’s boyfriend and after reviewing the video it’s clear how she seduced him: food. Seeing another cat eating the food didn’t come as a surprise and honestly was what I expected to see.
1:02 AM But as the video played on another player entered the game. I think this an opossum? Or possibly a small raccoon? It’s hard to tell.
1:15 AM The third contestant I think is a raccoon who was clearly disappointed that all of the food was gone. I had only left enough to ensure that there would be some left after Rory ate, but didn’t fill the whole container. Anyone who has used an old vending machine that refused to drop your snack will be familiar with this pose:
4:18 AM: Rory finally comes back in at the end of the night, just before sunrise, to find her closet had been ransacked yet again and all of her food stolen.
Conclusion and Next Steps
Dry food has been abandonded almost entirely. Rory now gets wet food twice a day and a little dry food is added early in the morning so she can snack without fear of nocturnal intruders. I already have plans to build her an NFC-enabled cat door, or maybe just a magnet on her collar and hall effect sensors. There are plenty of ways to skin…this problem.