And here a slightly modified version of the software, where the robot follows me while I am wearing a blue jersey:
The webcam (a Logitech ) is connected to a Beaglebone Black, which does the image processing. The Beaglebone is connected via SPI (which I bitbang on the Beaglebone) to an Atmega 328 hidden inside the robot which is responsible for all the menial tasks like controlling the motors and for the infrared remote. The robot also has a infrared distance sensor at the front so that it does not run into a wall and gets stuck.
This is a typical picture the webcam on the robot shoots:
The non-roundness of the ball on the left stems from me stepping onto it at some point, not from a camera failure.
The Beaglebone uses the Opencv Image processing library to process such a picture. The following is basically the image processing code the Beaglebone runs:
For compilation, you have to link against the DLLs of the included libraries - they should be easy to find online. I am using the C++ bindings of Opencv here; there are also C bindings, but the two are largely incompatible, so be wary when you google for examples.
The first 3 lines of the program launch the webcam and set the resolution to 320x240 - the actual program on the Beaglebone only uses 160x120 for speed reasons. The argument to cap is the number of the webcam you want to access - if you only have a single webcam, use 0; my laptop also has an internal one, so I use webcam 1, the external one.
Mat is the basic image format of Opencv, so we define two of those and save a snapshot from the webcam via cap.read(frame) into frame. I am doing this several times here since the first 2 or 3 pictures from the webcam always end up being way too dark for some reason. In line 24, we convert the image to the HSV color space, which is the preferable format for color-based image processing. Line 26 turns the HSV image into a black-and-white image: The blue pixels (those with a hue between 80 and 120) are white, all others black. The results is the following thresholded image:
The Beaglebone is fast enough to perform these calculations relatively quickly;the actual bottleneck is that the drivers for the webcam buffer the image of the webcam several times so that you get an outdated picture if you only request a single snapshot. I am currently "solving" this problem by requesting 5 pictures in a row and then only using the last for the image processing, but this takes some additional time.