A ball-chasing robot

 

The current version of my robot uses some basic image detection to find a blue ping pong ball. Here you can see it in action:


And here a slightly modified version of the software, where the robot follows me while I am wearing a blue jersey:



The webcam (a Logitech ) is connected to a Beaglebone Black, which does the image processing. The Beaglebone is connected via SPI (which I bitbang on the Beaglebone) to an Atmega 328 hidden inside the robot which is responsible for all the menial tasks like controlling the motors and for the infrared remote. The robot also has a infrared distance sensor at the front so that it does not run into a wall and gets stuck.

This is a typical picture the webcam on the robot shoots:

The non-roundness of the ball on the left stems from me stepping onto it at some point, not from a camera failure.


The Beaglebone uses the Opencv Image processing library to process such a picture. The following is basically the image processing code the Beaglebone runs:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

#include <opencv2/core/core_c.h> #include <opencv2/core/core.hpp> #include <opencv2/imgproc/imgproc_c.h> #include <opencv2/imgproc/imgproc.hpp> #include <opencv2/highgui/highgui_c.h> #include <opencv2/highgui/highgui.hpp> #include <iostream> #include <unistd.h> using namespace std; using namespace cv; int main(){ VideoCapture cap(1); cap.set(CV_CAP_PROP_FRAME_WIDTH, 320); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240); Mat frame, hsvframe; cap.read(frame); cap.read(frame); cap.read(frame); sleep(1); cap.read(frame); hsvframe = Mat::zeros(frame.size(), CV_8UC3); cvtColor(frame, hsvframe, CV_BGR2HSV); Mat imgthreshed = Mat::zeros(frame.size(), CV_8UC3); inRange(hsvframe, Scalar(80, 50,20), Scalar(120, 255,255), imgthreshed); vector< vector<Point> > contours; vector<Vec4i> hierarchy; Mat img = imgthreshed.clone(); findContours(img, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, Point(0,0)); Rect boundrect; Mat drawing = Mat::zeros(frame.size(), CV_8UC3); int area = 10; //this controls which size an object must have to be detected int IdLargestContour = -1; int areabuffer; for(unsigned int i = 0; i < contours.size(); i++){ areabuffer = contourArea(contours[i]); if( areabuffer > area){ area = areabuffer; IdLargestContour = i; } } if(IdLargestContour == -1){ cout << "Nothing to see here" <<endl; imwrite("purepicture.jpg", frame); } else{ cout << "I see something!"<<endl; boundrect = boundingRect(Mat(contours[IdLargestContour])); imwrite("purepicture.jpg", frame); rectangle(frame, boundrect, Scalar(0,255,0), 1,8,0); imwrite("picturewithrect.jpg", frame); imwrite("thresh.jpg", imgthreshed); } return 0; }

For compilation, you have to link against the DLLs of the included libraries - they should be easy to find online. I am using the C++ bindings of Opencv here; there are also C bindings, but the two are largely incompatible, so be wary when you google for examples.

The first 3 lines of the program launch the webcam and set the resolution to 320x240 - the actual program on the Beaglebone only uses 160x120 for speed reasons. The argument to cap is the number of the webcam you want to access - if you only have a single webcam, use 0; my laptop also has an internal one, so I use webcam 1, the external one.

Mat is the basic image format of Opencv, so we define two of those and save a snapshot from the webcam via cap.read(frame) into frame. I am doing this several times here since the first 2 or 3 pictures from the webcam always end up being way too dark for some reason. In line 24, we convert the image to the HSV color space, which is the preferable format for color-based image processing. Line 26 turns the HSV image into a black-and-white image: The blue pixels (those with a hue between 80 and 120) are white, all others black. The results is the following thresholded image:

In line 30, we search for the various white components of the picture (there is probably only one such component here). In line 36 and following, we search for the largest component among those we have just found. Finally, in line 53, we draw a green rectangle around the largest component into our original picture and save all images:

The Beaglebone is fast enough to perform these calculations relatively quickly;the actual bottleneck is that the drivers for the webcam buffer the image of the webcam several times so that you get an outdated picture if you only request a single snapshot. I am currently "solving" this problem by requesting 5 pictures in a row and then only using the last for the image processing, but this takes some additional time.