A ball-chasing robot

 

The current version of my robot uses some basic image detection to find a blue ping pong ball. Here you can see it in action:


And here a slightly modified version of the software, where the robot follows me while I am wearing a blue jersey:



The webcam (a Logitech ) is connected to a Beaglebone Black, which does the image processing. The Beaglebone is connected via SPI (which I bitbang on the Beaglebone) to an Atmega 328 hidden inside the robot which is responsible for all the menial tasks like controlling the motors and for the infrared remote. The robot also has a infrared distance sensor at the front so that it does not run into a wall and gets stuck.

This is a typical picture the webcam on the robot shoots:

The non-roundness of the ball on the left stems from me stepping onto it at some point, not from a camera failure.


The Beaglebone uses the Opencv Image processing library to process such a picture. The following is basically the image processing code the Beaglebone runs:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

#include <opencv2/core/core_c.h> #include <opencv2/core/core.hpp> #include <opencv2/imgproc/imgproc_c.h> #include <opencv2/imgproc/imgproc.hpp> #include <opencv2/highgui/highgui_c.h> #include <opencv2/highgui/highgui.hpp> #include <iostream> #include <unistd.h> using namespace std; using namespace cv; int main(){ VideoCapture cap(1); cap.set(CV_CAP_PROP_FRAME_WIDTH, 320); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240); Mat frame, hsvframe; cap.read(frame); cap.read(frame); cap.read(frame); sleep(1); cap.read(frame); hsvframe = Mat::zeros(frame.size(), CV_8UC3); cvtColor(frame, hsvframe, CV_BGR2HSV); Mat imgthreshed = Mat::zeros(frame.size(), CV_8UC3); inRange(hsvframe, Scalar(80, 50,20), Scalar(120, 255,255), imgthreshed); vector< vector<Point> > contours; vector<Vec4i> hierarchy; Mat img = imgthreshed.clone(); findContours(img, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, Point(0,0)); Rect boundrect; Mat drawing = Mat::zeros(frame.size(), CV_8UC3); int area = 10; //this controls which size an object must have to be detected int IdLargestContour = -1; int areabuffer; for(unsigned int i = 0; i < contours.size(); i++){ areabuffer = contourArea(contours[i]); if( areabuffer > area){ area = areabuffer; IdLargestContour = i; } } if(IdLargestContour == -1){ cout << "Nothing to see here" <<endl; imwrite("purepicture.jpg", frame); } else{ cout << "I see something!"<<endl; boundrect = boundingRect(Mat(contours[IdLargestContour])); imwrite("purepicture.jpg", frame); rectangle(frame, boundrect, Scalar(0,255,0), 1,8,0); imwrite("picturewithrect.jpg", frame); imwrite("thresh.jpg", imgthreshed); } return 0; }

For compilation, you have to link against the DLLs of the included libraries - they should be easy to find online. I am using the C++ bindings of Opencv here; there are also C bindings, but the two are largely incompatible, so be wary when you google for examples.

The first 3 lines of the program launch the webcam and set the resolution to 320x240 - the actual program on the Beaglebone only uses 160x120 for speed reasons. The argument to cap is the number of the webcam you want to access - if you only have a single webcam, use 0; my laptop also has an internal one, so I use webcam 1, the external one.

Mat is the basic image format of Opencv, so we define two of those and save a snapshot from the webcam via cap.read(frame) into frame. I am doing this several times here since the first 2 or 3 pictures from the webcam always end up being way too dark for some reason. In line 24, we convert the image to the HSV color space, which is the preferable format for color-based image processing. Line 26 turns the HSV image into a black-and-white image: The blue pixels (those with a hue between 80 and 120) are white, all others black. The results is the following thresholded image:

In line 30, we search for the various white components of the picture (there is probably only one such component here). In line 36 and following, we search for the largest component among those we have just found. Finally, in line 53, we draw a green rectangle around the largest component into our original picture and save all images:

The Beaglebone is fast enough to perform these calculations relatively quickly;the actual bottleneck is that the drivers for the webcam buffer the image of the webcam several times so that you get an outdated picture if you only request a single snapshot. I am currently "solving" this problem by requesting 5 pictures in a row and then only using the last for the image processing, but this takes some additional time.

Building your own Infrared-controlled car, Part 1: Receiving infrared signals

Well, technically not a car, but a tracked vehicle. Here you can see it in action:

Now, why do I write about this? After all, you can easily build such a car purely from Lego parts! That is true. However, later on - when this vehicle is upgraded to something you could call a robot - I want to connect my own little computer and some sensors, and this is not really possible with pure Lego parts.

First, we need a remote. We could build our own, but the result would be rather clumsy, so we use an existing remote. I use the Lego IR remote, but in principle you could use your TV remote - which, of course, would not only control the car, but also still control your TV...

Before we can use the remote, we have to read out the infrared protocol used - we have to know what the remote sends when we press, say, button 1 on the remote. You might luck out and find a complete instruction set online for your remote, but this is no sure thing, and the work needed for the readout is basically the same work we have to do anyway to receive infrared signals at all, so it comes basically for free.

The infrared LED in the remote has two states: It is either plain off, or it will flash at a high frequency, typically 38 kHZ. It will flash for some time, then it is off for some time, then it will flash again. The message we want to send is either encoded into the time it flashes before turning off again, or - more commonly - into the time between two sets of flashes. The point of the flashing is that a 38 kHz flashing signal is much less likely to be of natural origin compared to non-flashing infrared light, which makes receiving less error-prone.

The most convenient way for receiving the signal is using a receiver IC like the TSOP312xx. There are several different ones, depending on the carrier frequency: The 31238 is for 38 kHz, the 31236 for 36 kHz and so on. The 38 kHz frequency is the most common one, and the TSOP31238 should also work for 36 kHZ remotes, with somewhat decreased range.

The TSOP has three pins, two of which are for the power supply. The third is for the output of the demodulated signal: If the IC sees the remote flashing, it will pull the output pin low; otherwise, it is high. On the output pin, we hence do not see the carrier frequency at all, which makes reading the signal much easier.

Now we can connect the TSOP to some microcontroller and measure when the output line goes low. I will use an Arduino since it is easy to connect to a PC. A bare AVR can also easily receive the signal, but getting the data onto your PC is more difficult. The setup looks as follows:

Not much to see here: The orange output line goes to pin 2 on the Arduino, which is the pin for the external interrupt 0, the other two wires are ground and 5 volt. You should check the exact wiring in the data sheet of your receiver - it may be different, and a wrong wiring will probably destroy the receiver. Also, you might want to add a capacitator over ground and 5 volt, but it works for me without. Whenever the output line goes low, the interrupt routine saves the time passed since the last interrupt or (in case of the Arduino code) just the time in microseconds since startup.

Below you find the sourcecode in plain C and for the Arduino; but you should be careful with the Arduino code: the interrupt routine has lots of overhead, and this seems to be the outer limit of what is achievable in this way - I crashed it a few times when the program had a longer interrupt routine, so this approach is probably out for use on the car since there the Arduino has to do other stuff as well.

The source-code in Arduino-speak:

volatile unsigned long timevalues[150];
volatile int i = 0;
int finished = 0;
void setup(){
   attachInterrupt(0, external, FALLING); 
   Serial.begin(9600); 
   Serial.println("starting program");
}
void loop(){

  if(finished == 0){
 if(i > 140){
   finished = 1;
   for(int j = 1; j < 140; j++){
  Serial.println(timevalues[j]-timevalues[j-1]);
     
       }
    
 }
  
  
}
}
void external(){
  //if(timecounter > 1){
    timevalues[i] = micros();
    i++;
 //} 

}

and in AVR-GCC:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77

#define F_CPU 16000000UL #include <avr/io.h> #include <util/delay.h> #include <avr/interrupt.h> #define BAUDRATE 9600 #define BAUD_PRESCALLER (((F_CPU / (BAUDRATE * 16UL))) - 1) //Declaration of our functions void USART_init(void); void USART_send( unsigned char data); volatile int i = 0; volatile int timecounter = 0; uint8_t finished = 0; volatile uint8_t timevalues[150]; int main(void){ USART_init(); //Call the USART initialization code EICRA = (1 << ISC01); //interrupt on falling edge EIMSK = (1 << INT0); //enable INT0-interrupt TCCR0A = (1 << WGM01); //clear timer on compare match TCCR0B = (1 << CS01) | ( 1 << CS00); //prescaler 64 OCR0A = 16; //one timer interrupt every 16 ticks of the clock, which here means every 64 microseconds TIMSK0 |= (1 << OCIE0A); //turn timer interrupt A on sei(); while(1){ //Infinite loop if(finished == 0){ if(i > 100){ cli(); finished = 1; for(int j = 0; j < 100; j++){ USART_send(timevalues[j]); } } } } return 0; } ISR(INT0_vect){ if(timecounter > 1){ timevalues[i] = timecounter; i++; timecounter = 0; } } ISR(TIMER0_COMPA_vect){ timecounter++; } void USART_init(void){ UBRR0H = (uint8_t)(BAUD_PRESCALLER>>8); UBRR0L = (uint8_t)(BAUD_PRESCALLER); UCSR0B = (1<<RXEN0)|(1<<TXEN0); UCSR0C = (3<<UCSZ00); } void USART_send( unsigned char data){ while(!(UCSR0A & (1<<UDRE0))); UDR0 = data; }

The principle is the same both times: Wait for 100 received signals and then send the time intervals over UART. The output of the Arduino program is the time in microseconds between flashes; the output of the C-code (for which you will need a serial terminal, for example Realterm) is in 64 microseconds, i.e. a "10" means roughly 640 microseconds. The precision is still sufficient, and it saves memory and processor cycles.

Part of the output when I press 1 on my TV remote for the Arduino code is

13464 1124 1156 1100 1124 1156 1100 2240 1156 2216 2244 2240 2244 2244 2244 1152 2216 2244 1152 1128 1128 1100 1152 1100 1128 1152 2216 2248 2240 2240 2244 2244 2244 40312 11208 96448 11208 96452 11208 96452 11208 96448 11208 96452 11208 96452 11208 96452 11208 96448 11212 96448 11208 96452 11208 96452 11208 96452 11208 96452 11208 96448 11208 96452 11208 96452 11208 96452 11208 96452 11208 96452 11208 96448 11212

It uses the NEC protocol, and you will find all the numbers above also in the linked document. We start with a pause of 13 ms, signifying the beginning of the transmission. Then we send the code 00000010111111011000000001111111 - a 0 for the short pause of 1.1 ms, a 1 for the long break of 2.2 ms. Then the remote starts a "continue" pattern, which is also described in the link: This signifies the receiver that the button keeps getting pressed.