Building an autonomous flying drone with minimal resources
In this post, we will be exploring how to build an extremely simplified indoor “autonomous flying” drone using visual tracking. It uses an Arduino, a Raspberry Pi, and a cheap toy quadcopter and doesn’t need any fancy equipment.
Motivation
After seeing an awesome Ted talk about quadcopters by Raffaello D’Andrea I thought about if I could build something like this myself.
Now, of course, what I will be building won’t carry a glass of water, or balance a stick. It might seem boring but all it will do is hover in one fixed position. This, however, is the prerequisite for everything else you see in this video. Also, getting a toy drone to be stable in the air was difficult enough.
Moreover, this will be more of a proof of concept than anything else. It shows that even with low resources cool things can be done. With that being said, here we go.
Material
As I said, this doesn’t need any expensive equipment. What I used was:
- Raspberry Pi
- Raspberry Pi Camera
- Arduino Uno
- Hubsan X4
- Some additional resistors, capacitors, wires and a breadboard
The idea
Basically, the Raspberry Pi will constantly capture pictures from the camera under which the quadcopter will be flying. The camera should, therefore, be positioned at a high point in a room so the quad will be seen from a top-down view. As the quadcopter used has 4 LEDs, we can track it using image recognition by filtering out the LED colors. From this information, we can calculate stuff like position, height, direction and also speed if we compare the current with the previous position. Based on these calculations we will send a signal via a Serial connection (USB) to an Arduino, which will output the corresponding voltages to its output pins. This output will then be fed into the remote control of the quadcopter.
That is the basic idea. Theoretically, you could also use the GPIOs on the RPi. However, I wanted to separate the hardware and software side of things.
Hardware Set up
This is relatively straightforward. I just connected everything as previously described. For the remote control, I had to slightly modify it to accept external signals. I didn’t want to break it so even with this mod it still functions perfectly fine.
I attached some wires to the negative and positive leads of the remote and connected those to the ground and 5V of my Arduino so I wouldn’t need any batteries while also providing a common ground between the two devices. I then attached cables to the trimpot outputs of the joysticks.
By then using a highly complicated procedure of putting the trimpots into their 0V position and fixing them there (rubber-bands) we can then overwrite the values read by the radio with those output by the Arduino.
One last problem is that the Arduino doesn’t output a real analog voltage. It instead outputs a PWM signal with a duty cycle corresponding to the voltage. Since I didn’t have a DAC or digital potentiometer lying around I built one myself using two 47µΩ capacitors in parallel for each of the 4 channels and two 100Ohm resistors in parallel for each channel. Something equivalent would also have worked.
Software
This is the part where the magic happens. There are basically two parts of code: One part is the program running on the Arduino, which just writes the values it gets over the serial connection to the corresponding pin. The other part is the Python script running on the RPi. It handles the image recognition and calculations of values.
Detection
By far the most difficult part was getting the detection of the quadcopter to work reliably. For this, the colors red and blue are filtered from the image using the OpenCV inrange function. Since the LEDs are perceived as white, however, this would be problematic, as both detectors (for blue and for red) would detect the LEDs of the other color. So instead, white isn’t detected by the inrange function.
This way, in the detection mask, there will be a white area around the quad’s LEDs, except for where the LED itself is. This part will be black. We can use this to our advantage though. We will use a blob detector to detect these black blobs inside of the white area. This allows us to track the LEDs on the quadcopter and therefore the quadcopter itself.
Calculations
This is relatively straightforward. If we have 4 points we can calculate the position easily, as well as the rotation and height.
If we subtract the old values from the current values, we can also get the velocity of the quad.
The commands sent to the remote are a mixture between canceling out any movement and slightly steering into the direction of the point we want the quad to be.
See it in action
Roundup
Most of the things I did to accomplish this I have only summarized here. It took a lot of trial and error to get it working, but now everything except throttle control works.
Throttle control isn’t really possible on a quad of this scale. A slight decrease in e.g. voltage can already greatly influence how the quad flies. A reliable automated response is therefore extremely difficult and still needs some working.
However, most of the limitations here are due to the fact that I challenged myself to do this with the least resources I possibly could. For example, a larger quad would be by far more stable, but of course, it would also cost more and would also need more space. Also, a ‘regular’ computer instead of a low-powered RPi would be helpful, as a camera with a higher resolution could be used and tracking would be even more precise. Then again I find it quite amazing that a computer that fits on the palm of you hand is autonomously steering a quadcopter.
Possible improvements
- Get throttle control working
- Make fixed point movable
- Allow for path planning