SCE 12-0518

Original: AN AUTOMATED FLIGHT CONTROL SYSTEM FOR QUADROTOR FLYING VEHICLE USING EMBEDDED MICROCONTROLLER

Changed to: VISION BASED QUADROTOR NAVIGATION



Gaurav Gupta



School Of Computer Engineering

2013


AR. Drone Following Tag

Development of DJI Flamewheel F450 Quadcopter



Abstract

Quadrotors have been a subject of interest for military for surveillance and warfare purpose since 19th century. However recently they are becoming increasingly popular amongst RC hobby enthusiasts and researcher alike due to their agility, maneuverability and Vertical Takeoff and Landing capability. The rapid innovation and research in this area motivates us to explore the integration of computer vision with quadrotors to make a fully autonomous system in GPS enabled as well as GPS denied environments.


For the first phase of the project we used a commercial off the shelf AR. Drone quadrotor to integrate with ROS (Robot operating system) in order to chase a target tag. The AR. Drone is a ready to fly system, which uses Wi Fi to connect to a remote computer running ROS node to communicate with AR. Drone. The remote computer node takes the navigation data input from AR. Drone, does position estimation relative to tag and returns a command back to AR. Drone to maintain its position over the tag. The system performed well. The AR. Drone flew for 40 seconds and maintained its position over a tag moving in arbitrary fashion.


The second phase of the project concerns with developing a more powerful modular quadrotor with higher payload capacity and longer battery life. This phase was not the original objective of the project but was included later on considering the limitations of AR. Drone. A DJI Flamewheel quadrotor kit, Ardupilot Mega Autopilot and 3DR telemetry system was chosen to implement and improve the functionality of AR. Drone. The quadrotor was assembled and tested successfully, however the integration of camera and implementation of image processing over ROS still remains to be done.


Acknowledgements

The author would like to express his sincere gratitude to the following people for successful completion of this project.

Assistant Professor Suresh Sundaram - Foremost, the author would like to thank his FYP professor for his invaluable guidance and encouragement throughout the course of this project. He continuously evaluated the author’s progress and provided many helpful tips and insights to steer the project in the right direction. The author believes that Prof. Suresh’s encouragement and patience was instrumental in the fulfillment of the project.

Mr. Arunesh Waran – For explaining the author the basics and mathematical models of quadrotors and having mind broadening intellectual discussions with the author. He guided the author during the initial days of the project based on author’s interests and passions. He also helped in ordering the parts of APM based quadcopter while author was on vacations.

Mr. Badrinarayanan Rangarajan- For helping the author purchase the parts for new quadrotor, providing and arranging for all the tools and equipment required for the successful completion of this project. He also helped author explore his passions and mentored him in his future career endeavors.

Mr. Liu Zhichao- For helping the author in control engineering lab with the AR. Drone quadcopter used for this project. He gave the author a moral encouragement and helped him get started with ROS and AR. Drone integration.

Mr. Tan Piah Chye- For helping the author as the lab in-charge of Hardware Projects Lab where the author was assigned a workstation. He helped author make arrangements for the in lab testing of quadrotor. He also helped author replace the faulty workstation CPU with a new one.

Family and Friends - For encouraging, supporting and backing the author.

Table of Contents

Abstract ii

Acknowledgements iii

Table of Contents iv

List of Figures vi

1 Introduction 1

1.1 Background 1

1.1.1 Definition and purpose of a quadrotor 1

1.1.2 Quadrotor Dynamics 1

1.1.3 History of Quadrotors 3

1.2 Motivation 7

1.3 Objective And Scope 8

1.3.1 Objective 8

1.3.2 Scope 8

2 Project Planning 9

2.1 Project Requirements 9

2.2 Project Strategy 9

2.3 Project Schedule 10

3 Vision Based Navigation: AR. Drone 11

3.1 Resources Required 11

3.1.1 Hardware 11

3.1.2 Software 11

3.2 Making AR. Drone fly 12

3.3 Learning ROS 13

3.3.1 What is ROS? 13

3.3.2 Installing ROS 13

3.3.3 Create ROS workspace 14

3.3.4 Create a Package in Workspace 15

3.3.5 Understanding ROS Nodes 15

3.3.6 Understanding ROS Topic 17

3.3.7 Understanding ROS Services and Parameters 20

3.3.8 Using roslaunch 22

3.4 Controlling AR. Drone with ROS 23

3.4.1 Required Packages 23

3.4.2 First Flight 24

3.4.3 Keyboard Commands 24

3.4.4 Terminal Commands 25

3.5 Getting Video and Navigation Data on ROS 26

3.5.1 Video 26

3.5.2 Navdata 27

3.6 Following a Tag 28

4 Vision Based Navigation: APM: Copter 31

4.1 Resources Required 31

4.1.1 Hardware 31

4.1.2 Software 34

4.2 Assembly of Quadrotor 34

4.2.1 Assembly of Frame 34

4.2.2 ESC wiring 35

4.2.3 Installation of APM 35

4.2.4 Connecting RC inputs to APM 36

4.2.5 Connecting APM outputs to ESCs 36

4.2.6 Mounting GPS module. 37

4.2.7 Mounting 3DR radio module 37

4.2.8 Full Quadrotor 38

4.3 Hardware Calibration and Testing 39

4.3.1 Loading firmware onto APM 39

4.3.2 Selecting frame type 40

4.3.3 Compass calibration 41

4.3.4 Accelerometer Calibration 41

4.3.5 Radio Calibration 42

4.3.6 Telemetry Setup 43

4.4 The Flight 45

4.4.1 In Lab Testing 45

4.4.2 Outdoor Testing 45

5 Conclusion 47

6 Recommendations/Future Work 48

References 49

Appendix 50

Appendix 1: Safety Information for Quadrotors 50








List of Figures

Figure 1: Yaw, Pitch and Roll rotations of a quadrotor 2

Figure 2: Illustration of the dynamics of a quadrotor 3

Figure 3: Bréguet-Richet Gyroplane No. 1 of 1907 4

Figure 4: The Oemichen No. 2 of 1922 5

Figure 5: Quadrotor designed by Dr. Bothezat an Ivan Jerome. 6

Figure 6: Convertawings Model A helicopter 6

Figure 7: Gantt chart for initial project schedule 10

Figure 8: Gantt chart for final project schedule 10

Figure 9: Parrot AR. Drone 2.0 11

Figure 10: AR. FreeFlight 2.3.3 Menu 12

Figure 11: User Interface of AR. FreeFlight 2.3.3 12

Figure 12: TurtleSim demo 16

Figure 13: rqt_graph showing two nodes 18

Figure 14: Example of message published over a topic 18

Figure 15: Output of rosservice call spawn 21

Figure 16: Graph of nodes running while AR. Drone flying 25

Figure 17: Horizontal camera view 26

Figure 18: Vertical camera view 26

Figure 19: View of Navdata 27

Figure 20: Oriented Roundel tag 28

Figure 21: rqt_graph of node while tag following 29

Figure 22: AR. Drone following oriented roundel tag 30

Figure 23: DJI Flame Wheel F450-ARF 31

Figure 24: 3-Cell 2200MAh 40C Li-Po Battery 31

Figure 25: XT60 Male Connector 32

Figure 26: Battery Charger 32

Figure 27: ArduPilot Mega Kit 32

Figure 28: 3DR Radio Telemetry Kit 33

Figure 29: Spektrum Dx5e Transmitter 33

Figure 30: Desktop running APM Mission Planner 33

Figure 31: DJI Flamewheel assembly diagram 34

Figure 32: ESC circuit schematic 35

Figure 33: APM mount on DJI Flamewheel 35

Figure 34: RC receiver to APM connection pin layout 36

Figure 35: APM to ESC pin layout 37

Figure 36: Quadrotor fully assembled 38

Figure 37: Install Firmware screen of APM mission planner 39

Figure 38: Selecting Frame Type 40

Figure 39: APM compass calibration 41

Figure 40: Accelerometer calibration 41

Figure 41: Radio calibration 42

Figure 42: Flight mode setup 43

Figure 43: 3DR radio configuration values 43

Figure 44: Configuring 3DR radio module 44

Figure 45: First liftoff in Lab 45

Figure 46: Flight path of the quadrotor during outdoors testing 46



1Introduction

1.1Background

1.1.1Definition and purpose of a quadrotor

A quadrotor, or quadrotor helicopter, is an aircraft that becomes airborne due to the lift force provided by four rotors usually mounted in cross configuration, hence its name. It is an entirely different vehicle when compared with a helicopter, mainly due to the way both are controlled. Helicopters are able to change the angle of attack of its blades, quadrotors cannot.

UAVs like quadrotors have several advantages over fixed-wing airplanes. They can move in any direction and are capable of hovering and fly at low speeds. In addition, the Vertical Take-Off and Landing (VTOL) capability allows deployment in almost any terrain while fixed-wing aircraft require a prepared airstrip for takeoff and landing. Given these characteristics, quadrotors can be used in search and rescue missions, meteorology, penetration of hazardous environments (e.g. exploration of other planets) and other applications suited for such an aircraft [1].

1.1.2Quadrotor Dynamics

Each rotor in a quadrotor is responsible for a certain amount of thrust and torque about its center of rotation, as well as for a drag force opposite to the rotorcraft’s direction of flight. The quadrotor’s propellers are not all alike. In fact, they are divided in two pairs, two pusher and two puller blades that work in contra-rotation. As a consequence, the resulting net torque can be null if all propellers turn with the same angular velocity, thus allowing for the aircraft to remain still around its center of gravity.

In order to define an aircraft’s orientation (or attitude) around its center of mass, aerospace engineers usually define three dynamic parameters, the angles of yaw, pitch and roll. This is very useful because the forces used to control the aircraft act around its center of mass, causing it to pitch, roll or yaw. Changes in the pitch angle are induced by contrary variation of speeds in propellers 1 and 3 (Figure 1), resulting in forward or backwards translation. If we do this same action for propellers 2 and 4, we can produce a change in the roll angle and we will get lateral translation. Yaw is induced by mismatching the balance in aerodynamic torques (i.e. by offsetting the cumulative thrust between the counter-rotating blade pairs). So, by changing these three angles in a quadrotor we are able to make it maneuver in any direction (Figure 2)[1].


Figure 1: Yaw, Pitch and Roll rotations of a quadrotor


Figure 2: Illustration of the dynamics of a quadrotor


1.1.3History of Quadrotors

This story begins in the 20th century, when Charles Richet, a French scientist and academician, built a small, un-piloted helicopter [2]. Although his attempt was not a success, Louis Bréguet, one of Richet’s students, was inspired by his tutor’s example. Later in 1906, Louis and his brother, Jacques Bréguet began the construction of the first quadrotor. Louis executed many tests on airfoil shapes, proving that he had at least some basic understanding of the requirements necessary to achieve vertical flight.

In 1907 they had finished the construction of the aircraft which was named Bréguet-Richet Gyroplane No. 1 (Figures 3), a quadrotor with propellers of 8.1 meters in diameter each, weighting 578 kg (2 pilots included) and with only one 50 hp (37.3 kW) internal combustion engine, which drove the rotors through a belt and pulley transmission. Of course at that time they had no idea how they would control it, the main concern was to ensure the aircraft would achieve vertical flight. The first attempt of flight was done in between August and September of 1907 with witnesses saying they saw the quadrotor lift 1.5 m into the air for a few moments, landing immediately afterwards. Those same witnesses also mentioned the aircraft was stabilized, and perhaps even lifted by men assisting on the ground [1].

Figure 3: Bréguet-Richet Gyroplane No. 1 of 1907


Discouraged by the lack of success of the Gyroplane No. 1, Bréguet and his mentor continued their pursuit to build vertical flight machines and afterwards also temporarily dedicated themselves to the development of fixed-wing aircraft, area where they became very successful. Louis never abandoned his passion for vertical flight aircraft and in 1932 he became one of the pioneers of helicopter development [2].

Etienne Oemichen, another engineer, also began experimenting with rotating-wing designs in 1920. He designed a grand total of six different vertical lift machines. The first model failed in lifting from the ground but Oemichen was a determined person, so he decided to add a hydrogen-filled balloon to provide both stability and lift. His second aircraft, the Oemichen No. 2 (Figure 4), had four rotors and eight propellers, supported by a cruciform steel-tube framework layout. Five of the propellers were meant to stabilize the machine laterally, another for steering and two for forward propulsion. Although rudimentary, this machine achieved a considerable degree of stability and controllability, having made more than a thousand test flights in the middle of that decade. It was even possible to maintain the aircraft several minutes in the air. In the 14th of May the machine was airborne for fourteen minutes and it flew more than a mile. But Oemichen was not satisfied with the poor heights he was able to fly, and the next machines had only a main rotor and two extra anti-torque rotors.




Figure 4: The Oemichen No. 2 of 1922



The army also had an interest for vertical lift machines. In 1921, Dr. George de Bothezat and Ivan Jerome were hired to develop one for the US Army Air Corps. The result was a 1678 kg structure with 9 m arms and four 8.1 m six-blade rotors (Figure 5). The army contract required that the aircraft would hover at 100 m high, but the best they achieved was 5 m. At the end of the project Bothezad demonstrated the vehicle could be quite stable, however it was underpowered and unresponsive, among other technical problems [1].

Figure 5: Quadrotor designed by Dr. Bothezat an Ivan Jerome.



Later in 1956, a quadrotor helicopter prototype called “Convertawings Model A” (Figure 6) was designed both for military and civilian use. Varying the thrust between rotors controlled it, and its flights were a success, even in forward flight. The project ended mainly due to the lack of demand for the aircraft [1].

Figure 6: Convertawings Model A helicopter




1.2Motivation

In the last few decades, small-scaled Unmanned Aerial Vehicles (UAVs) have become more commonly used for many applications. The need for aircraft with greater maneuverability and hovering ability has led to current rise in quadcopter research. The four-rotor design allows quadcopters to be relatively simple in design yet highly reliable and maneuverable [3]. Quadcopters have several advantages over other VTOL systems, they are [1]:

With advantages and features that a quadrotor has to offer, it has attracted the attention of researchers from around the world. With the research and development in quadrotors, proceeding at such a rapid pace, it is evident that in future, these machines would become an indispensible part of our daily lives. Hence, it would be of interested to push the limits of these machines to next level by integrating computer vision in them. This would be a step towards a completely autonomous quadrotor system, which can efficiently perform navigation and maneuvers in GPS enabled as well as GPS denied environment.

1.3Objective And Scope


1.3.1Objective

The Objective of this project is to make a quadrotor follow an object using vision and maintain its position over the object.


1.3.2Scope

The project focuses on using commercial off-the-shelf AR. Drone to follow a tag based on the vision from the vertical view camera attached on board. The software used for this project will be Robot Operating System (ROS) hosted on a remote laptop which will receive the navigation data from AR. Drone, perform a position estimate with respect to the tag and send command to the quadrotor to position it on top of the tag in near real time, hence following the tag. The interface used for communication will be Wi-Fi 802.11 wireless local area network. The system will also allow user on the remote computer to view the image from the vertical camera along with the navigation data and allow near real time human intervention to all the commands to the quadrotor including takeoff, land, roll, pitch, yaw and emergency shut down.


The project can be further extended to a more modular and open source hardware and software platform with higher payload capacity, longer communication distance and longer battery backup time.


2Project Planning

2.1Project Requirements

The requirements for the project are listed below:

2.2Project Strategy

To meet the above requirements and extend the project to an open-source high performance modular platform the project was divided into two major phases, which were further divided into sub-phases as:


Phase 1: Vision Based Navigation on AR. Drone

Sub-Phase 1: Making AR. Drone fly

Sub-Phase 2: Learning ROS

Sub-Phase 3: Controlling AR. Drone with ROS

Sub-Phase 4: Getting video and navigation data on ROS

Sub-Phase 5: Designing a control loop to make ROS follow a tag


Phase 2: Development of open-source high performance modular quadrotor

Sub-Phase 1: Purchase of Material

Sub-Phase 2: Assembly of Quadrotor

Sub-Phase 3: Calibration and Testing

Sub-Phase 4: Integration of FPV

Sub-Phase 5: Integration of ROS

Sub-Phase 6: Implementation of tag detection and tracking

2.3Project Schedule

The Figure 7 shows the Gantt chart for project as was initially planned. Initially the project only required working on AR. Drone, as this was the quadrotor already available. Hence, only one phase.

Figure 7: Gantt chart for initial project schedule

However there were several factors responsible for deviation from the initial plan. The final version of the plan is presented in Figure 8. The reason for this deviation was that Phase 1 progress was faster than expected hence it could be completed earlier. Based on the author’s email to Prof. Suresh (Appendix 5), the project was extended to include the development of another open source high-performance modular quadrotor which called for inclusion of Phase 2.

Due to inclusion of Phase 2 in the middle of the project, all the deadlines had to be tightened. The purchase was scheduled during the summer break as research for products to be ordered, negotiation and delivery would take time. Nevertheless there was still a delay in purchase of the flight controller board and telemetry system (Appendix 6 and Appendix 7). This caused the Phase 2 to be completed only to the point where quadrotor was assembled and tested.

Figure 8: Gantt chart for final project schedule

3Vision Based Navigation: AR. Drone

3.1Resources Required

3.1.1Hardware

The hardware resources required for this phase of the project were:

  1. AR. Drone: An 802.11 Wi-Fi controlled quadcopter built by a French company Parrot [4]. It can be bought from Apple store for S$ 449 [5].

Figure 9: Parrot AR. Drone 2.0

  1. Laptop: For this project MacBook Pro 2012 with 2.9 GHz Intel Core i7, 8GB RAM, running OS X Mountain Lion 10.8.4 was used.

3.1.2Software

The software resources required for this phase of the project were mainly free software:

  1. Oracle VirtualBox: VirtualBox 4.2.18 was used to host Ubuntu on MacBook Pro 2012 [6].

  2. Ubuntu OS: Ubuntu 12.04.03 LTS was used to run ROS [7].

  3. Robot Operating System: ROS Hydro was used to create the necessary nodes to control AR. Drone [8].

3.2Making AR. Drone fly

The first flight was achieved by flying the AR. Drone with the default app AR. FreeFlight 2.3.3 provided by Parrot on Google Play Store for Android [9] as shown in Figure 10 and Figure 11. Different settings were tried, like camera switch, joystick and gyro control, video recording and hands on experience and familiarity with the model were gained.

Figure 10: AR. FreeFlight 2.3.3 Menu

Figure 11: User Interface of AR. FreeFlight 2.3.3

3.3Learning ROS

3.3.1What is ROS?

Robot Operating System (ROS) is a software framework for robot software development, providing operating system-like functionality on a heterogeneous computer cluster. ROS was originally developed in 2007 under the name switchyard by the Stanford Artificial Intelligence Laboratory in support of the Stanford AI Robot STAIR project. As of 2008, development continues primarily at Willow Garage, a robotics research institute/incubator, with more than twenty institutions collaborating in a federated development model [10].


ROS provides libraries and tools to help software developers create robot applications. It provides hardware abstraction, device drivers, libraries, visualizers, message passing, package management, and more. ROS is licensed under an open source, BSD license [11].


3.3.2Installing ROS

To install ROS hydro on Ubuntu 12.04.03 Open the terminal and enter command

More information and official documentation can be found at [11].


1. Setup sources.list

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu precise main" > /etc/apt/sources.list.d/ros-latest.list'


2. Set up keys

wget http://packages.ros.org/ros.key -O - | sudo apt-key add -


3. Install

sudo apt-get install ros-hydro-desktop-full


4. Initialize rosdep

sudo rosdep init

rosdep update


5. Getting rosinstall

sudo apt-get install python-rosinstall


6. Source setup.bash

$ source /opt/ros/hydro/setup.bash


3.3.3Create ROS workspace

1. Create workspace

$ mkdir -p ~/catkin_ws/src $ cd ~/catkin_ws/src $ catkin_init_workspace


2. Make workspace

$ cd ~/catkin_ws/ $ catkin_make


3. Source setup.bash

$ source devel/setup.bash


3.3.4Create a Package in Workspace


1. Change directory to workspace

$ cd ~/catkin_ws/src


2. Create the package

$ catkin_create_pkg beginner_tutorials std_msgs rospy roscpp


3. Make the package

$ rosmake beginner_tutorials


3.3.5Understanding ROS Nodes

A node really isn't much more than an executable file within a ROS package. ROS nodes use a ROS client library to communicate with other nodes. Nodes can publish or subscribe to a Topic. Nodes can also provide or use a Service.

1. Install

$ sudo apt-get install ros-hydro-ros-tutorials


2. Run roscore

$ roscore


3. To see what roscore did, open a new terminal and type

$ rosnode list

You will see:

/rosout


4. Using rosrun, in a new terminal type

$ rosrun turtlesim turtlesim_node

You will see the turtlesim window:

Figure 12: TurtleSim demo


5. To see the nodes running, in a new terminal type

$ rosnode list

You will see:

/rosout

/turtlesim


3.3.6Understanding ROS Topic

1. Run each in a new terminal

$ roscore


$ rosrun turtlesim turtlesim_node


$ rosrun turtlesim turtle_teleop_key


Now you can use the arrow keys of the keyboard to drive the turtle around. If you can not drive the turtle select the terminal window of the turtle_teleop_key to make sure that the keys that you type are recorded.

2. Using rqt_graph, in a new terminal run

$ rosrun rqt_graph rqt_graph





You will see something similar to:

Figure 13: rqt_graph showing two nodes

As you can see, the turtlesim_node and the turtle_teleop_key nodes are communicating on the topic named /turtle1/command_velocity.


3. Using rostopic, In a new terminal type

$ rostopic echo /turtle1/cmd_vel


Run the turtle using arrow keys and you will see a message being published like this

Figure 14: Example of message published over a topic


4. Using rostopic list, In a new terminal type

$ rostopic list -v

This displays a verbose list of topics to publish to and subscribe to and their type.

Published topics:

* /turtle1/color_sensor [turtlesim/Color] 1 publisher

* /turtle1/command_velocity [turtlesim/Velocity] 1 publisher

* /rosout [roslib/Log] 2 publishers

* /rosout_agg [roslib/Log] 1 publisher

* /turtle1/pose [turtlesim/Pose] 1 publisher

Subscribed topics:

* /turtle1/command_velocity [turtlesim/Velocity] 1 subscriber

* /rosout [roslib/Log] 1 subscriber


5. To see the type of message and its data type

$ rosmsg show geometry_msgs/Twist

You will see

geometry_msgs/Vector3 linear

float64 x

float64 y

float64 z

geometry_msgs/Vector3 angular

float64 x

float64 y

float64 z


6. To publish a data on to a topic currently advertised

$ rostopic pub -1 /turtle1/cmd_vel geometry_msgs/Twist -- '[2.0, 0.0, 0.0]' '[0.0, 0.0, 1.8]'


This command will send a single message to turtlesim telling it to move with an linear velocity of 2.0, and an angular velocity of 1.8

3.3.7Understanding ROS Services and Parameters

Services are another way that nodes can communicate with each other. Services allow nodes to send a request and receive a response.

1. To see the list of services being offered

$ rosservice list


The list command shows us that the turtlesim node provides nine services: resetclearspawnkill,turtle1/set_pen/turtle1/teleport_absolute/turtle1/teleport_relative, turtlesim/get_loggers, and turtlesim/set_logger_level. There are also two services related to the separate rosout node: /rosout/get_loggers and

/rosout/set_logger_level.


2. rosservice type

To see the type of the service

$ rosservice type clear

You will see

std_srvs/Empty

This service is empty, this means when the service call is made it takes no arguments (i.e. it sends no data when making a request and receives no data when receiving a response). 

3. rosservice call

$ rosservice call clear

This does what we expect, it clears the background of the turtlesim_node.

$ rosservice call spawn 2 2 0.2 ""

The service call returns with the name of the newly created turtle

name: turtle2

Figure 15: Output of rosservice call spawn

3.3.8Using roslaunch


roslaunch starts nodes as defined in a launch file.

1. Go to beginner_tutorial package

$ roscd beginner_tutorials


2. The make launch directory

$ mkdir launch

$ cd launch


3. Create a launch file called turtlemimic.launch and paste the following and save.

<launch>

<group ns="turtlesim1">

<node pkg="turtlesim" name="sim" type="turtlesim_node"/>

</group>

<group ns="turtlesim2">

<node pkg="turtlesim" name="sim" type="turtlesim_node"/>

</group>

<node pkg="turtlesim" name="mimic" type="mimic">

<remap from="input" to="turtlesim1/turtle1"/>

<remap from="output" to="turtlesim2/turtle1"/>

</node>

</launch>


4. roslaunching

$ roslaunch beginner_tutorials turtlemimic.launch

Two turtlesims will start and in a new terminal

3.4Controlling AR. Drone with ROS

3.4.1Required Packages

To control AR. Drone using ROS, a ROS driver created by Autonomy Lab ardrone_autonomy was used [12]. To install this driver change to ROS workspace and pull the driver from the GitHub as follows.


$ roscd

$ git clone https://github.com/AutonomyLab/ardrone_autonomy.git

In addition to the above driver, a tutorial node made by Mike Hammer [13] was also used to fly the AR. Drone using ardrone_autonomy. To download the code, use.


$ git clone https://github.com/mikehamer/ardrone_tutorials.git

$ rospack profile

Now to compile the codes use:


$ roscd ardrone_autonomy

$ ./build_sdk.sh

$ rosmake –a


3.4.2First Flight


To fly the AR. Drone use following steps


1. Turn on the AR. Drone by connecting battery


2. Connect the laptop to AR. Drone Wi-Fi The network will be called ardrone2_v2.1.20


3. Open terminal and type

$ roslaunch ardrone_tutorials keyboard_controller.launch

4. Use the commands below to fly and control the AR. Drone

3.4.3Keyboard Commands


The following commands can be issued to AR. Drone via keyboard, giving human operator the access to interrupt AR. Drone at any time.

3.4.4Terminal Commands

1. To view the Navigation data

$ rostopic echo /ardrone/navdata

2. To toggle between horizontal and vertical camera

$ rosservice call /ardrone/togglecam

3. To view the nodes running and topics in a graph form

$ rqt_graph

You will see something like this:

Figure 16: Graph of nodes running while AR. Drone flying


3.5Getting Video and Navigation Data on ROS

3.5.1Video

The video can be streamed by subscribing to /ardrone/image_raw. A python script used in ardrone_tutorials to stream the image and display it on a GUI can be found in Appendix 3. The video image from the horizontal camera (front facing camera) is HD (720p-30fps) video stream as shown in Figure 17. The video from the vertical camera (bottom facing camera) is QVGA (320*240) 60fps video stream up scaled to 720p as shown in Figure 18.

Figure 17: Horizontal camera view

Figure 18: Vertical camera view

3.5.2Navdata

The navdata can be streamed by subscribing to /ardrone/navdata. The code to subscribe navdata and use it for tag following can be found in Appendix 1. The navdata contains all the values of IMU, accelerometers, velocity, orientation and info about the detected tag as show in Figure 19 below.


Figure 19: View of Navdata

3.6Following a Tag

The tag to be followed was rounded oriental, which came inside the box with AR. Drone

as shown in Figure 20 below.


Figure 20: Oriented Roundel tag


A separate package fyp_ardrone was created to run a node which would subscribe to /ardrone/navdata and publish /ardrone/takeoff, /ardrone/land and /cmd_vel messages to /ardrone_driver from ardrone_autonomy package


The source code takeoff.cpp (Appendix 1) was written in the src folder. A separate launch file (Appendix 2) was created to run ardrone_driver and keyboard_controller from ardrone_autonomy and ardrone_tutorial packages respectively. The settings in the launch file were set to make the AR. Drone detect the Oriented Roundel tag from the bottom camera in an indoor environment according to AR. Drone Developer Guide 2.0

The AR. Drone was placed over the oriented roundel tag and launch file was launched followed by takeoff node


$ roslaunch fyp_ardrone keyboard_controller_with_roundel.launch

$ rosservice call /ardrone/togglecam

$ rosrun fyp_ardrone takeoff

The rqt_graph for the above nodes was displayed like in Figure 21


Figure 21: rqt_graph of node while tag following

The AR. Drone performed as shown in the snapshots of the video below in Figure 22. The original video can be accessed from [14]. The log file output while tag following was recorded. A portion of log file can be obtained from Appendix 4.



Figure 22: AR. Drone following oriented roundel tag


4Vision Based Navigation: APM: Copter

4.1Resources Required

4.1.1Hardware

The hardware resources required for this phase of the project are:

1. DJI Flame Wheel F450-ARF (ESC & MOT)

Figure 23: DJI Flame Wheel F450-ARF

2. Haiyin 3-Cells 2200 MAh 40C Li-Po battery

Figure 24: 3-Cell 2200MAh 40C Li-Po Battery








3. XT60 Male Connector

Figure 25: XT60 Male Connector

4. Yuntong 2S/3S balance Li-Po battery charger

Figure 26: Battery Charger

5. ArduPilot Mega 2.5+ Kit with uBlox GPS


Figure 27: ArduPilot Mega Kit

6. 3DR Radio Telemetry Kit – 433 Mhz

Figure 28: 3DR Radio Telemetry Kit


7. Spektrum Dx5e 5 channel transmitter and receiver

Figure 29: Spektrum Dx5e Transmitter

8. A desktop or laptop running Windows 7.


Figure 30: Desktop running APM Mission Planner

4.1.2Software

The software required for this Phase of the project is APM Mission Planner. Which can be downloaded from [15].

4.2Assembly of Quadrotor

4.2.1Assembly of Frame

The quadrotor frame was assembled as shows in the Figure below. All contents necessary were found in DJI Flame wheel kit, except the 2.0mm Hex key, which was purchased separately.


Figure 31: DJI Flamewheel assembly diagram


The ESCs were stuck on each respective arm approximately equidistant from the center.

The battery was the placed between the top and the bottom board maintaining the center of gravity in the geometric center of the frame.


4.2.2ESC wiring

The ESC for each motor was wired as shown in the Figure 32 with the power wires soldered on the power distribution board.


Figure 32: ESC circuit schematic


4.2.3Installation of APM

The APM board was placed on the top board of the frame with foam buffer in between. This was done to reduce high frequency vibrations from the motors causing disturbance in IMU readings [16].


Figure 33: APM mount on DJI Flamewheel

4.2.4Connecting RC inputs to APM

A special rainbow jumper cable was created to connect the signal pins of the RC receiver to APM input pins as can be seen in Figure 33. The receiver was also powered using this cable, the power source being the 5V power rails on APM. The connections from the RC receiver to APM were made according to the Figure below.


Figure 34: RC receiver to APM connection pin layout





4.2.5Connecting APM outputs to ESCs

The 3wire cables on ESC were used to connect to the output pins of the APM. The +5V (red) wires of the ESCs 3wire connector were cut to isolate the APM rails from ESC. (Note: since ESCs were OPTO ESC which do not supply power to servos, this step was not necessary but was still performed to isolate APM from damage in case of current leakage).


The quadrotor was configured in X orientation and the connections between the APM and ESC were made according to the Figure 35.


Figure 35: APM to ESC pin layout


4.2.6Mounting GPS module.


The GPS module was placed such that:


  1. The square antenna is facing up.

  2. The GPS has a clear view of the sky and is not under spinning propellers.

  3. It is away from the radio transmission equipment like 3DR radio module.


4.2.7Mounting 3DR radio module


The 3DR radio module was placed beside the RC receiver so that it is farthest from the GPS module and the antenna does not collides with the propellers.

4.2.8Full Quadrotor


The full quadrotor after assembly look like as in Figure 36


Figure 36: Quadrotor fully assembled


4.3Hardware Calibration and Testing

To install firmware on APM and calibrate it, APM Mission Planner is needed. The APM Mission Planner was downloaded from [15] and installed on the desktop computer.

4.3.1Loading firmware onto APM

To load the firmware onto the APM, the APM was connected to Mission Planner via micro USB (With battery disconnected). Windows automatically detected the driver. COM3 Arduino Mega 2560 was selected with baud rate 115200. Under the INITIAL SETUP tab under Install Firmware Arducopter v3.0.1 Quad was selected and firmware was installed. As shown in the figure below:


Figure 37: Install Firmware screen of APM mission planner

4.3.2Selecting frame type

After firmware installation, the APM was connected to Mission Planner using MAVLink [17] by pressing the connect button. Under the Mandatory Hardware section, the frame type was selected as ‘X’ configuration as shown in the Figure below.

Figure 38: Selecting Frame Type


4.3.3Compass calibration

Under the Mandatory Hardware section the compass calibration was selected. Enable and AutoDec boxes were checked and live calibration button was clicked. The quadrotor was rotated around all the axis to calibrate the compass. Upon completion of the calibration the GUI returned the offset values.

Figure 39: APM compass calibration

4.3.4Accelerometer Calibration

Under the Mandatory Hardware section the Accel calibration was selected. AC 3.0+ box was checked. The quadrotor was placed in Level, Left Side, Right Side, Nose Down, Nose Up and Back Side position as directed by GUI. Upon completion of the calibration the GUI returned an acknowledgement.


Figure 40: Accelerometer calibration

4.3.5Radio Calibration

We are using Spektrum Dx5e 5 channel radio, which has Mode 2 configuration. For Mode 2 transmitters, the left stick controls throttle and yaw; the right stick controls pitch and roll. The channel configuration is as follows.


Channel 1: low = roll left, high = roll right.
Channel 2: low = pitch forward, high=pitch back.
Channel 3: low = throttle down (off), high = throttle up.
Channel 4: low = yaw left, high = yaw right.

Channel 5: low = flight control 1, high = flight control 2


To calibrate the radio, radio calibration was selected under Mandatory Hardware section. The calibrate radio button was pressed and the control sticks were moved to their minimum and maximum position until all the results were recorded by the red lines in the GUI. (The normal values are around 1100 for minimum and 1900 for maximum.)


Figure 41: Radio calibration


Under the Mandatory Hardware section, Flight Modes was selected and Flight Mode 1 was set to Stabilize and Flight Mode 6 was set to Loiter. These are the modes that can be switched by channel 5 switch during the flight to manually control the quadrotor in stabilize mode or use GPS assisted loiter mode to hover and control the position.

Figure 42: Flight mode setup

4.3.6Telemetry Setup

The 3DR telemetry system was used to send and receive data between the ground station and quadcopter. The telemetry operates at 433 MHz. To configure the telemetry system one radio was attached to computer via USB and the other one attached to the APM using the cable provided. The Windows detected the drivers automatically assigning COM 4 to the telemetry module. Baud rate of 57600 was selected in Mission Planner and under the 3RD Radio section the following values were input:

Figure 43: 3DR radio configuration values


The same process was repeated on the other 3DR radio module and it was configured with the same values.


After the configuration the radio module on computer was powered with USB and radio module on APM was powered with battery and it was found that both established a connection and real time values could be observed in the Mission Planner from APM.

It is to be noted that both USB and Radio cannot be used to connect to APM at the same time as they both share the same port.


Figure 44: Configuring 3DR radio module


4.4The Flight

4.4.1In Lab Testing

The first lift off was achieved in Hardware Projects Lab with a safety arrangement such that one end of a string attached to a pen underneath a cardboard, the cardboard taped firmly to the ground and the other end of the string attached to quadrotor as shown in the figure below.

Figure 45: First liftoff in Lab

The quadrotor appeared to be stable and in control and was therefore scheduled for an outdoor flight test.

4.4.2Outdoor Testing

The outdoor testing of the quadrotor was performed in an open tennis court, with the safety string of 5 meters tied so that the quadrotor doesn’t flies off and poses a threat to people nearby. The quadrotor flight was stable showing that the default PID values were fine and did not need and tuning. However when flying in Altitude Hold mode it was observed that quadrotor had difficulty maintaining the altitude, which might be attributed to fluctuations in barometer readings due varying atmospheric pressure and the air condition was windy. A log of the flight was recorded and plotted on the map to visualize the location and path of the quadrotor during the flight.


Figure 46: Flight path of the quadrotor during outdoors testing


5Conclusion

The project ‘Vision Based Quadrotor Navigation’ has been successful project in achieving its designated objective. The phase 1 of the project has been fully completed with its demonstration well recorded [14]. The quadrotor AR. Drone used with ROS to perform vision based object (tag) tracking flew for 40 seconds and maintained its position over a perpetually moving target in a randomized trajectory. This proves that the implemented communication architecture (Drone to ROS to Drone) can be used to perform vision based quadrotor navigation. However, there were occasional cases where AR. Drone lost its position over the target, especially when the target moved too quickly. This could be explained by two reasons:


  1. The inherent latency in the communication, which occurs due to time delay between capturing the frame and receiving the respective command to respond.

  2. The presence of Operating System, Virtual Box and ROS all of which are non real time systems and hence have unpredictable delays in processing time.


The phase 2 of the project has been partially completed. The state of the Ardupilot Mega quadrotor at the time of writing the report is ‘fully assembled and flying’. The progress can be considered a success till now, however a significant amount of work still remains to be done to make the quadrotor follow an object.


6Recommendations/Future Work

A substantial amount of work needs to be done in this area. Following is the list of the future prospects:


  1. Adding a First Person View (FPV) camera and video transmission module can be the future extension of the pending part of this project. This would further call for integration of the video receiver and MAVLink over ROS to process the video, determine the location of the quadrotor with respect to the target object and sending command back to the quadrotor over telemetry to follow the target.

  2. A different controller other than APM can be used to control the quadrotor. APM has limited computational power, which might cause problems for on board computationally intensive task, which might be incorporated later. A state of the art controller called PX4 PIXHAWK Autopilot has been developed by ETH Zurich, which can be explored further.

  3. An on board ARM based image processing computer can be incorporated on the quadrotor itself running real time operating system to perform a fully autonomous object tracking, obstacle avoidance and simultaneous localization and mapping (SLAM) with minimum possible latency.


References

[1] Domingues, J. M., “Quadrotor prototype”. Instituto Superior Técnico, Engenharia Mecânica, 2009.

[2] J.G. Leishman, “The Breguet-Richet Quad-Rotor Helicopter of 1907”, Vertiflite, 47(3):1–4, 2001.

[3] http://en.wikipedia.org/wiki/Quadrotor, October 2013.

[4] http://ardrone2.parrot.com, October 2013.

[5] http://store.apple.com/sg/product/H8859PA/A/parrot-ar-drone-2-0, October 2013

[6] https://www.virtualbox.org/wiki/Downloads, October 2013.

[7] http://www.ubuntu.com/download/desktop, October 2013.

[8] http://wiki.ros.org/hydro/Installation/Ubuntu, October 2013.

[9] https://play.google.com/store/apps/details?id=com.parrot.freeflight, October 2013.

[10] http://en.wikipedia.org/wiki/Robot_Operating_System, October 2013.

[11] http://wiki.ros.org, October 2013.

[12] https://github.com/AutonomyLab/ardrone_autonomy, October 2013.

[13] http://robohub.org/up-and-flying-with-the-ar-drone-and-ros-getting-started, October 2013.

[14] http://www.youtube.com/watch?v=DbNwq4RQiYg, October 2013.

[15] http://ardupilot.com/downloads/?did=82, October 2013.

[16] http://copter.ardupilot.com/wiki/vibration-damping, October 2013.

[17] http://qgroundcontrol.org/mavlink/waypoint_protocol, October 2013.

[18] https://pixhawk.ethz.ch/px4/en/start, October 2013.

Appendix

Appendix 1: Safety Information for Quadrotors

! Safety - Read Before Flying !
Your first priority must absolutely be the safety of people!

Multicopters are essentially Flying Lawnmowers!

o Get cheaper more flexible and more breakable plastic ones.

o Some carbon fiber props cut better than a Ginsu and while they are almost indestructible, you are not.

o Circumstances will require that you will need to make your own determination of what is a "safe distance" from people and property.

o At a minimum, consider: at least 10ft (3m) but not further than 30ft (10m) from you.
o Keep all other people, property and obstacles considerably further away from your copter.
o Ensure that no one gets between you and your copter.
o Spectators should always be a safe distance behind the pilot.
o If people intrude beyond what you have determined to be the "safe" area, land immediately and do not

take off until they are clear.
o At full power, an average sized multi-copter can exceed 20 mph (32 km/h), can ascend to hundreds of feet

and easily travel more than a mile in distance before running out of battery.

Always ensure that the battery cable is (NOT) connected to the power distribution board or harness until you are ready to fly.

o After landing the first thing you should do is disconnect your battery cable. o Always remove your props while you are testing motors.
o Your hands, arms and face and those of your friends will thank you.
o When the battery is connected, always assume the motors are armed.

o Check with a short throttle pulse.
o Don't pick up the model and the radio at the same time, you may bump the throttle. o Do not attempt to fly longer than your batteries safe capacity.

􏰺At a minimum it is very hard on the battery and at worst you will crash destructively. The APM and PX4 flight controllers we use incorporate a motor arming safety feature.

o Immediately prior to flight after the battery has been connected, the RC transmitters throttle stick needs to be held down and to the right for several seconds to arm the motors.

o After landing your first response should be to hold the throttle down and to the left for several seconds to "Disarm" the motors.

o Disarm condition can be tested by moving the throttle stick up, if the motors do not move it is dis armed.

o Even when disarmed, the throttle stick should always be kept in the full down position except when flying. Get used to switching back to Stabilize mode from other modes and reassuming full manual control.

o This is the single most important recovery technique (practice it).
o Stabilize mode can have Simple mode added to it, but if you do, practice with it till you are proficient.
o Do not use any modes other than Stabilize or Stabilize plus Simple until you are VERY comfortable flying.

Important primary response to a crash, inadequate landing or unknown flight controller state.
o The first thing to do is throw a towel over your copters propellers (Propellers may start spinning

unexpectedly).
o Then immediately disconnect the battery.
o A large towel is your most important piece of safety equipment followed by a fire extinguisher and a first

aid kit. - - Generally better to use the first one than the last one.

When testing or flying any of the navigation modes (using GPS):
o Ensure that your GPS has "Lock" before arming and takeoff.
o Check that your home position on the Mission Planner is in fact correct.

o Sometimes GPS's do not report the home position accurately, reboot if not accurate and wait for 8 or more satellites (not just 3D lock) and check again.

Keep a safe distance between your Copter and People!

These tips can also help protect your multicopter from damage. Avoid sudden or extreme transmitter control stick deflections

o Move the control sticks in small measured increments and don't "yank" on them.
o If the copter is properly calibrated and balanced it should require only small stick inputs to control altitude,

direction and speed.
o Your copter should be more or less stable on the horizontal plane without any control inputs.
o If you are "fighting" the copter, land and fix it - something is not right - Hardware adjustment or software

calibration may be required.
o Be especially careful of large throttle inputs, as a copter can gain (or lose) altitude very rapidly.

Because MultiCopters are symmetrical it is especially easy to lose Visual Orientation.
o For manual flight modes, maintaining a clear vision of the Copters Orientation (direction it is facing) is the

most critical part of successful flight.
o Especially while learning it is very important to keep your copter appropriately close to you to aid in

maintaining visual orientation.
o Generally: more than 10ft (3m) but not further than 30ft (10m) from you.
o If the copter gets further than about 100ft (30m) it starts getting difficult to be able to

maintain orientation and can easily crash.
o If you lose Yaw orientation while flying in Stabilize mode, try only flying forward and using yaw to steer like

a car.
o It is much better to simply descend and land rather than have an orientation-induced crash or fly away.

􏰺Fly-Aways often happen when the copter is commanded to tilt back towards the pilot but has rotated in the meantime and is so far away that orientation is lost.

􏰺Result: the copter flies further away and crashes or is lost.

o High winds can prevent forward progress or spin the copter around causing you to become disoriented. o The higher you are, the more likely high winds will be a problem.
o Switching to Stabilize mode and landing before you reach your skill limits can help you save your copter.

o Gusting winds around objects can also worsen the problem and radio signal loss can also occur.
o If your copter is approaching a potentially interfering object, immediately switch to stabilize mode and land

or retrieve the copter to your location.

ArduPilot specific safety modes: RTL, FailSafe and GeoFence.
o RTL can provide a safe Return to Launch if it starts to get away from you.
o FailSafe and GeoFence can assist you with keeping the copter in a safe proximity.
o Do not rely on the above safety modes, always be ready to take control in stabilize and land the copter. o Especially do not rely on the above safety modes to perform maneuvers or training that you would

otherwise consider dangerous.
o These modes are a supplement to, not a replacement for sound safety practices.

On your first takeoff after tuning or hardware setup
o In stabilize mode advance the throttle very slowly until the copter is almost hovering.
o If the copter is trying to flip over turn it off and correct the problem, a motor could be turning the wrong

direction, or a wrong direction prop could be installed.
o If the copter tries to rotate on it's axis or fly off in some direction the transmitter or RC setup may be

incorrect, a motor or ESC may not be performing properly or the props may be installed incorrectly.
o When all problems are fixed it should be fairly easy to get the copter to hover a foot or 2 above the ground. o If a stable and stationary hover a foot or 2 above the ground cannot be achieved, land and fix the problem

When flying FPV "First Person View" (with a video camera), Set modes to: STABILIZE, SIMPLE, and RTL. o Ensure RTL is working properly before using FPV,
o Use Stabilize mode to fly FPV and If you lose FPV video switch to Simple or RTL to get back.

Make sure your battery can't fall out, use a Velcro Strap to hold it in place.