5.1 Introduction
The quad-rotor UAV is very useful to help people carry out the hard and dangerous works.
Their applications attracted significant interests of researchers [36,37]. Specially, the vision application was presented exciting, such as tracking a ground moving target [38], autonomous landing on a moving unmanned ground vehicle (UGV) [39], 3D mapping [40]. In these studies, the researchers used the realistic UAV for demonstrating their applications. The testing process may have high risk of property damage. To restrict the accidents and safety for human, the HILS is often performed for the UAVs system on the real time platform.
For some research associated with vision in HILS, the real camera is used to estimate the state of UAV based on the virtual image which is shown on the desktop monitor screen [41,42], projector screen [43,44]. Although the developed HILS systems showed some interesting results, their applicability is limited due to the cost and associated complicated components of the camera. Beside, the presented 3D information of scene is also the trouble of using the virtual image on the screen.
In another approach, the gazebo software [45] was utilized to simulate the 6DOF model of the UAV, virtual camera and virtual 3D environment workspace for the simulation. For example, Odelga el at. [46] combined the Odroid board with the gazebo to present the operation of the UAV. Therein, the hardware board is installed for the flight control algorithm and the vision control algorithm. In this case, the hardware system may overload because the operating system(OS) must perform many processes at the same time. And the Odroid board lacks the basic sensor for a UAV such as IMU sensor, GPS sensor, and etc. Therefore, the real UAV needs to equip these sensors to provide state of UAV for the flight controllers. Consequently, the system becomes complex, and needs expensive solution. As another solution, the pixhawk was
65 used to setup the flight control algorithm, which is integrated with the full sensor for a quad-rotor UAV [24]. In this study, the vision algorithms were performed on the a desktop computer which is difficult to apply to the real UAV because the weight of desktop computer is too heavy to integrate on the UAV.
This chapter a new configuration of a general hardware-in-the-loop-simulation (HILS) setup of Unmanned Aerial vehicle (UAV) especially for vision algorithm. In our setup, the gazebo software is used to simulate six-degree-of-freedom (6 DOF) model and corresponding sensor readings such as the inertial measurement unit (IMU)and the camera for a quad-rotor UAV. Meanwhile, the flight control algorithm is performed on the pixhawk hardware. The raspberry hardware is installed the vision algorithms to estimate the position of the quad-rotor UAV for the landing task. The middle software named control application software(CAS) is developed to collect the communication between the gazebo, pixhawk and raspberry by using the multithread architecture. Numerical implementation has been performed to prove effectiveness of the suggested HILS components approach.
5.2 Development of general HILS
The HILS is performed to test the effectiveness of the flight control algorithm for the UAV in the real-time platform. Based on the modeling of the quad-rotor UAV, the flight control is developed by using the PX4 open source, which is compiled and uploaded to the pixhawk board. The gazebo software is used to configure the dynamic model, the sensor model and the 3D visualization of the quad-rotor UAV. In the gazebo, the camera model is also defined to generate the camera data for the vision algorithm which is installed on the raspberry board. The CAS is used to collect the signals between the gazebo, pixhawk and raspberry components. The general HILS is proposed as shown in Fig. 5.1.
66 Fig. 5.1 Proposed HILS system for vision application
5.3 Software development
As previous chapter, the gazebo software can create the dynamic model, sensor model and 3D visualization for simulating the operation of the quad-rotor UAV. In gazebo, the measurement state and control UAV are configured in the plugin through the libraries such as physics (physics.hh), sensor (sensor.hh), camera(cameraSensor.hh). The camera sensor is built to provide the video data for the vision algorithms. For example, the code in the list below describes for camera and control plugin.
//Camera
CameraPlugin::Load(_parent, _sdf);
this->width_ = this->width;
this->height_ = this->height;
this->depth_ = this->depth;
for (int i=0; i<_fov ; ++i) {
int index = startingPix + i*_width;
for (int j=0; j<_fov ; ++j) illum += _image[index+j]; } msg.illuminance = illum/(_fov*_fov);
_sensorPublisher.publish(msg);
//Control
link_->AddForce(air_drag);
physics::Link_V parent_links = link_->GetParentJointsLinks();
math::Pose pose_difference = link_->GetWorldCoGPose() -
67 parent_links.at(0)->GetWorldCoGPose();
math::Vector3 drag_torque(0, 0, -turning_direction_ * force * moment_constant_);
math::Vector3 drag_torque_parent_frame =
pose_difference.rot.RotateVector(drag_torque);
parent_links.at(0)->AddRelativeTorque(drag_torque_parent_frame);
The parameter of the camera is given from real camera which is camera module for raspberry board. The camera and raspberry board are shown in Fig. 5.2.
Fig. 5.2 Camera module and Raspberry board
5.4 Hardware development
The pixhawk hardware board is used to perform the flight control algorithm for the quad- rotor UAV on the real time. The flight controller is developed based on thePX4 source, which is presented in Fig.5.3.
, ,
r
,
rX Y
, , X Y Z
r, r
Z
Fig. 5.3 Schema trackingflight controller of the quad-rotor UAV
68 To track a desired trajectory, the position control and attitude control based on PID control techniques are applied. The output of attitude controller is sent to the inverse function(IF) (obtain from Eq. (2.3)) to calculate the angular velocity for each motor. The feedback signal for each controller is provided by the IMU sensor model in the gazebo [75,76]. All controllers are written by C program which is compiled to make a firmware and then uploading to microchip of the pixhawk hardware.
The raspberry board is adopted to perform the vision algorithm in our HILS development.
Being installed with the Linux(Ubuntu Mate 16.04LTS), this board can run the C program which allows access and analysis data from a camera source. Moreover, it provides the input/output ports such as the ethernet, USB and serial for easy communicate with other peripheral devices especially for the camera. A standalone system of the raspberry board is configured as in Fig. 5.4.
Fig. 5.4 Standalone system of the raspberry board
In our HILS setup, the camera signal is generated by the gazebo simulation software, which is used for the input signal of the raspberry board in order to test the vision algorithm.
69 Fig. 5.5 Full HILS setup
5.5 Implementation of HILS and results
In this section, the simulation has been carried out to demonstrate the effectiveness of our HILS setup which is constructed as in Fig. 5.5. The capability of HILS is appreciated in two flight scenarios such as the position control, and the autonomous landing using a vision algorithm. At initial time, the quad-rotor UAV is configured as in Fig. 5.6.
Fig. 5.6 Workspace of the quad-rotor UAV in gazebo software
70 First, he quad-rotor UAV is tested with the desired trajectory
X Y Zr, r, r,r
which is defined from a program in the raspberry board with the desired values of
Xr 0,Yr 0,Zr 2.5,r0
for the takeoff,
Xr 8,Yr 5,Zr 6,r 0
for moving to a position and
Xr 8,Yr 5,Zr 0,r 0
for landing. At the first time, the quad-rotor UAV performs takeoff to 2.5[m] and keeps waiting the desired signal from Raspberry. As result in Fig.5.7, the pixhawk receives the desired trajectory signal at time 0.4[min], 1.5[min] and 3.2[min] to change position for the quad-rotor UAV. Herein, the UAV can move to the desired position with a small error. This test shows that the combination among the gazebo, pixhawk and raspberry worked well. Specially, the flight controllers in pixhawk can ensure tracking position for the quad-rotor UAV. Other parts such as gazebo and raspberry can perform correctly the tasks in HILS setup.
0 1 2 3 4 5
0 3 6 9
Time (min)
Position (m)
Desired X Desired Y Desired Z X Y Z
Fig. 5.7 Responses from the quad-rotor UAV
Thirdly, to evaluate the effectiveness of the raspberry board, the vision algorithm is installed on this board to detect a marker pad [77]. In this test, a virtual camera in the gazebo and a real camera are connected to the raspberry board via the CAS and USB connection support, respectively. As the results in Fig. 5.8, in both simulation and real experimental cases, the vision algorithm installed on the raspberry board can recognize the marker pad with high accuracy(red rectangle). And it can apply to the quad-rotor UAV to generate the position for the landing process.
71
(a)Virtual camera (b)Real camera
Fig. 5.8 Raspberry board recognizes the marker pad by using the vision algorithm
Next, the vision algorithm is used to estimate the position for the autonomous landing of the quad-rotor UAV by using the marker pad [77]. An ArUco marker is used for the landing pad, which is binary square fiducial markers. It can provide enough correspondences to obtain the camera pose by four corners. The inner binary codification ensures the robust, allows the possibility to apply the error detection and the correction techniques. The pose estimation of the ArUco marker was developed in the OpenCV libraries [78].
Based on the OpenCV and the landing pad ArUco marker, the flight task is performed in two steps. Step 1, the quad-rotor UAV is takeoff to
Xr 0,Yr 0,Zr 2.5
. Step 2, the vision algorithm is run to detect the landing marker and then generate desired position
X Y Zr, r, r
for the controller in the pixhawk. As the results in Figs. (5.9)(5.10), the quad-rotor UAV can land to the marker pad with the small error. All results showed that the our HILS could provide well simulation for testing the flight control algorithm and vision algorithm of the quad-rotor UAV based on the pixhawk, gazebo, raspberry and CAS.5.6 Chapter summary
This chapter presents HILS setup to simulate the flight operation of the quad-rotor UAV by using vision. The proposed HILS could work well based on the pixhawk, gazebo, raspberry and
72 CAS. Herein, the pixhawk and raspberry provide the real time hardware to perform the flight control and vision algorithm. The gazebo is used to present the dynamic model, sensor model and 3D visualization for the quad-rotor UAV. Meanwhile, the CAS is developed based on the multithread architecture to ensure the communication between the parts of HILS. As a result, our proposed techniques could establish the HILS better performed in a sense of high speed communication and camera application for the quad-rotor UAV.
0.0 0.5 1.0 1.5 2.0
0 1 2 3
Time (min)
Position (m)
Desired X Desired Y Desired Z X Y Z
Fig. 5.9 Responses from the quad-rotor UAV in the landing task
(a) Initial position (b) Takeoff and detect the marker pad
(c) Camera view from Raspberry board (d) Finished landing Fig. 5.10 Quad-rotor UAV landing to the marker pad
73