Land Cover Classification with High Spatial Resolution Using Orthoimage and DSM Based on Fixed-Wing UAV

https://doi.org/10.7848/ksgpc.2017.35.1.1

Land Cover Classification with High Spatial Resolution Using Orthoimage and DSM Based on Fixed-Wing UAV

Kim, Gu Hyeok

ㆍChoi, Jae Wan

Abstract

An UAV (Unmanned Aerial Vehicle) is a flight system that is designed to conduct missions without a pilot.

Compared to traditional airborne-based photogrammetry, UAV-based photogrammetry is inexpensive and can obtain high-spatial resolution data quickly. In this study, we aimed to classify the land cover using high-spatial resolution images obtained using a UAV. An RGB camera was used to obtain high-spatial resolution orthoimage.

For accurate classification, multispectral image about same areas were obtained using a multispectral sensor. A DSM (Digital Surface Model) and a modified NDVI (Normalized Difference Vegetation Index) were generated using images obtained using the RGB camera and multispectral sensor. Pixel-based classification was performed for twelve classes by using the RF (Random Forest) method. The classification accuracy was evaluated based on the error matrix, and it was confirmed that the proposed method effectively classified the area compared to supervised classification using only the RGB image.

Keywords: UAV, Multispectral, DSM, Modified NDVI, RF

Original article

Received 2017. 01. 26, Revised 2017. 02. 02, Accepted 2017. 02. 17

1) Member, Dept. of Civil Engineering, Chungbuk National University (Email: [email protected])

2) Corresponding Author, Member, Dept. of Civil Engineering, Chungbuk National University (E-mail: [email protected])

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://

1. Introduction

The classification of land cover is used to map the homogeneous characteristics of a region by classifying its features according to a certain criterion. These data are used in policy formulation and environmental analyses to provide a spatial understanding of the usage status of a land area. Data on the specific time and area for terrain monitoring, such as land cover mapping and change detection, can generally be obtained through satellite sensors and aerial photogrammetry.

However, because the acquisition times of satellite and aerial images are limited, spatial information about a user’s desired area is difficult to acquire quickly. Compared to conventional aerial photogrammetry, UAV (Unmanned Aerial Vehicle) can acquire images of a desired region quickly. Therefore, they are being used increasingly in various fields related to

spatial information, including surveying and remote sensing.

Several studies have conducted quantitative evaluations of the

accuracy of orthoimages and DSM (Digital Surface Model)

acquired through UAV-based image processing as well as of

the laws and systems applied in the field of surveying. Kim

et al. (2014) studied the application of UAVs in the field of

land monitoring. Yun and Lee (2014) analyzed regulations

and trends in the operation of UAVs. Recently, studies related

to applications of spatial data have been conducted to use

UAVs. To analyze damage to waterside structures, Rhee et

al. (2015) acquired three-dimensional data about waterside

structures using a UAV and analyzed their positional

accuracy. Lee et al. (2015) analyzed the accuracy of DEM

(Digital Elevation Model) and orthoimage using a UAV; they

studied the quality of the results in the lateral, longitudinal,

and cross directions. Park and Kim (2016) generated a DSM

and an orthoimage with 3-cm-spatial resolution, which were obtained using a UAV, and they evaluated their applicability through virtual reference station surveying. Feng et al. (2015) proposed a hybrid method that combined the RF (Random Forest) classification technique and textural information in complex areas, including vegetation; their results showed that textural information improved the classification accuracy. As mentioned above, most studies on UAVs have been related to the generation and evaluation of orthoimages and DSMs by using aerial photos obtained by the UAV.

However, in studies on UAV-based photogrammetry, only the orthoimage obtained using an RGB camera has been used for image classification, despite the fact that it is possible to generate a thematic map by using the spectral information of orthoimages and spatial information of DSMs. For example, Debes et al. (2014) proposed an algorithm for generating thematic maps based on a parallel process that combined supervised classification and a graph-based fusion approach to integrate spectral and spatial information through aerial images and DSMs. In addition, previous studies classified only a few classes using UAV-based images with high spatial resolution. In this study, we propose a method that effectively classifies land cover by using DSMs and orthoimage generated by the point cloud extracted from an RGB image obtained using a UAV. Especially, multispectral image was used to improve the classification accuracy of the vegetation area, and the RF classification technique, a typical

machine-learning technique, was also used. In addition, to evaluate how to effectively classify various small objects by using UAV-based orthoimages with high-spatial resolution, supervised classification was conducted on twelve classes including cars and pavements of various colors. Finally, the results were compared with the classification results obtained using orthoimage alone.

2. Study Site and Data

The study area was at Chungbuk National University, which is located in Cheongju City in Chungbuk. This area consists of various land cover types in urban areas, such as buildings, artificial turf, soil, cars, grassland, and trees.

The eBee UAV manufactured by senseFly was used in the experiment (SenseFly, 2017). The eBee is a representative fixed-wing UAV that can be used for surveying purposes. It includes a GNSS (Global Navigation Satellite System) and an IMU (Inertial Measurement Unit). Table 1 shows the detailed specifications of the system. The eBee system can be equipped with two types of cameras: RGB camera and multispectral sensor. In this study, an RGB camera is used for the orthoimage and DSMs, and the NIR (Near Infra-Red) band of the multispectral sensor is used for the vegetation index. Table 2 shows the specifications of the RGB camera and multispectral sensor.

Table 1. Specifications of eBee

UAV Wingspan Weight Maximum

flight time Radio link

range Mounting

sensor Characteristic

Sensefly eBee 96cm < 0.69kg 50 min < 3km GNSS, IMU,

camera

Automatic flight according to

flight plan

Table 2. Characteristics of RGB camera and multispectral sensor in eBee

Sensor Bands Resolution Weight

Canon IXUS127HS Band 1 (Blue)

Band 2 (Green)

Band 3 (Red) 16 megapixel ~ 135g

multiSPEC 4C

Band 1 (Green, 530-570 ㎚) Band 2 (Red, 640-680 ㎚) Band 3 (Red edge, 730-740 ㎚)

Band 4 (NIR, 770-810 ㎚)

1.2 megapixel ~ 160g

and multispectral images. We aimed to generate a land cover map by using the RF classifier based on spatial and spectral features. Fig. 2 shows the workflow.

3.1 Pre-processing for generation of orthoimage and DSM

Most cameras that are mounted on UAVs for surveying purposes are non-surveying cameras. In these cameras, the greater the distance from the center of the image, the greater is the distortion of the image. We aimed to remove distortions in the radial and tangential directions in the image by using the interior orientation parameters provided by the eBee system. For example, the images with lens distortion shown in Fig. 3(a) were corrected as shown in Fig. 3(b).

Based on the exterior orientation parameters obtained using the GNSS and IMU in the UAV, matching points were automatically extracted in the image, and lens distortion To acquire images using each sensor, UAV flights were

performed in cross directions on July 13 and 15, 2016. The captured RGB images had a spatial resolution of about 5 cm at a flight altitude of about 156 m. A total of 184 images were recorded in a flight area of around 0.4 ㎢ with 80% sidelap and 90% overlap. The multispectral images had a spatial resolution of about 15 cm at a flight altitude of about 148 m.

A total of 72 images were recorded in a flight area of around 0.51 ㎢ with 80% sidelap and 90% overlap. Fig. 1 and Table 3 respectively show the flight path and specifications of the flight plan.

3. Methodology

In this study, orthoimages, DSMs, and a NDVI (Normali- zed Difference Vegetation Index) were generated using RGB

Fig. 1. Flight area and path

Altitude RGB: 156 m

Multispctral: 148 m

Resolution RGB: 5 cm

Multispctral: 15 cm

Number of photos RGB: 184

Multispctral: 72 Total ground

coverage RGB: 0.4 ㎢

Multispctral: 0.51 ㎢

sidelap 80%

overlap 90%

Table 3. Specification of flight plan

Fig. 2. Workflow

Fig. 3. Example of lens distortion removal (a) Image before lens

distortion removal (b) Image after lens

distortion removal

was removed. Based on the matching points, the exterior orientation parameters were optimized, and the point cloud was created by densifying the matching points. The DSM was created by rasterizing the extracted point clouds. In rasterization, point clouds are interpolated using the IDW (Inverse Distance Weighting) method, which is advantageous for various objects such as buildings. In addition, noise filtering and surface filtering are used to minimize the noise information that can occur in the process of generating orthoimage and DSMs. Fig. 4 shows the orthoimage and DSM of the experimental data extracted in the flight area shown in Fig. 1. Orthoimages are generated by using images obtained from both an RGB camera and a multispectral sensor. In the case of an RGB camera, both RGB bands and DSMs with high-spatial resolution are used for land cover mapping (Figs.

4(a) and (c)). The multispectral sensor is only used to generate an NDVI based on an orthorectified multispectral image with low spatial resolution to improve the classification accuracy in the vegetated area (Fig. 4(b)). The size of Figs. 1(a)~(c) is 7,501×4,935 pixels.

3.2 NDVI extraction

Because the multispectral image and DSM obtained by a multispectral camera have low spatial resolution compared with the images obtained by RGB camera, the data obtained from the multispectral camera were intended to be used only for NDVI extraction. NDVI is one of the representative vegetation indices used in the field of remote sensing (Rouse et al., 1974). It is calculated by Eq. (1).

(1)

where

 _{ }

  

  





  _{ }

   

   

 

is the spectral reflectance of the NIR band and

 _{ }

  

  





  _{ }

   

   

 

is that of red band.

However, it is considered that the spectral reflectance information of the image obtained using the multispectral sensor may be relatively inaccurate. In this study, to obtain accurate NDVI images, the spectral reflectance of the ground material was measured using a spectroradiometer, and empirical line calibration was performed. Empirical linear correction is used to transform the spectral radiance or reflectance of the entire image into reflectance by using a linear form between the spectral radiance known in the image and the reflectance on the ground obtained by using the spectroradiometer (Chang et al., 2011). The NDVI was extracted using the reflectance information of the multispectral image that was finally generated. However, when the existing NDVI formula was used in the experimental area, high NDVI values occurred in some buildings and non- vegetated areas. For example, some blue-colored rooftops showed relatively moderate NDVI values similar to that of grassland in high-spatial resolution imagery (Figs. 5(a) and (c)). Therefore, a modified NDVI image was produced by using the red edge band, as given by Eq. (2) (Fig. 5(d))

(2)

where

 _{ }

  

  





  _{ }

   

   

  is the spectral reflectance of the red edge band.

Fig. 4. Orthoimage and DSM in the experimental area (a) Orthoimage by using RGB camera (b) Orthoimage by using multispectral

sensor (c) DSM by using RGB camera

 _{ }

  

  





  _{ }

   

   

 

 _{ }

  

  





  _{ }

   

   

 

3.3 nDSM generation

Our objective is to effectively classify artificial objects and vegetation with height values. In this study, a TIN (Triangulated Irregular Network) is generated by using the points corresponding to the ground (Fig. 6(a)). Ground points from the point clouds were extracted manually, because our experiments are not focused on the automation of DEM generation. The DEM is created by rasterizing the TIN.

The nDSM (normalized DSM) is generated using the difference between the DSM and the DEM (Fig. 6(b)) (Straub et al., 2008). In nDSM, objects with a ground point

have a value of zero, and artificial and tree objects have a value corresponding to the height from the ground. In this study, we aim to classify ground and non-ground objects into separate clusters.

3.4 Image classification by RF

To perform UAV-based image classification, we apply the RF technique, which is the most widely used machine- learning technique in the fields of image classification and statistics. The RF classifier is based on a decision tree that consists of nodes and edges. The results of the decision tree vary widely depending on the data that are learned, which makes it difficult to generalize error propagation. This is because this technique divides the decision tree into a hierarchical structure. Because the RF classifier is a method that arbitrarily learns a large number of decision trees by using several of them, it can overcome and generalize the disadvantages of the traditional decision tree classifier (Belgiu and Dr gu , 2016). The experiment consists of two steps. First, non-ground objects such as buildings, cars, and trees based on nDSM are classified using the RF. Then, the RF technique is applied to the remaining ground classes, such as asphalt, pavement, and artificial turf. The classification results are combined to generate the final land cover map.

Several image features, RGB bands, nDSM, and modified NDVI images are used in the RF classifier.

4. Experimental Results and Discussion

4.1 Training and ground truth data

In this study, the experimental method was divided into four cases (Fig. 7). Case 1 was classified by using only RGB images, and cases 2~4 were classified by adding the nDSM and modified NDVI images to case 1. To confirm the possibility of classifying vegetation, we divided trees, grassland, and artificial turf into classes. We also divided buildings, automobiles, and various packaging materials into classes to confirm the possibility of using high-resolution images. Training data were obtained manually in the experimental area, and reference data were constructed to evaluate the classification results. The training data in the experimental area contained 9692 pixels and the reference Fig. 6. DEM and nDSM of experimental area

(a) TIN model based on ground points for generating

DEM

(b) nDSM Fig. 5. Example of NDVI and modified NDVI image (a) True color composit using

RGB image (b) False color composite using multispectral image

(c) NDVI image (d) Modified NDVI image

data, 13,365 pixels (Table 4). The training and reference data were extracted independently and manually so as to not overlap each other. The location of the training data is shown in Fig. 8.

4.2 Experimental results

Fig. 9 shows the classification results for each case. In case 1, many areas with ground points were classified as buildings or cars. In addition, some vegetated areas were misclassified.

However, the classification result based on case 2 minimizes omission and commission errors in the vegetated area. It indicated that the modified NDVI image can improve the classification performance for the vegetated area. In case 3, the problem of case 1 was improved by dividing ground and non- ground points. However, some vegetated areas were classified as non-vegetated areas, and vice versa. In case 4, in which the modified NDVI and nDSM was added, the misclassification that occurred in cases 1~3 tended to decrease. Therefore, the results indicate that the proposed method was effective in classifying high-resolution images that are acquired using a UAV. To evaluate the accuracy of the classification results, the total accuracy and kappa coefficient were compared and evaluated for each experimental method by using the error matrix of the training and reference data. As shown in Tables 5~8, the classification accuracy was the highest in case 4 (overall accuracy = 95.69%, kappa = 0.951). In case 1, the results of the classification of buildings, trees, grassland, and cars were largely erroneous. In case 2, the classification result of the vegetated areas was improved, but other classes included many errors. In case 3, in which the nDSM image was used, the results of the classification of buildings, trees, and cars were improved.

In case 4, in which the modified NDVI and nDSM images were added, the classification accuracy of most classes was improved.

Fig. 7. Experimental methods

Table 4. Types of training data for classification No. Class name Training data

(pixels) Reference data (pixels)

1 artificial turf 138 530

2 asphalt 1031 1860

3 concrete 353 879

4 grassland 708 1017

5 pavement (green) 472 908

6 pavement (red) 338 651

7 pavement (white) 163 426

8 pavement (yellow) 130 198

9 bare soil 219 1714

10 building 2632 1864

11 car 698 949

12 tree 2810 2369

Total 9692 13365

concrete asphalt grassland bare soil pavement(white) pavement(green)

pavement(red) pavement(yellow) artificial turf building tree car

Fig. 8. The location of the training data

Fig. 9. Experimental results (a) Orthoimage (b) Classification result of case 1

(overall accuracy : 84.77%, kappa coefficient: 0.829)

(b) Classification result of case 2 (overall accuracy : 88.44%, kappa

coefficient: 0.870)

(c) Classification result of case 3 (overall accuracy : 91.01%, kappa

coefficient: 0.899)

(d) Classification result of case 4 (overall accuracy : 95.96%, kappa

coefficient: 0.951)

ground truth data concrete asphalt grass

land bare soil pavement (white) pavement (green) pavement

(red) pavement (yellow) artificial

turf building tree car

cla ssi fie d i m ag e

concrete 651 0 0 0 0 0 0 0 0 0 0 0

asphalt 0 589 0 18 0 0 0 0 42 0 0 0

grassland 0 0 1722 0 0 77 0 118 0 0 0 0

bare soil 0 60 0 180 0 0 0 0 0 0 0 0

pavement

(white) 0 0 0 0 460 0 0 0 0 0 0 0

pavement

(green) 0 1065 138 0 0 761 19 0 7 0 0 0

pavement

(red) 0 0 0 0 70 41 998 0 0 0 0 0

pavement

(yellow) 0 0 0 0 0 0 0 790 0 0 0 0

artificial turf 0 0 0 0 0 0 0 0 377 0 0 0

building 0 0 0 0 0 0 0 0 0 1826 11 160

tree 0 0 0 0 0 0 0 0 0 0 2309 123

car 0 0 0 0 0 0 0 0 0 38 49 666

overall accuracy(%) 84.77 kappa coefficient 0.829

Table 5. Confusion matrix about classification result of case 1

ground truth data concrete asphalt grass

land bare soil pavement (white) pavement (green) pavement

(red) pavement (yellow) artificial

turf building tree car

cla ssi fie d i m ag e

concrete 803 253 0 141 11 0 0 4 0 0 0 0

asphalt 4 1572 0 0 0 241 0 0 0 0 0 0

grassland 0 0 1017 0 2 0 0 0 0 0 0 0

bare soil 63 0 0 1573 55 0 0 28 0 0 0 0

pavement

(white) 0 0 0 0 358 0 0 0 0 0 0 0

pavement

(green) 0 35 0 0 0 667 0 0 0 0 0 0

pavement

(red) 0 0 0 0 0 0 651 0 0 0 0 0

pavement

(yellow) 0 0 0 0 0 0 0 166 0 0 0 0

artificial turf 9 0 0 0 0 0 0 0 530 0 0 0

building 0 0 0 0 0 0 0 0 0 1248 0 72

tree 0 0 0 0 0 0 0 0 0 0 2369 11

car 0 0 0 0 0 0 0 0 0 616 0 866

overall accuracy(%) 88.44 kappa coefficient 0.870

Table 6. Confusion matrix about classification result of case 2

ground truth data concrete asphalt grass

land bare soil pavement (white) pavement (green) pavement

(red) pavement (yellow) artificial

turf building tree car

cla ssi fie d i m ag e

concrete 784 65 4 684 9 0 0 6 0 0 0 0

asphalt 57 1788 0 0 0 121 0 0 0 0 0 0

grassland 38 7 901 0 0 0 0 0 42 0 0 0

bare soil 0 0 0 1030 2 0 0 12 0 0 0 0

pavement

(white) 0 0 0 0 415 0 0 0 0 0 0 0

pavement

(green) 0 0 0 0 0 787 0 0 0 0 0 0

pavement

(red) 0 0 0 0 0 0 651 0 0 0 0 0

pavement

(yellow) 0 0 0 0 0 0 0 180 0 0 0 0

artificial turf 0 0 0 0 0 0 0 0 488 0 0 0

building 0 0 0 0 0 0 0 0 0 1864 31 0

tree 0 0 112 0 0 0 0 0 0 0 2327 0

car 0 0 0 0 0 0 0 0 0 0 11 949

overall accuracy(%) 91.01 kappa coefficient 0.899

Table 7. Confusion matrix about classification result of case 3

5. Conclusion

In this study, land cover classification was performed using RGB images, modified NDVI obtained using multispectral images, and nDSM images obtained using a UAV. The experimental results showed that the proposed method effectively classified various terrain features. In particular, the addition of nDSM as input data increased the classification accuracy of buildings, trees, and cars. These results demonstrated that modified NDVI images could be used effectively to classify grass and trees. Based on the results, we conclude that land cover maps can be generated from high-resolution images obtained using UAVs. In future work, we will conduct experiments with various UAV-based datasets, such as large regions and complex urban areas. In addition, we will modify our proposed algorithm to improve the classification accuracy, for which an automatic DEM generation algorithm should be required.

Acknowledgment

This work was supported in the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning under Grants NRF-2013R1A1A1060343.

References

Belgiu, M. and Drăguţ, L. (2016), Random forest in remote sensing: A review of applications and future directions, ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 114, pp. 24-31.

Chang, A.J., Kim, Y.I., Choi, S.K., Han, D.Y., Choi, J.W., Kim, Y.M., Han, Y.K., Park, H.L., Wang, B., and Lim, H.C. (2011), Construction and data analysis of test-bed by hyperspectral airborne remote sensing, Korean Journal of Remote Sensing, Vol. 29, No. 2, pp. 161-172. (in Korean with English abstract)

Table 8. Confusion matrix about classification result of case 4 ground truth data

concrete asphalt grass

land bare soil pavement (white) pavement (green) pavement

(red) pavement (yellow) artificial

turf building tree car

cla ssi fie d i m ag e

concrete 833 128 0 98 12 0 0 15 0 0 0 0

asphalt 29 1732 0 0 0 156 0 0 0 0 0 0

grassland 12 0 905 0 0 0 0 0 0 0 0 0

bare soil 0 0 0 1616 0 0 0 8 0 0 0 0

pavement

(white) 0 0 0 0 414 0 0 0 0 0 0 0

pavement

(green) 0 0 0 0 0 752 0 0 0 0 0 0

pavement

(red) 0 0 0 0 0 0 651 0 0 0 0 0

pavement

(yellow) 0 0 0 0 0 0 0 175 0 0 0 0

artificial turf 5 0 0 0 0 0 0 0 530 0 0 0

building 0 0 0 0 0 0 0 0 0 1864 0 0

tree 0 0 112 0 0 0 0 0 0 0 2369 1

car 0 0 0 0 0 0 0 0 0 0 0 948

overall accuracy(%) 95.69 kappa coefficient 0.951

Debes, C., Hahn, J., Liao, W., Gautama, S., and Du, Q. (2014), Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 7, No. 6, pp. 2405-2418.

Feng, Q., Liu, J., and Gong, J. (2015), UAV remote sensing for urban vegetation mapping using random forest and texture analysis, Remote Sensing, Vol. 7, No. 1, pp. 1074-1094.

Kim, D.I., Song, Y.S., Kim, G.H., and Kim, C.W. (2014), A study on the application of UAV for Korean land monitoring, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 32, No.

1, pp. 29-38. (in Korean with English abstract)

Lee, G.S., Choi, Y.W., Jung K.S., and Cho G.S. (2015), Analysis of the spatial information accuracy according to photographing direction of fixed wing UAV, Journal of the Korean Cadastre Information Association, Vol. 17, No. 3, pp. 141-149. (in Korean with English abstract)

Park, J.K. and Kim, M.G. (2016), Applicability verification of rotary wing UAV for rapid construction of geospatial information, Asia-pacific Journal of Multimedia services convergent with Art, Humanities, and Sociology, Vol. 6, No. 4, pp. 73-80. (in Korean with English abstract) Rhee, S.A., Kim, T.J., Kim, J.E., Kim, M.C., and Chang,

H.J. (2015), DSM generation and accuracy analysis from UAV images on river-side facilities, Korean Journal of Remote Sensing, Vol. 31, No. 4, pp. 183-191. (in Korean with English abstract)

Rouse, J.W., Haas, R.H., Schell, J.A., and Deering, D.W.

(1974), Monitoring vegetation systems in the great plains with ERTS, Proceedings of the third Earth Resource Technology Satellite (ERTS) Symposium, NASA, 10-14 December, Washington, D.C, USA, Vol. 1, pp. 309-317.

SenseFly. (2017), Drones for professionals, mapping &

photogrammetry, flight planning & control software, senseFly SA, Switzerland, https://www.sensefly.com (last date accessed: 25 January 2017).

Straub, C., Weinacker, H., and Koch, B. (2008), A fully automated procedure for delineation and classification of forest and non-forest vegetation based on full waveform laser scanner data, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information

Sciences, ISPRS, 3-11 July, Beijing, China, Vol. 37, pp.

1013-1019.

Yun, B.Y. and Lee, J.O. (2014), A study on application of

the UAV in Korea for integrated operation with spatial

information, Journal of the Korean Society for Geospatial

Information Science, Vol. 22, No. 2, pp. 3-9. (in Korean

with English abstract)