Comparison of Orthophotos and 3D Models Generated by UAV-Based Oblique Images Taken in Various Angles

https://doi.org/10.7848/ksgpc.2018.36.3.117

Comparison of Orthophotos and 3D Models Generated by UAV- Based Oblique Images Taken in Various Angles

Lee, Ki Rim

ㆍHan, You Kyung

ㆍLee, Won Hee

Abstract

Due to intelligent transport systems, location-based applications, and augmented reality, demand for image maps and 3D (Three-Dimensional) maps is increasing. As a result, data acquisition using UAV (Unmanned Aerial Vehicles) has flourished in recent years. However, even though orthophoto map production and research using UAVs are flourishing, few studies on 3D modeling have been conducted. In this study, orthophoto and 3D modeling research was performed using various angle images acquired by a UAV. For orthophotos, accuracy was evaluated using a GPS (Global Positioning System) survey that employed VRS (Virtual Reference Station) acquired checkpoints. 3D modeling was evaluated by calculating the RMSE (Root Mean Square Error) of the difference between the outline height values of buildings obtained from the GPS survey to the corresponding 3D modeling height values. The orthophotos satisfied the acceptable accuracy of NGII (National Geographic Information Institute) for a 1/500 scale map from all angles. In the case of 3D modeling, models based on images taken at 45 degrees revealed better accuracy of building outlines than models based on images taken at 30, 60, or 75 degrees. To summarize, it was shown that for orthophotos, the accuracy for 1/500 maps was satisfied at all angles; for 3D modeling, images taken at 45 degrees produced the most accurate models.

Keywords : Oblique Angle Images, UAV, Orthophoto, GCP, 3D Modeling

Original article

Received 2018.04.23, Revised 2018. 05. 03, Accepted 2018. 06. 07

1) School of Geospatial Information, Kyungpook National University (E-mail: [email protected])

2) Member, School of Convergence & Fusion System Engineering, Kyungpook National University (E-mail: [email protected])

3) Corresponding Author, Member, School of Convergence & Fusion System Engineering, Kyungpook National University (E-mail: [email protected]) This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://

1. Introduction

Recently, with the development of the UAV (Unmanned Aerial Vehicle), research and production of high resolution orthophotos are increasing. UAVs are expected to replace aerial photographs and satellite images for producing spatial information updates for small areas. In addition, national policy is being implemented through 3D (Three- Dimensional) map creation using UAVs, and research activity regarding 3D maps is also increasing. Research using UAVs includes use of a UAV by Watanabe and Kawahara (2016) to study river topography and vegetation changes of the Jyoge River in Hiroshima, Japan. Lin et al. (2015) conducted a study on tree detection using oblique images. Also, Uysal et

al. (2015) analyzed the generation and accuracy of a digital elevation model through UAV imagery at Sahitler Hill in Turkey, and Eker et al. (2018) conducted a case study on the Gallenzerkogel landslide through UAV-based landslide monitoring. Han and Hong (2017) obtained sub-meter ground sample distance images using a UAV and analyzed the images visual resolution.

In a study on 3D modeling and building texturing using

UAVs, Lee and Lee (2017) compared 3D modeling with

orthophotos using vertical and high-oblique images taken

by UAV. Chiabrando et al. (2018) conducted a survey of

archeological sites via UAV surveys in the region of Phrygia

Hierapolis in Turkey. Rossi et al. (2017) generated mining

topography using UAV-photographed oblique images and

analyzed the feasibility thereof. Khaloo et al. (2018) generated 3D models of bridges using UAV-photographed bridges and dense structure from motion algorithms and compared them with laser scanning results.

In previous studies, only vertical images acquired by UAV were used for orthophoto production or research, and studies using oblique images are insufficient. Also, there is a lack of research on 3D modeling based on UAV surveying. In the case of Lee and Lee (2017), vertical and high-oblique images were added for orthophoto production and 3D modeling, but the various angles were arbitrarily determined. In this study, we used high-oblique-angle images that are not well used in existing UAV survey studies. Also, orthophoto production and 3D modeling were performed by acquiring images from various angles without setting any arbitrary angle images at high-oblique angles. We compared and analyzed whether orthophotos and 3D models can be generated from various angle images and which angle is superior for the generation.

2. Methodology

2.1 Study area and research methods

The study area was selected from the Sangju campus of Kyungpook National University located in Sangju-si, Gyeongsangbuk-do, Republic of Korea. In this study, high- oblique images from various angles over the study area were taken using an Inspire 1 rotary-wing UAV. Orthophotos, DSM (Digital Surface Model), and 3D models were produced using the images. We then assessed the accuracies of the orthophotos and 3D models. The study area is shown in Fig.

1. The workflow is summarized in Fig. 2.

2.2 Research equipment and acquisition of various angle images

The UAV and camera used in this study are the Inspire 1 and FC350, respectively. Inspire 1 is a rotary-wing UAV, and the FC350 uses a 94° wide-angle lens. Table 1 shows UAV and camera specifications.

Using the UAV, we acquired images from various angles from 12:00 am to 1:00 pm in 6 to 12th November, 2017. All images were obtained with the Pix4d capture application, with an altitude of 100 m, an overlap of 70 %, a sidelap of 50 % and flight speeds of 9 to 10 m/s. Fig. 3 shows the flight course documented by the Pix4d capture application, and Fig. 4 shows examples of single images acquired from various angles.

Fig. 1. Study area highlighted in red

Fig. 2. Workflow of orthophoto and 3D model production

using oblique images acquired by UAV in various angles

UAV Camera

Inspire 1 FC350

Weight 2935g Resolution 4,000 x 3,000 (4:3) Flight altitude Max: 4500 m Pixel size 1.561 x 1.561 μm

Flight time Max: 18 min FOV 94°

Speed Max: 22 m/s Focal length 3.61 mm

Maximum wind

resistance 10 m/s Max. aperture f/2.8

Table 1. Specifications of Inspire 1 and FC350

Fig. 3. Flight course over the study area.

Gray circles indicate location of image acquisition

(a) 30°

(c) 60°

(b) 45°

(d) 75°

Fig. 4. Single images acquired from various angles

2.3 Acquisition of GPS data

GCP (Ground Control Point) and checkpoint measurements were performed to assess the accuracies of orthophotos produced with imagery from various angles. To verify the accuracy of 3D models, a GPS (Global Positioning System) survey was performed on the outlines of the buildings.

Feature points on road lanes and parking lines substitued conventional GCP and checkpoints because they also are easily identified from the air. GPS survey was conducted using a GPS receiver in VRS (Virtual Reference Station) mode. GPS satellite navigation was used for the VRS survey;

the L1C/A, L1C, L2C and L5 signals were received. Table 2 lists the specifications of the GPS receiver.

The VRS survey was conducted in compliance with the RTK (Real Time Kinematics) survey regulations. The number of satellites used was 10–17, and the horizontal accuracy and vertical accuracy were 0.007 m and 0.013 m, respectively;

this satisfied the network RTK measurement regulations of the Work Provision for Public Survey (Republic of Korea) No. 2017-1323. 18 GCPs and checkpoints were obtained from the VRS survey; 7 GCPs and 11 checkpoints were used.

GCPs and checkpoints were equally applied to each image taken in different angles. Fig. 5 indicates the GCPs and checkpoints used. The red circles represent the GCPs, and the yellow circles represent the checkpoints. GPS surveying for comparison of 3D modeling was also done using VRS, and was performed on the outline of the building in accordance with the measurement regulations as above.

3. Image Processing

3.1 Camera lens distortion correction

Lens distortion correction was performed before image matching, using Photoscan software. The model used for lens distortion correction was Brown's Distortion Model, with the following Eq (1):

(1)

,

X, Y, Z, : Point coordinates in the local camera coordinate system,

: Projected point coordinates in the image coordinate system (in pixels),

: Focal length,

: Principal point offset,

K

, K

, K

, K

, : Radial distortion coefficients, P

, P

, P

, P

, : Tangential distortion coefficients,

B

, B

, : Affinity and non-orthogonality (skew) coefficients, : Image width and height in pixels.

Fig. 6 shows the residuals for images corresponding to various angles, and Table 3 shows the interior orientation parameters. The FC350 camera used in the Inspire 1 is a wide-angle camera with a viewing angle of 94°, which means that residuals are high near the edges of each image. The FC350 is a low-cost camera and its manufacturing process is not precise, which explains the large residuals.

Fig. 5. GCPs and checkpoints

Type Specifications

No. of channels 440 channels

Weight 3.81 kg

Dimensions 19cm * 10.4 cm

Satellite signals GPS: L1C/A, L1C, L2C, L2E, L5 VRS precision Horizontal: 8 mm + 0.5 ppm RMS

Vertical: 15 mm + 0.5 ppm RMS

Table 2. Trimble R8s specifications

from Motion) method. The SIFT method is suitable for extraction of feature points because it is invariant to changes in image contrast and scale (Wang et al., 2012; Han et al., 2012). The SfM method is used to process feature points extracted by the SIFT method (Verhoeven, 2011; Lucieer et al., 2014; Doneus et al., 2011). The SfM method connects feature points extracted by two adjacent images, estimates the relative posture and direction of the images using epipolar geometry, and estimates the 3D position of feature points as well. When this process is applied to all images, the estimated position, direction, and 3D position of the camera are optimized and a sparse point cloud is generated (Hartley, 1997). Fig. 7 shows feature points extracted through the SIFT method; the numbers of feature points for images taken at 30 degrees, 45 degrees, 60 degrees and 75 degrees are 10,776, 10,390, 10,926, and 10,498, respectively.

(a) 30° (b) 45°

(c) 60° (d) 75°

(e) vertical

Fig. 6. Image residuals corresponding to various angles

(a) 30° (b) 45°

(c) 60° (d) 75°

(e) Vertical

Fig. 7. Feature points corresponding to various angles 3.2 Image matching

Image matching for the acquired images was also performed using the Photoscan software, which is implemented by SIFT (Scale-Invariant Feature Transform) and SfM (Structure

30° 45° 60° 75°

Focal length

(pixel) 2322.26 2321.81 2317.17 2315.36 2312.04 2314.66 2311.71 2314.07 Principal

point (pixel) 1965.52 1964.48 1963.64 1954.62 1523.46 1526.83 1524.40 1519.16 Skew coefficient -0.577973 -1.20985 -1.44937 -1.86215

Radial distortion

(mm)

0.000692 0.000379 0.000595 0.001174 0.000134 0.000221 0.000258 -0.000044

Tangential distortion

(mm)

-0.133500 -0.134432 -0.134226 -0.142983 0.120585 0.122726 0.125227 0.155982 -0.036011 -0.039159 -0.041544 -0.080931

0.009597 0.011360 0.011802 0.028931

Table 3. Interior orientation parameters for vertical images

3.3 Orthophoto and 3D model production DSM data is required to create an orthophoto. The GPS data acquired through VRS surveying in then entered to convert the relative coordinates to absolute coordinates (Lee and Lee, 2017; Lim et al., 2015). Fig. 8 shows orthophotos produced by various angle images.

The 3D models were generated using the Photoscan software’s build tiled model function. The build tiled model function for 3D modeling is performed using high-density point cloud. With the high-density point cloud, it is possible to obtain accurate 3D models for various types of terrain, such as planes and slopes. Fig. 9 shows the 3D modeling results corresponding to various angles.

(a) 30° (b) 45°

(c) 60° (d) 75°

(e) Vertical

Fig. 8. Orthophotos generated by various angle images

(a) 30°

(b) 45°

(c) 60°

(d) 75°

Fig. 9. 3D models generated by oblique images taken from

different angles

4. Result and Discussion

The accuracy of each orthophoto was evaluated with 11 checkpoints obtained by VRS survey. The maximum errors in the horizontal and vertical directions were 0.10 m and 0.13 m at 30 degrees, 0.14 m and 0.14 m at 45 degrees, 0.11 m and 0.13 m at 60 degrees, and 0.14 m and 0.12 m at 75 degrees, respectively. The RMSE (Root Mean Square Error) in the horizontal and vertical directions were 0.03 m and 0.07 m at 30 degrees, 0.04 m and 0.05 m at 45 degrees, 0.03 m and 0.06 m at 60 degrees, and 0.04 m and 0.08 m at 75 degrees, respectively. Tables 4 and 5 show checkpoint results for various angle images.

Checkpoint accuracy results for orthophotos were evaluated based on "Aerial Photogrammetry Regulation No.

2016-2609, Chapter 5, Article 56" prescribed by the NGII (National Geographic Information Institute) (reproduced in Table 6). The checkpoint accuracy of the orthophotos

produced by the images taken from all angles satisfied the RMSE and maximum error thresholds of a 1/500 scale map. In addition, it can be seen that there is no significant difference when compared with the vertical image produced in our previous study (Lee and Lee, 2017).

The comparison of 3D modeling using images taken from various angles included a qualitative evaluation of some buildings. The quantitative comparison was performed by comparing the RMSE of the z-directional discrepancies of building outlines between GPS survey and 3D models. Table 7 shows the RMSE of the Z discrepancies for 3D models for various angles. The RMSE result for the Z discrepancies showed that 3D modeling based on the 45 degree images was excellent for all buildings compared.

This research shows that it is possible to produce orthophotos using images taken at various angles. This study had several limitations. First, the research was conducted over a wide area with relatively wide spacing between buildings.

The proposed method is difficult to be applied to a study site covered by closely spaced buildings. This problem will be dealt with in the future studies under various conditions such as various heights and areas of buildings along with image taking directions and altitudes. Second, the evaluation of building walls was insufficient. As a result, assessment of 3D models was based on only building outlines.

Angle XY error (m) Z error (m)

30° 0.10 0.13

45° 0.14 0.14

60° 0.11 0.13

75° 0.14 0.12

90° 0.12 0.11

Table 4. Image checkpoint errors (maximum error) corresponding to various angles

Angle XY error (m) Z error (m)

30° 0.03 0.07

45° 0.04 0.05

60° 0.03 0.06

75° 0.04 0.08

90° 0.03 0.07

Table 5. Image checkpoint errors (RMSE) corresponding to various angles

Scale RMSE (m) Maximum error

(m)

1/500 – 1/600 0.14 0.28

1/1,000 – 1/1,200 0.20 0.40

1/2,500 – 1/3,000 0.36 0.72

1/5,000 0.72 1.44

1/10,000 0.90 1.80

1/25,000 1.00 2.00

Table 6. Tolerance criteria for RMSE and maximum error

5. Conclusion

In this study, orthophotos and 3D models based on various angle images were made using an Inspire 1 rotary-wing UAV equipped with an FC350 camera. The four angles used were compared by evaluating the accuracy of the respective products, resulting in the following conclusions.

First, the maximum errors in the horizontal and vertical directions were 0.10 m and 0.13 m at 30 degrees, 0.14 m and 0.14 m at 45 degrees, 0.11 m and 0.13 m at 60 degrees, and 0.14 m and 0.12 m at 75 degrees, respectively. The RMSE in the horizontal and vertical directions were 0.03 m and 0.07 m at 30 degrees, 0.04 m and 0.05 m at 45 degrees, 0.03 m and 0.06 m at 60 degrees, and 0.04 m and 0.08 m at 75 degrees,

Building name 30° (m) 45° (m) 60° (m) 75° (m)

±0.305 ±0.217 ±0.619 ±0.603

±0.558 ±0.212 ±0.264 ±0.536

± 0.402 ± 0.290 ± 0.572 ± 0.529

±0.178 ±0.165 ±0.434 ±0.634

± 0.588 ± 0.269 ± 1.388 ± 0.421

Table 7. RMSE of Z discrepancies corresponding to various angles

Building No. 3

Building No. 4

Building No. 5

Building No. 7

Building No. 8

respectively. These error levels satisfies the maximum error and RMSE criteria for a 1/500 scale map according to the aerial photogrammetry regulation. It is also remarkable that there is no significant difference when compared with the result of the orthophoto produced by vertical images.

Second, in the quantitative results for 3D modeling at various angles, building showed the best 3D modeling results for images taken from a 45 degree angle. As a result, we suggest that 45-degree imagery is best for 3D modeling.

Henceforth, it will be necessary to evaluate the accuracy of 3D modeling on the front, sides, and back of modelled buildings. In addition, 3D modeling comparison and evaluation research is needed regarding accuracy of orthophotos and 3D models produced using cross- directional vertical and high-oblique-angle (45°) images.

Finally, research is also needed on sites with densely-packed buildings and/or tall buildings.

Acknowledgment

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(No. NRF- 2017R1D1A1B03030611)

References

Chiabrando, F., D'Andria, F., Sammartano, G., and Spanò, A. (2018), UAV photogrammetry for archaeological site survey. 3D models at the Hierapolis in Phrygia (Turkey), Virtual Archaeology Review, Vol. 9, No. 18, pp. 28-43.

Doneus, M., Verhoeven, G., Fera, M., Briese, C., Kucera, M., and Neubauer, W. (2011), From deposit to point cloud–a study of low-cost computer vision approaches for the straightforward documentation of archaeological excavations, Geoinformatics FCE CTU, Vol. 6, pp. 81-88.

Eker, R., Aydın, A., and Hübl, J. (2018), Unmanned aerial vehicle (UAV)-based monitoring of a landslide:

Gallenzerkogel landslide (Ybbs-Lower Austria) case study, Environmental Monitoring and Assessment, Vol.

190, No. 1, pp. 1-14.

Han, S.H. and Hong, C.K. (2017), Acquisition of subcentimeter

GSD images using UAV and analysis of visual resolution, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 35, No. 6, pp. 563- 572. (in Korean with English abstract)

Hartley, R.I. (1997), Self-calibration of stationary cameras, International Journal of Computer Vision, Vol. 22, No. 1, pp. 5-23.

Han, Y.K., Byun, Y.G., Choi, J.W., Han, D.Y., and Kim, Y.I.

(2012), Automatic registration of high-resolution images using local properties of features, Photogrammetric Engineering & Remote Sensing, Vol. 78, No. 3, pp. 211-221.

Khaloo, A., Lattanzi, D., Cunningham, K., Dell’Andrea, R., and Riley, M. (2018), Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling, Structure and Infrastructure Engineering, Vol.

14, No. 1, pp. 124-136.

Lee, K.R. and Lee, W.H. (2017), Comparison of orthophoto and 3D modeling using vertical and high oblique images taken by UAV, Journal of the Korean Society for Geospatial Information Science, Vol. 25, No. 4, pp. 35-45.

(in Korean with English abstract)

Lim, Y.S., La, P.H., Park, J.S., Lee, M.H., Pyeon, M.W., and Kim, J.I. (2015), Calculation of tree height and canopy crown from drone images using segmentation, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 33, No. 6, pp.

605-613.

Lin, Y., Jiang, M., Yao, Y., Zhang, L., and Lin, J. (2015), Use of UAV oblique imaging for the detection of individual trees in residential environments, Urban Forestry &

Urban Greening, Vol. 14, No. 2, pp. 404-412.

Lucieer, A., Jong, S.M.D., and Turner, D. (2014), Mapping landslide displacements using Structure from Motion (SfM) and image correlation of multi-temporal UAV photography, Progress in Physical Geography, Vol. 38, No. 1, pp. 97-116.

Rossi, P., Mancini, F., Dubbini, M., Mazzone, F., and Capra, A. (2017), Combining nadir and oblique UAV imagery to reconstruct quarry topography: methodology and feasibility analysis, European Journal of Remote Sensing, Vol. 50, No. 1, pp. 211-221.

Uysal, M., Toprak, A.S., and Polat, N. (2015), DEM generation

with UAV photogrammetry and accuracy analysis in Sahitler Hill, Measurement, Vol. 73, pp. 539-543.

Verhoeven, G. (2011), Taking computer vision aloft–

archaeological three-dimensional reconstructions from aerial photographs with photoscan, Archaeological Prospection, Vol. 18, No. 1, pp. 67-73.

Wang, L., Niu, Z., Wu, C., Xie, R., and Huang, H. (2012), A robust multisource image automatic registration system based on the SIFT descriptor, International Journal of Remote Sensing, Vol. 33, No. 12, pp. 3850-3869.

Watanabe, Y. and Kawahara, Y. (2016), UAV photogrammetry

for monitoring changes in river topography and vegetation,

Procedia Engineering, Vol. 154, pp. 317-325.