IEG 환경지질연구정보센터

BUILDING DETECTION BY MULTIPLE IMAGE MATCHING AND SPECTRUM ANALYSIS

Hui-Hsin Kao ¹ , Liang-Chien Chen ²

1 Department of Civil Engineering, National Central University, Taiwan

2 Center for Space and Remote Sensing Research, National Central University, Taiwan Email: ¹ [email protected], ² [email protected]

ABSTRACT Building detection is one of the major works in building reconstruction and change detection for land cover. The major features in building detection include shape and spectrum information when images are employed.

Multiple image matching may determine the shape by generating three-dimensional point clouds with high reliability.

On the other hand, multi-spectral images provide color information for building detection. Hence, we combine those two information sets to achieve the detection. The detected building regions provide a solid ground for the successive building reconstruction.

This paper comprises four major steps: (1) extraction of features, (2) multiple image matching, (3) generation of digital surface model (DSM), and (4) spectrum analysis. Extraction of features by Canny operator and Target-defined ground operator (TDGO) from the reference image is the first step. Second, geometrically constrained cross-correlation (GC ³ ) algorithm is selected in the multiple image matching to generate three-dimensional point clouds. Third, the DSM derived by the rasterization of three-dimensional point clouds is analyzed. Finally, we use spectrum information from imagery to assist building detection.

The test datasets include six DMC images with 17cm spatial resolution. The overlap and sidelap are 80% and 30%, respectively. The experimental results indicate that the proposed scheme may reach high fidelity.

KEY WORDS: Features Extraction, Images Matching, Spectrum Analysis, Building Detection

1. INTRODUCTION

Three-dimensional building models are the important part in geospatial information applications, thus, building modeling is significant. Before reconstruction of three- dimensional building models, building detection is a necessary work (Rottensteiner & Briese, 2002). The focus of this study is on the detection part.

In building detection, it can use structural, contextual, and spectral information to detect building (Jin et al., 2005). Both, LIDAR data and images are the common materials. LIDAR point clouds provide highly accurate information of elevation. On the other hand, aerial or satellite images provide fruitful image details (Huang, 2010). However, aerial images are getting easier to obtain together with high spatial resolution and multi-spectral information. Thus, aerial images are selected in this study.

There are four major steps in this study, namely, extraction of features, multiple image matching, generation of digital surface model (DSM), and spectrum analysis. Extraction of features is the important work in the beginning. Features can be divided into feature points, feature lines, and feature surfaces. Since this study needs feature with uniform distribution to get three-dimensional point clouds, we decide to extract feature points by means of the improved TDGO (Chen & Lee, 1992). On the other hand, Canny (1986) operator for the extraction of features lines can help us to verify the building areas,

so we also extract feature lines. Multiple image matching deals with finding conjugate points and estimate height in the images. The strategies include area-based, feature- based and relational matching (Rosenholm, 1987; Habib et al., 2003). This paper employs the concept of GC³ method, as proposed by Zhang & Gruen (2006), in match images.

To improve the automation degree of building detection, combining various information is a tendency of nowadays. Following the development of digital, highly overlapping, fine resolution, and multi-spectral aerial imagery becomes useful information for the building detection. Highly similar texture with consecutive aerial images can benefit to analyze building areas. Taking advantages of this merit, this investigation proposes a scheme to detect building areas using multiple aerial images.

We employ image matching to extract large amount of conjugate image points to generate the surface, i.e. digital surface models (DSMs). DSMs then can be directly used to estimate candidate building areas in object space with elevation threshold. However, some building areas may be occluded by tree crowns and lead larger error in the further investigation like building reconstruction. To overcome this problem, we also combine the multi- spectral information for tree detection in image space.

Therefore, tree areas are removed from above ground

areas.

2. METHODOLOGIES

The paper comprises four major steps: (1) extraction of features, (2) multiple image matching, (3) generation of DSM, and (4) spectrum analysis. The workflow is shown in Figure 1.

Figure 1. Workflow of the propose scheme

2.1 Feature Extraction

First of all, we need to extract candidate building areas using multiple aerial images. TDGO from the reference image are employed for multiple image matching to keep favourable point distribution. To retain more points in the feature point extraction, low threshold is incorporated.

On the other hand, Canny (1986) proposed an effective edge detection method. Extracting feature lines by Canny can help us for verifying the final decision.

2.2 Multiple Image Matching

The purpose of multiple image matching is to find the conjugate points in a reliable way. We employ the concept of GC³ method to generate three-dimensional point clouds, as shown in Figure 2.

Figure 2. Multiple image matching with the GC3 algorithm.

(Zhang & Gruen, 2006)

This process selects an image as the reference image and then extracts features from all used images. For each feature point in the reference image, we give an initial height value to calculate an approximate three- dimensional position. The following step back-projects this position to slave images and finds the correspondent

conjugate points using normalized correlation coefficient (NCC) technique. We iteratively modify the height value and compute the summation of NCC values of all images.

This process then derives a refined three-dimensional position of each feature point when the maximum NCC is estimated.

2.3 Generation of nDSM

After multiple image matching for generation of three- dimensional point clouds, we use bilinear interpolation to generate the DSM. Since, the building is higher than 3m in general, we exclude the areas from nDSM where its height is less than 3m.

2.4 Spectrum analysis

Because the DSM are not only building characteristic, it might contain other surface cover likes trees, vehicles and so on. We, thus, use the multi-spectral images which provide color information for building detection.

Because the test images don’t have the near infra red (NIR) and thinking of the characteristic of vegetation, we use the Greenness Index (GI) to differentiate between vegetation and building. Its principle is that vegetation absorbs red-spectral (R) and reflects green-spectral (G).

After calculating the GI, we can detect vegetation areas (Niederöst, 2001). This count is shown in equation (2).

GI = (G-R) / (G+R) (2)

If the value is close to 1, there is vegetation covering with high probably. On the other hand, if the value is close -1 to 0, there are buildings covering with high probably. We exclude the vegetation areas where GI value is more than 0.3. Assuring that the buildings are big enough, those areas with small size, less than 30m ² for instance, are also excluded.

3. EXPERIMENT AND RESULTS

This test site locates in Taipei City of Northern Taiwan.

The datasets include six DMC images with 17cm spatial resolution, as shown in Figure 3. The overlap and sidelap are 80% and 30%, respectively.

Figure 3. Six DMC images

As shown in Figure 3., the first image is selected as the reference, and others are slave images. In the test areas, experimental results include four parts. First, feature points by TDGO and feature lines by Canny. Second, the DSM by multiple image matching generating the three-dimensional point clouds. Third is the result of spectrum analysis. Finally, we examine the results of building areas.

3.1 Results of Feature Extraction.

There are 9957 feature points by TDGO. These points on the reference image are used to match with others images. Feature lines by Canny let us indentify building contours more clearly. Feature points by TDGO and feature lines by Canny are shown in Figure 4.

Reference image

TDGO Canny

Figure 4 . Feature point by TDGO and feature lines by Canny 3.2 nDSM Generation

This step uses the extracted features to generate DSM.

The matching process selects the minimum NCC threshold as 0.7 and finds conjugations to generate three- dimensional point clouds. We then interpolate those points to generate DSM. In addition, this step also uses the terrain height to produce the nDSM for the extraction of candidate building areas. The height threshold is 3m.

The results are shown in Figure 5 .

DSM DSM’

Figure 5. DSM & DSM’

3.3 Results of Spectrum Analysis

This investigation analyzes the spectral information to detect candidate vegetation areas. The GI is used with the threshold, 0.3. The result shows in Figure 6. This figure illustrates that the vegetation covers white areas.

Figure 6. Greenness Index image

3.4 Results of Building Areas

The building areas are shown in Figure 7. The building areas are then detected after the analyses of DSM’ and spectrum analysis.

Figure 7. The building areas image

4. CONCLUSIONS AND FUTURE WORKS This paper has proposed a scheme to use highly overlapped multiple strips and multi-spectral aerial images in building detection. The preliminary results show that this research has the ability to locate building areas. The future works may improve the automation degree of building reconstruction and change detection.

5. REFERENCES 5.1 References

Canny, J., 1986, A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, pp. 679-698.

Chen, L.C. & Lee, L.H., 1992. Progressive Generation of Control Frameworks for Image Registration, Photogrammetric Engineering and Remote Sensing, Vol.

58, No. 9, pp. 1321-1328.

Habib, A., Lee, Y. and Morgan, M., 2003. Automatic Matching and Three-Dimensional Reconstruction of Free-Form Linear Features from Stereo Images, Photogrammetric Engineering and Remote Sensing, 69(2):189-197.

Huang, Y.H., 2010, Multiple Image Matching for three- dimensional Building Modeling, Master degree dissertation, National Central University, Taiwan. (In Chinese)

Jin, X., Davis, C. H., 2005. Automated Building Extraction from High-Resolution Satellite Imagery in Urban Areas Using Structural, Contextual, and Spectral Information, EURASIP Journal on Applied Signal Processing, Vol. 14, pp.2196-2206.

Niederöst, M., 2001. Automated update of Building Information in Maps Using Medium-scale Imagery (1:15,000), Automatic Extraction of Man-Made Objects from Arial and Space Images, Vol. 3, pp.161-170.

Rosenholm, D., 1987. Least-squares Matching Method:

Some Experimental Results, Photogrammetric Record, 12(70):439-512.

Rottensteiner, F., & Briese, Ch., 2002. A New Method For Extraction In Urban Areas From High-Resolution LIDAR Data, ISPRS, vol. XXXIII, pp. 295-301, Graz, Austria.

Zhang, L.& Gruen, A., 2006, Multi-image matching for DSM generation from IKONOS imagery, ISPRS Journal of Photogrammetry and Remote Sensing, pp 60 (2006) 195–211.

6. ACKNOWLEDGEMENTS

Those images in this investigation were provided by

the Department of Land Administration, Ministry of the

Interior in Taiwan.