Evaluation of Geo-based Image Fusion on Mobile Cloud Environment using Histogram Similarity Analysis

1. Introduction

Cloud computing means the computing infrastructure based on assembled technologies that relies on sharing computing resources, independent of local servers or specific devices, to implement and operate applications in the diverse fields (Barnatt, 2010).

In fact, it is the result of evolution and adoption of existing technologies and paradigms, so as to allow users to take benefits from those technologies. The web services on cloud computing service are known to be cost-effective regarding to their development and operation, and help the providers and users focus on their core business instead of being impeded by many

Evaluation of Geo-based Image Fusion on Mobile Cloud Environment using Histogram Similarity Analysis

Kiwon Lee ^† and Sanggoo Kang

Department of Information Systems Engineering, Hansung University

Abstract : Mobility and cloud platform have become the dominant paradigm to develop web services dealing with huge and diverse digital contents for scientific solution or engineering application. These two trends are technically combined into mobile cloud computing environment taking beneficial points from each.

The intention of this study is to design and implement a mobile cloud application for remotely sensed image fusion for the further practical geo-based mobile services. In this implementation, the system architecture consists of two parts: mobile web client and cloud application server. Mobile web client is for user interface regarding image fusion application processing and image visualization and for mobile web service of data listing and browsing. Cloud application server works on OpenStack, open source cloud platform. In this part, three server instances are generated as web server instance, tiling server instance, and fusion server instance.

With metadata browsing of the processing data, image fusion by Bayesian approach is performed using functions within Orfeo Toolbox (OTB), open source remote sensing library. In addition, similarity of fused images with respect to input image set is estimated by histogram distance metrics. This result can be used as the reference criterion for user parameter choice on Bayesian image fusion. It is thought that the implementation strategy for mobile cloud application based on full open sources provides good points for a mobile service supporting specific remote sensing functions, besides image fusion schemes, by user demands to expand remote sensing application fields.

Key Words : Bayesian image fusion, Histogram similarity, Mobile cloud computing, OpenCV, OpenStack

Received November 23, 2014; Revised December 29, 2014; Accepted December 29, 2014.

† Corresponding Author: Kiwon Lee ([email protected])

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons. org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited

Article

obstacles caused by information communication technology (ICT) (Jeong et al., 2011).

Mobile cloud computing can be implemented and used in the style of the combination of cloud computing and mobile networks or cloud computing on mobile devices. In fact, cloud computing service already considers the mobile environment according to a multi- device supporting strategy (Hung et al., 2012). Mobile application service provider provides digital content, applications or services to mobile or cellular subscribers. Mobile cloud computing inherits the main features and benefits of the cloud computing technology on mobile devices for anywhere and anytime (Fernando et al., 2013; Sanaei et al., 2014).

Despite many beneficial points of cloud computing service and platform, cloud-based applications in the geo-spatial fields are still on the stage of the proof of concept (Yang et al., 2011; Kang et al., 2012; Yue et al., 2013; Evangelidis et al., 2014; Tang and Feng, 2014); needless to say, those of mobile cloud environment are rare. As the previous works, a smartphone application for satellite image information processing service were implemented on Android OS (Kang and Lee, 2013a). Mobile application providing image processing algorithms, such as Maximum Auto- correlation Factor (MAF) and edge detection,was designed and developed on Amazon Web Service Elastic Cloud Computing (AWS EC2) and Simple Storage Service (S3) (Lee and Kang, 2013). Gasperi (2013) applied OTB processing chains as Open Geospatial Consortium (OGC)-Web Processing Services (WPS) in cloud computing architecture.

The purpose of this study is to design and develop a mobile cloud application service model. This study presents a test case for practical remote sensing application on the mobile cloud computing, and deals with image fusion and open source cloud platform.

Image fusion is regarded as one of important functionalities according to the continuously increasing the types of multiple geo-based data sources (Zhang,

2010). As for the numerous image fusion schemes or algorithms in Zhang(2010), data fusion algorithm by Bayesian approach in the OTB (OTB, 2014) was applied. Cloud computing platform based on open source is also the most important thing to implement cloud-based application or service model (Kang and Lee, 2013b), and OpenStack was used for open source cloud computing platform, which is an open and scalable operating system for building public and private clouds. Taking advantage of mobility and cloud computing portability, users can perform satellite image fusion processing. While, whatever any approaches or algorithms are applied, it is natural that fused image is not a unique. In contrast with desktop processing, most mobile processing is prompt, so user selection or criterion regarding the instant result is important.

Histogram similarity of color image can be applied for quantitative analysis for this task, by means of distance metrics (OpenCV, 2014). The result of Bayesian image fusion varies according to user input variables.

Normally, users select their final result on their experiences or rule of thumb. Analysis of fusion resultsis the second research theme followed by Bayesian image fusion as the main content in this study.

Histogram similarity method was used for this task.

2. Applied technology and schemes

1) Briefing of OpenStack

OpenStack is an open source software cloud computing platform started by a joint project of Rackspace and NASA in 2010, now which is managed and governed by a non-profit foundation, a technical committee and a global user committee. This project aims to deliver solutions for all types of cloud computing environment for cloud application developers or operators (OpenStack, 2014).

The technologies within OpenStack consist of a

series of interrelated open projects containing processing, storage, and networking resources. All components used in a cloud application of OpenStack are managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface (Jackson and Bunch, 2013). Technically, OpenStack has as a modular architecture, composing of a fabric controller, a scalable storage system, block-level storage devices, a system for managing networks and IP addresses, and so on, including database service, and API sets compatible with Amazon EC2 and Amazon S3. Owing to virtualization server technologies of OpenStack cloud platform, cloud-based application services utilize those features and beneficial points at anytime and anywhere, only on internet-accessible environment.

2) Bayesian image fusion

Orfeo Toolbox abbreviated to OTB, providing a variety of open source satellite image processing algorithms, was applied to perform image fusion by Bayesian estimation. The concept of data fusion scheme applied in this study was proposed and explained in Bogaert (2007). Variables of interest are denoted as vector Z i , linked to the observable variables Y i, j through an error-like model expressed in Equation (1) (Fasbender et al., 2009),

Y i, j = g i, j (Z i ) + E i, j (1) where i corresponds to the channel number and g i, j

(Z i ) is a set of functional and E means a vector of random errors, stochastically independent from Z.

Assuming that the E i,j ’s of E are stochastically independent, conditional probability density function of thevector of interest given the observed variables can be obtained. Using Bayes theorem with this concept as Equation (2),

f (z | y) ∝ f z (z) ∏ ⁿ

i = 1 ∏ ^p

j = 1 ) (2)

where f(z i ) is the a priori distribution of Z. The

Bayesian image fusion processing in OTB requires one parameter λ from user input value, the weight to be given to the panchromatic image compared to the multispectral one.

3) Histogram similarity metric

OpenCV provides the function cvCompareHist to compare two histogram comparison with different metrics (OpenCV, 2014). Among several quantitative metrics, two methods such as correlation and Bhattacharyya distancewere applied. As OpenCV methods, correlation, CV_COMP_CORREL, and Bhattacharyya distance, CV_COMP_BHATTACHARYYA, are computed by the equation (3) and the equation (4), respectively. In both equations, H 1 and H 2 represent two histogram to compare, and N is the number of bins of the histogram. And d(H 1 , H 2 ) represents a distance metric to express how well both histograms match.

d (H 1 , H 2 ) = (3)

d (H 1 , H 2 ) = (4) where H′ i = ∑ ^{H i} . For correlation, it is convention that a high score represents a better match than a low score. The range of a full discrepancy to a perfect match is -1 to +1, and a value of 0 indicates no correlation. As for Bhattacharyya distance metric, a perfect match between two histograms is 0 and amaximal mismatch is a 1. Thus, low scores indicate good matches and high scores indicate bad matches.

3. System design and implementation

The system component and processing units are presented in Fig. 1. The basic architecture is two sides:

mobile client and cloud application server. Mobile client is for mobile web service linked to OpenStack f (z i | y i, j )

f (z i )

∑ (H 1 (I) _ H′ 1 )(H 2 (I) _ H′ 2 )

∑ (H 1 (I) _ H′ 1 ) ² ∑ (H 2 (I) _ H′ 2 ) ² 1 _ ∑ H ¹ (I)·H 2 (I)

H′ 1 H′ 2 N ²

N 1

cloud computing environment and for user interface for image fusion application. GitHub-based libraries applied in this implementation were the version on 6 December 2013.

Table 1 is the summary for development and operation

environment and open sources for the implementation of mobile cloud application applied in this study. In the cloud application server by OpenStack cloud environment, three server instances were generated, and each was operated on Ubuntu 12.04. Web server

Fig. 1. System components and processing procedures of mobile application service for satellite image fusion based on full open sources and cloud platform.

Table 1. Development/operation environment and open sources for the implementation of mobile cloud application Development/operation environment Open source/platform/integrated

development environment

Cloud-based Server Side

Web Server Instance

Operating system Ubuntu 12.04 64bit

Web server Apache httpd 2.2.22

Message broker RabbitMQ 3.2.2.-1

Database MongoDB 2.4.9

Image tiling and converting GDAL 1.8.1

System supporting libraries Pika 0.9.13, PyMongo 2.6.3

Fusion Server Instance

Operating system Ubuntu 12.04 64bit

Satellite image processing OTB 3.20.0 System supporting library Pika 0.9.13

Image converting GDAL 1.8.1

Tiling Server Instance

Operating system Ubuntu 12.04 64bit

System supporting library Pika 0.9.13

Image tiling GDAL 1.8.1

Web and Mobile Client Side

Bayesian Image Fusion Application

Operating system iOS 7.0

User interface and networking support libraries

Route-me, AFNetworking, SWRevealView, ProgressHUD, HATransparentView

Web Service User interface library jQuery 2.0.3

instance receives user requests through Apache web server, and then works communication to other server instances using scripting codes and RabbitMQ, as messaging broker server.

While, MongoDB, document store server, was used for managing the user information and cataloging of archived satellite images. Tiling server instances uses Geospatial Data Abstraction Library (GDAL) for tiling and file format conversion of satellite image. Pika is library for Python to access RabbitMQ servers and PyMongo is for working with MongoDB. Fusion server instance is for satellite image processing functionalities based on OTB, and the processed results were stored in web server instances and display them on mobile devices.

Mobile application on iPad2 tablet provides functions for image visualization and browsing. In this application, route-me and AFNetworking was used visualization of satellite images within a user interface and server communication, respectively. As well, libraries such as SWRevealView, ProgressHUD, and HATransparentView were used for additional user interfacing. A cross-platform JavaScript library, jQuery, was used for the client-side scripting of dynamic mobile web pages. Being implementation by

full open source, cloud computing environment in this study is applicable to scalable operation to expand or reduce the necessary instances in any side according to the request types and the number of users. It also works for load balancing to optimize mobile connecting condition.

4. Experiment results

Fig. 2 represents the satellite images searching process for the accessed users in this implemented application. Metadata information regarding image sets selected by user are also provided with a small-sized image from cloud storage system. In a mobile application compared to a desktop application, the user interface design is more important because the user interaction on mobile device is limited. Thus, user experience, rather than user interface, has been emphasized on in the system implementation and its actual services.

Fig. 3 shows a user interface for data fusion process of mobile application on iPad2. It shows the main menu system and the sub menu system for selection of

Fig. 2. Data listing and metadata browsing on iPad2 mobile environment.

functions and satellite image data sets for image fusion, taking intuitive grasp and uses of users into consideration. Touch commands or core gestures such as tap, double tap, drag, flick, pinch, spread, or press are also applicable as major user action of this mobile application.

The area covering data set applied image fusion is on the center coordinate of 127˚ 26’ 45.05”E, 36˚ 26’

38.02”N, acquired on 12 April, 2009. Fig. 4 shows the results of different parameter application of

Bayesian image fusion of Korea Multi-Purpose Satellite (KOMPSAT) 2 image sets. Parameter λ of 0.5, 0.75, and 0.99 was applied as examples.

As for further processing of color image similarity measure with these results, RGB components of the fused image by Bayesian image fusion are transformed into hue-saturation-value (HSV) space. The HSV components of a base image to be compared to the other target imagesand those of target images are calculated as normalized histograms with the value of Fig. 4. Examples of Bayesian image fusion from different value of λ: (a) 0.50, (b) 0.75, and (c) 0.99.

Fig. 3. User interface of iPad2 mobile application for image fusion: main menu and selection of data and fusion processing mode.

bin, as histogram size. For comparison of histogram distance, hue-saturation (H-S) histogram is applied according to the computation scheme in Bradski and Kaehler (2008). Histogram size or the number of bin are an important factor to interpret distance metric of color image histogram (Ma et al., 2010). As for the number of bin, the same number is used for H-S space.

Fig. 5 shows H-S histogram similarity among images fused by different value of λ of Bayesian image fusion, with respect to multispectral image as a base image. This result shows that the case of the lower λ value in Bayesian image fusion conserves color information of multispectral data sets, relatively that of the higher value.

As well, the number of bins is crucial factor to interpret the results by histogram metrics computation. While, higher λ increases the sharpness or details in the visual investigation with these results. Of course, this result is not generalized, because distance metrics can be varied according to the type and number of features in the processed image sets. In any cases, the optimal parameter selection are important to get plausible results by instant usage on the mobile device.

5. Concluding Remarks

Cloud computing is a computing paradigm that

delivers resources such as a processing, storage, network and software, as services over the internet in a remotely accessible fashion. In applications on cloud platform, uses can access cloud applications using web browser, thin client machines or mobile devices, while all data and software are stored on servers at a remote location, which are also used to perform even time- consuming or heavy processing. As the cloud-based applications and their concerned researches in many other fields are increasing, a cloud computing paradigm is still new to remote sensing communities containing developers and users, despite many known beneficial points from using cloud platform. As a simple advantageous instance of system operation, users who access the cloud computing service in any device including mobile or desktop can perform remote sensing image processing what they want, except other computing works such as software installation, maintenance, or patch upgrading. As well, web-based software service for common uses of specific advanced algorithms or value-added processing scheme in a research group can be implemented in the private cloud environment.

This study implements a test case with Bayesian image fusion functions in the mobile application on an open source cloud computing platform. OpenStack was used in this implementation and system architecture but Fig. 5. Result of H-S histogram similarity on images fused by different value of λ of Bayesian image fusion in Fig. 4 with respect to

multispectral image as a base image: (a) correlation and (b) Bhattacharyya distance metric.

other cloud platforms, including commercialized or proprietary cloud environments, are also applicable, as long as they support mobile web environment. Bayesian image fusion is one of practical schemes for remotely sensed images usages, and instant processing with multiple satellite image sets with metadata browsing of the processing datasometime needs in a mobile environment. While, similarity of fused images with respect to input image set is estimated by histogram distance metrics. This result provides reference criterion for user parameter choice in this mobile application usage, and the parameter value can be tuned toward either a color consistency or sharpness level in details.

Finally, the implementation strategy for mobile cloud application based on full open sources provides good points coping with target-oriented request functions and processing types by user demands to expand remote sensing application fields, besides image fusion purposes.

Acknowledgment

This research was financially supported by Hansung University during sabbatical year (2014).

References

Barnatt, C. 2010. A brief guide to cloud computing, pp.

1-289 (UK: Constable & Robinson Ltd.).

Bogaert, P. and D. Fasbender, 2007. Bayesian data fusion in a spatial prediction context: a general formulation, Stochastic Environmental Research and Risk Assessment, 21(6): 695-709.

Bradski, G. and A. Kaehler, 2008. Learning OpenCV computer vision with OpenCV library, pp. 199- 206 (USA: O’Reilly).

Evangelidis, K., K. Ntouros, S. Makridis, and C.

Papatheodorou, 2014. Geospatial services in

the cloud, Computers and Geosciences, 63:

116-122.

Fasbender, D., V. Obsomer, P. Bogaert, and P.

Defourny, 2009. Updating scarce high resolution images with time series of coarser images: a bayesian data fusion solution, In Sensors and Data Fusion, N. Milisavljevic (Ed.), pp. 245-262 (Vienna: I-Tech Education and Publishing).

Fernando, N., S.W. Loke, and W. Rahayu, 2013.

Mobile cloud computing: a survey, Future Generation Computer Systems, 29: 84-106.

Gasperi, J., 2013. Cloud computing & web processing services, Available online at: https://

speakerdeck.com/jjrom/cloud-computing-and- web-processing-services (accessed May 2014).

Hung, S.-H., C.-S. Shih, J.-P. Shieh, C.-P. Lee, and Y.- H. Huang, 2012. Executing mobile applications on the cloud: framework and issues, Computers and Mathematics with Applications, 63(2):

573-587.

Jackson, K. and C. Bunch, 2013. OpenStack cloud computing cookbook, 2nd Edition, pp. 1-396 (Birmingham-Mumbai: Packt Publishing).

Jeong, U., D. Kang, and S. Jung, 2011. Trend of open source SW-based cloud computing technology, Electronics and Telecommunications Trends, 26: 43-54 (in Korean).

Kang, S., K. Lee, and Y. Kim, 2012. Preliminary performance testing of geo-spatial image parallel processing in the mobile cloud computing service, Korean Journal of Remote Sensing, 28(4): 467-475 (in Korean with English abstract).

Kang, S. and K. Lee, 2013a. Mobile app approach by open source stack for satellite images utilization, Remote Sensing Letters, 4(4): 648- 656.

Kang, S. and K. Lee, 2013b. Testing implementation

of remote sensing image analysis processing

service on OpenStack of open source cloud platform, Journal of the Korean Association of Geographic Information Studies, 16(4): 141- 152 (in Korean with English abstract).

Lee, K. and S. Kang, 2013. Mobile cloud service of geo-based image processing functions: a test iPad implementation, Remote Sensing Letters, 4(9): 910-919.

Ma, Y., X. Gu, and Y. Wang, 2010. Histogram similarity measure using variable bin size distance, Computer Vision and Image Understanding, 114(8): 981-989.

OpenCV, 2014. OpenCV 2.4.9.0 documentation, Available online at: http://docs.opencv.org/

doc/tutorials/ imgproc/histograms/histogram_

comparison/histogram_comparison.html (accessed April 2014).

OpenStack, 2014. OpenStack Installation Guide for Ubuntu 12.04 (LTS) Havana, Available online:http://docs.openstack.org/havana/install- guide/install/apt/content/index.html (accessed August 2014)

OTB Development Team, 2014. the ORFEO Tool box software guide updated for OTB-4.0, Available online at:http://www.orfeo-toolbox.org/packages/

OTBSoftwareGuide.pdf (accessed August 2014).

Sanaei, Z., S. Abolfazli, A. Gani, and R. Buyya, 2014.

Heterogeneity in mobile cloud computing:

taxonomy and open challenges, IEEE Communications Surveys & Tutorials, 16(1):

369-392.

Tang, W. and W. Feng, 2014. Parallel map projection of vector-based big spatial data: Coupling cloud computing with graphicsprocessing units, Computers, Environment and Urban Systems, http://dx.doi.org/10.1016/j.compenvurbsys.201 4.01.001

Yang, C., M. Goodchild, Q. Huang, D. Nebert, R.

Raskin, Y. Xu, M. Bambacus, and D. Fay, 2011.

Spatial cloud computing: how can the geospatial sciences use and help shape cloud computing?, International Journal of Digital Earth, 4(4): 305-329.

Yue, P., H. Zhou, J. Gong, and L. Hu, 2013.

Geoprocessing in cloud computing platforms - a comparative analysis, International Journal of Digital Earth, 6(4): 404-425.

Zhang, J. 2010. Multi-source remote sensing data

fusion: status and trends, International Journal

of Image and Data Fusion, 1(1): 5-24.