1 INTRODUCTION PiotrSt ę pie ń DamianKorbuszewski Ma ł gorzataKujawi ń ska DigitalHolographicMicroscopywithextendedfieldofviewusingtoolforgenericimagestitching

(1)

S P E C I A L I S S U E

Digital Holographic Microscopy with extended field of view using tool for generic image stitching

Piotr St ępień | Damian Korbuszewski | Ma łgorzata Kujawińska

Warsaw University of Technology, Institute of Micromechanics and Photonics, Warsaw, Poland.

Correspondence

Piotr Stępień, Warsaw University of Technology, Institute of Micromechanics and Photonics, Warsaw, Poland.

Email: [email protected]

Funding information

Foundation for Polish Science, Grant/

Award Number: TEAM TECH/2016-1/4;

European Union

This paper describes in detail the processing path leading to successful phase images stitching in digital holographic microscope for the extension of the field of view. It applies FIJI Grid/Collection Stitching Plugin, which is a general tool for images stitching, non-specific for phase images. The FIJI plugin is extensively supported by aberration and phase offset correction. Comparative analysis of different aberration correction methods and data processing strategies is presented, together with the critical analysis of their applicability. The proposed processing path provides good background for statistical phase analysis of cell cultures and digital phase pathology.

K E Y W O R D S

aberration correction, cell culture analysis, digital holographic microscopy, digital holography, stitching

1 | I N T R O DU C T I O N

Microscopic investigation of transparent objects is a crucial aspect of modern biology and technology. One of the techniques that enable such investigation is Digital Holographic Microscopy (DHM), which is a label-free quantitative technique capable of assessment of transparent objects [1, 2].

This work is an attempt to bring DHM closer to virtual microscopy (VM), known also as digital microscopy, which is an approach that enables digital transmission and investigation of a measured sample without actually pos- sessing it. In the past two decades, VM has been steadily adopted in the optical microscopy society, in research, medical practice [3] and teaching [4]. VM gives great opportunity for education and telemedicine purposes and allows multiple research groups to work on exactly the same data, taking advantage of modern digital image processing techniques. On the other hand, the libraries of digital histopathological samples allow to make use of such tools as artificial intelligence to search for cancer cells or other abnormalities in an image. Typical biological samples like tissues and cell cultures have extensive dimensions in

the order of a few centimeters, compared to the field of view (FoV) of a microscope—a few tens to a few hundreds of micrometers. For this reason, extension of the FoV while maintaining high spatial resolution is crucial for comprehensive investigation of those objects. These require- ments can be met by alternative imaging techniques such as lensless digital in-line holography [5], contact-imaging microscopy [6], ptychography [7] and Fourier ptychography [8]. The other approach is the mechanical extension of the FoV by modern digital slide scanners used for optical microscopy [9] or by synthetic aperture approach i.e.

sequential gathering of small FoVs and stitching image information to obtain much bigger synthetic FoV[10]. In the case of DHM it is impossible or very difficult to stitch sequential holograms, therefore it is advised to combine phase maps calculated from separate holograms as they should create a continuous object phase map.

The DHM enables quantitative measurement of the phase delay introduced by the label-free transmissive sample stored conveniently on a Petri dish. Contact imaging microscopy requires the objects to be placed directly on the light-sensitive sensor, which enables a highly compact

- - - - This is an Open Access article distributed under the term of Korea Open Government License (KOGL) Type 4: Source Indication + Commercial Use Prohibition + Change Prohibition (http://www.kogl.or.kr/info/licenseTypeEn.do).

ETRI Journal. 2019;41(1):73–83. wileyonlinelibrary.com/journal/etrij | 73

(2)

solution, but is not convenient in a biological laboratory, where the samples are cultivated, stored and measured on a Petri dishes. Additionally, the image is an intensity map, without any quantitative information about biophysical features (e.g. refractive index, dry mass) of an object. Lens- less DHM does provide the same type of quantitative measurement as regular DHM, but it is not convenient for whole slide imaging (WSI), analogously to the contact imaging microscopy. Ptychography and Fourier ptychography provide very similar benefits to the stitching of DHM phase maps, but offer different trade-offs in terms of single shot measurements [1,11,8,7]. Very few approaches for the phase images stitching have been described in literature [12,13] and none of them try to use algorithms developed for other imaging modalities. Instead those works are focused on developing new tool for phase images stitching.

This tool requires initial calibration to compensate stage- camera misalignment and deals with phase errors in a way that is dependent on the calibration.

Here, we present an accustomed processing path that leads to successful usage of a common generic image stitching algorithm [14] for the automatic stitching of phase images. This work proposes a new approach to enable seamless stitching of phase maps obtained by the means of DHM technique.

The paper is composed as outlined below. In Section 2 we describe the measurement method and hardware, as well as data processing path. Sections 3–5 are devoted to detailed description and experimental evaluation of the main processing steps. Finally in Section 6 the results of complete processing path are presented and discussed.

2 | HARDWARE AND DATA PROCESSING OUTLINE

In this section, we provide the description of the Digital Holographic Microscope setup accustomed for extended FoV measurement, hologram processing, as well as indication of the main processing modules required for highly accurate phase images stitching.

2.1 | Digital Holographic Microscope setup

Custom built bi-modal Mach-Zehnder based inverted microscope has been used throughout this work (see Figure 1). The two modalities that are enabled by the hardware are DHM and white light microscopy. The system has been built around the Olympus CKX41 microscope body, which makes the setup mobile and compact. Addi- tionally, the microscopes capabilities have been enhanced by mounting motorized sample stage Zaber ASR050B050B that allows for extended FoV measurement. Integrated

Optics MatchBox2 CW DPSS laser with wavelength λ = 532 nm has been used as a light source. C1 lens colli- mates the light, that is than split by a beam-splitter cube.

Transmissive sample is illuminated by the object beam and imaged on the CCD by MO1-TL1 telecentric microscope setup. Second telecentric setup of of MO2 microscope objective and TL2 tube lens forms the plane wave in the reference beam. Beam-splitter cube is used to combine both beams. Microscope objective that has been utilized is LUCPLFLN60x Olympus, that nominally should work with f_TL ¼ 180 mm tube lens. Because fTL ¼ 150 mm tube lens has been used, the magnification is 50x instead of 60x, as implied by the MO description.

JAI GO-5101M-PGE camera with (2,056 × 2,464 active pixels and pixel size a = 3.45μm) captures the off-axis image plane hologram, that is than analyzed using Fourier Transform Method (FTM) [15] for phase retrieval followed by unwrapping using algorithm based on sorting by reliability following a noncontinuous path [16]. Due to win- dowing functions that are necessary for the FTM, a single FoV was 125μm × 150 μm. Because the MO1 transmits only the information that lies within its numerical aperture (here, NA = 0.7), the output phase image was significantly smaller than the camera matrix size—final size of the image was (298 × 358). After a single hologram is captured by the CCD camera, the motorized stage moves into the next position and another hologram is recorded. The nominal overlap of neighboring regions was 13%. All phase maps retrieved from the sequentially captured holograms are then processed in the procedure described in the following subsection 2.2.

The investigated sample was a HeLa 21.4 cultured on the Ibidiμ-Dish 35 mm, fixed in 4% formaldehyde solution for 30 minutes and labeled over 12 hours with acridine orange (AO), 10μm. AO properties are useful in Confocal Laser Scanning Microscopy and have been utilized in a different study [14]. Here it should be also noted that the F I G U R E 1 Schematic of the Digital Holographic Microscopy setup used in this work: C, collimating lens; M, mirror; S, sample;

MS, motorized stage; MO, microscope objective; TL, tube lens; CCD, charge-coupled device camera. In the blue rectangle the scheme of acquiring data for the stitching procedure is shown. Note that the exemplary image is a phase map, not a hologram

(3)

investigated cell culture has been densely populated. In such case as well as in the case of tissue studies it is difficult or impossible to find an area where objects are not seen in the FoV. This has significant consequences in further choice of processing path and algorithms.

Some of the measurements contained regions with non- transparent objects (impurities) in the sample where phase retrieval has been impossible. These regions have been masked in order to prevent corruption of processing steps.

2.2 | Data processing path

In this subsection, we outline the problems that are specific for the phase images stitching process. The more general problems of generic images stitching, like globally opti- mized image registration and image blending are beyond the scope of this work and are dealt with by the stitching algorithm [14]. Here we focus on two problems. The first one is the presence of inherent aberrations of the DHM system, leading to reduced correlation between the overlapping regions and skewed quantitative measurement. The second problem is the presence of relative offset of phase values in sequential measurements innate to the interferometric systems. For each of the problems we provide a concise theoretical background and propose processing paths to mitigate them. The main procedures of the processing path are performed sequentially and include:

• main aberrations correction during which the most sig- nificant aberration components are removed;

• residual aberrations correction to reduce the object dependent error specific for the chosen main aberration correction method and tilt introduced during movement of the object.

• phase offset correction where phase offset is estimated and removed from each individual image. This step ensures that the neighboring images are representing values using the same reference.

• stitching of phase maps using globally optimal stitching algorithm [14]. The output is a stitched phase image.

In further sections these processing steps are thoroughly described.

3 | ABERRATIONS CORRECTION 3.1 | Methods and strategies

Phase images stitching has specific additional problems in comparison to generic images stitching. In the general case DHM phase images represent differential phase map of object and reference beams at the detector plane, which in our case is a plane conjugated to the object plane.

However, object beam includes not only object information but also aberrations introduced by DHM optical system, while reference beam might also be distorted by aberrations. That is why phase shift introduced by the object is corrupted by additional phase components, not related to the investigated specimen and therefore specimen-shift invariant.

The problem is best illustrated by the equation of complex amplitude encoded into hologram captured by CCD:

Ψðx; yÞ ¼ Aðx; yÞ exp iϕ½ oðx; yÞ exp iϕ½ aðx; yÞ (1) where A is amplitude of the wavefront, ϕo is phase related to the investigated object,ϕais additional phase component.

The additional phase component ϕa contains all wavefront aberrations and unwanted phase information introduced e.g. by impurities within the system like a speck of dust on the MO, random tilt due to motorized stage movement, however most significant influence have the second order wavefront aberrations originating from the optical components and first order aberrations due to the acquisition scenario, for example, off-axis architecture.

Those components are present in the FoV of each single hologram and most significantly corrupt the object phase data at the edges of FoV. This is the reason why the neighboring phase data (retrieved from holograms) may exhibit low similarity. From our experience this is the main reason for the malfunctioning of the stitching algorithm.

To mitigate the problems described above, researchers have provided multiple solutions, either hardware based or numerical. The most commonly used experimental method to remove the influence of optical system aberrations (systematic phase error) is the double exposure (DE) method [18]. We use this method as a reference to all techniques described in this paper. DE method consists of capturing a hologram with no object in the FoV, calculating the phase map which should represent the phase error ϕa and finally subtracting this estimate from the phase calculated from hologram captured after introduction of an object into the FoV. The benefits of the method are the possible removal of all aberrations not related to the object, and that once the reference hologram has been captured, the method is com- pletely independent of the object. In many cases this method is feasible because objects, for example, cells are sparsely distributed. However in the case of densely populated cell cultures (see, Section 2.1) or tissues it is difficult or impossible to find a region with no objects. Here, we present our proposal how to deal with this problem as well we describe the methods which are used by other researchers.

In order to address the issue of DE method for densely populated samples we propose an averaged multiple exposure method (AME), that to the best of our knowledge is a novel method. It assumes the non-regular distribution of the object under investigation and specimen-shift invariance

(4)

of the undesired phase components. After acquisition of many single FoVs of the shifted sample and processing them to retrieve complex amplitude, all unwrapped phase components are averaged,¹ producing a single aberration phase map estimate cϕa with object information ϕo filtered out due to averaging. The b indicates the estimate of the true value. The estimate of the object phase is calculated as a simple difference:

cϕoðx; yÞ ¼ ϕtðx; yÞ cϕaðx; yÞ (2) whereϕtðx; yÞ is the total measured phase map.

In the case of periodic (or quasi-periodic) samples, additional caution is advised, as it is possible that the estimated aberration term cϕa might contain information about the objects. In such situation modifying the nominal overlap between consecutive measurements could resolve the issue.

The accuracy of the AME method is additionally affected by the dynamic range of phase values within the sample and the number of averaged measurements.

The method resembles the double exposure (DE) method [15], but without the requirement of finding a specimen-free region. The method is not limited to the stitching dataset, since the movement between consecutive averaged phase images does not have to follow a regular path and can be achieved manually. Averaging may be performed on-line, taking advantage of the recursive equation for the arithmetic mean. Our method, similarly to the DE method corrects not only slowly varying systematic error, but as well the high frequency disturbances that are present in all phase images and originate, for example, from the dust in the optical path of the DHM. The disadvantages of the method are to some extent object dependence and requirement of having ade- quately many measurements to average. The latter comes from the fact that the additional, unwanted information about objects is filtered at a rate of_N¹, where N is the num- ber of averaged frames. The method can be applied com- pletely a posteriori in the case of multiple measurements acquired with the same microscope setting. This condition is automatically fulfilled in the case of stitching scenario, but the method may be used to correct a single field of view, by measuring additional fields of view just for averaging.

Here, we describe some of the methods that we have chosen to compare with our approach, that is, AME method. For more comprehensive overview of phase aberration compensation methods we recommend an up-to-date paper [19]. Our constraints are the following: procedure needed to be applicable automatically, without the need for selecting specimen- free region and it should be applicable without readjustment

of the hardware. Methods that we have used for comparison are double exposure (DE) method [18], least-squares surface fitting (SF) methods [20] for a total reconstructed phase using either Zernike or Legendre polynomials decomposition, principal component analysis (PCA) method [21], and enhanced fast empirical modes decomposition (EFEMD) [22]. We con- sider the DE method to be the reference for other methods, as it is capable of removal of all types of phase disturbances.

SF method uses a set of orthonormal polynomials as a basis for reconstructing the aberrated phase image. This method works selectively on the aberrations by truncating high degree polynomials that are capable of reproducing higher frequencies, generally attributed to the object instead of slowly varying aberrations. The output of such reconstruction is subtracted from the original phase image, producing aberration free estimate of object phase map cϕoðx; yÞ.

In this work, we decided to use Legendre polynomials, as our preliminary tests did not yield any significant benefit of using Zernike polynomials, which are a more natural choice for the circular optics, but are less computationally efficient in the case of rectangular FoVs imposed by the CCD shape.

EFEMD is a fast implementation of 2D empirical modes decomposition (EMD) [22]. EMD is an adaptive approach designed for analysis of nonstationary and nonlinear data.

It decomposes a signal into a set of modes, called intrinsic mode functions (IMF). Each of the IMFs contains a limited spectral range and the higher the number of the mode, the lower the frequencies it contains. Sum of all IMFs fully reconstructs the original signal. The way we used EFEMD for aberration compensation was to truncate the number of modes, removing low frequency information. In principal, with EMD algorithms it is possible to remove the high frequency noise as well, but in preliminary tests it has been found that in the case of EFEMD algorithm this approach is too degrading for the objects high frequency features.

For this reason in the Table 1 aberrations removed by EFEMD have been described as slowly varying. It should be noted, that other EMD algorithms might represent better behavior in high frequency range.

PCA method states the problem of aberration correction as the problem of finding the first principal component of the exponential term of a complex amplitude modeled by:

Ψðx; yÞ ¼ Aðx; yÞ exp iϕ½ oðx; yÞ

exp iðk xxþ kyyÞ

exp iðl xx²þ lyy²Þ (3) where k_xand k_yare linear phase coefficients resulting from off-axis architecture, lxand lyare quadratic phase coefficients resulting from optical components. The last two exponential terms can be rewritten as Qðx; yÞ ¼ pq^H, wherefg^Hstands for a complex-conjugate transpose. The problem can be solved by applying singular value decomposition (SVD) algorithm. The linear coefficients kx and ky, as well as

1Alternatively, phase components in complex fields may be averaged taking into account properties of exponential function.

(5)

quadratic coefficients lxand lycan be found by least squares fitting of 1D quadratic function to the unwrapped left and right dominant singular vectors provided by the SVD algorithm.

All discussed methods are compared according to their features, summing up the descriptions given for each method in this section (see Table 1). As the most important features we identified object dependence, variability and type of removed aberrations. Object dependence describes if the objects optical path difference (OPD) and location within the FoV influence the retrieved aberration component. Variability describes if the detected aberration components may change between different FoVs. DE and AME have been described as partially object dependent, because the former depends on the quality of the single FoV that has been used as a reference and the latter depends on the dataset size, objects OPD and spatial distribution. However, once the aberration component has been established DE and AME do not vary when applied to different FoVs. We describe this as partial dependence, because the dependence is not as direct as in the other methods in which we calculate the aberration term for each individual FoV separately.

3.2 | Experimental evaluation

In this section, we evaluate the aberration correction methods described in Section 3.1. The methods are either graded individually or quantitatively compared to the DE method, as it is a widely used method capable of correcting not only aberrations originating form optical elements but as well other aberrations due to, for example, impurities in the system. For the experimental evaluation four images have been chosen from the set of images discribed in Section 2.1, try- ing to represent both sparsely and densely populated regions of the sample. For the DE method, the object-free measurement has been made in the marginal region of the sample.

According to our experience with the Ibidi μ-Dish we can safely assume that the measurement far from samples center will not cause any obstruction due to irregular optical properties of the dish. Please note that it is not the case for most commercial dishes. The AME method has been achieved by averaging of 120 images. The stitched subset of the images is the intersection of columns 8–17 and rows 8–19.

Applying the SF method we truncated the Legendre polynomials degree above two. Including the third degree of polynomials and higher has been visually evaluated on the exemplary images as too object dependent, especially in cases where cells did not densely populate the whole FoV, but still were present. Using EFEMD the decision about the number of truncated modes has been made visually on the exemplary images, similarly to the SF method. Throughout this work we truncated the last two modes. Table 2 shows the coordinates of chosen images within the grid of the dataset. Those images are presented in the Figure 2, together with the central horizontal cross sections through the aberration terms retrieved by all investigated methods.

The cross-sections in Figure 2 visualize the features presented in Table 1. The aberrations phase maps provided by DE and AME methods are the same throughout all examples, and AME method very closely follows the DE method, which indicates its capabilities for phase removal.

On the contrary, other methods display their changeability between examples. Additionally, the cross-sections show that both DE and AME are capable of removing not only low frequency, but also high frequency aberrations. In contrast SF, PCA and EFEMD address only the issue of low frequency terms that originate from optical components.

In the Table 3 all methods have been compared with the DE method. The comparison has been made by utilizing commonly used image similarity metrics—structural similarity index (SSIM) [23], mean square error (MSE) and peak signal-to-noise ratio (PSNR). SSIM is a similarity metric that aims to take into account not only per-pixel dif- ferences, but also images textures, and returns values in range from −1 to 1, where 1 represents perfect similarity.

MSE (ranges from 0 to ∞, the lower the value the better) and PSNR (ranges from 0 to ∞, the higher the value the T A B L E 1 Comparison of phase aberration removal methods according to their features

DE AME SF PCA EFEMD

Object dependence

Partial Partial Yes Yes Yes

Variability No No Yes Yes Yes

Removed aberrations

All present in the reference FoV

All recurring in the dataset

Slowly varying, expressed by the chosen basis

Slowly varying, expressed by the Equation 3

Slowly varying, no analytical form

T A B L E 2 Numbering of the exemplary images with their positioning in the dataset grid

Image number Grid coordinates (x, y)

1 (5, 16)

2 (10, 8)

3 (11, 15)

4 (14, 11)

(6)

better) represent image similarity based only on per-pixel dif- ferences between the images. According to the results of this comparison AME performed the closest to the DE method, reaching SSIM score over 0.7 for each example and for most other cases over an order of magnitude better in terms of MSE. The unfairness in the comparison is that the DE method is not guaranteed to give perfect results and therefore might not be a perfect reference. The unfairness mentioned above is visible when the total variations (TV) of the images processed with given methods are analyzed (see Table 4).

TV has been calculated according to the (4)

TV ¼ ∑^M₁∑^N₁kGðx; yÞk ¼ ∑^M₁∑^N₁ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi g²_xþ g²y

q

(4) where M and N are image dimensions,‖•‖ is magnitude of a vector, G is image gradient vector, gx and gy are scalar components of the gradient vector.

Figure 3 shows crucial aspects of aberration removal compared for each investigated method. Variation row shows a per-pixel representation of the ‖G‖. The AME method represents the lowest TV throughout all examples.

Furthermore, looking at the AME aberration component in Figure 3, it is visible that the lower variation is not related to the loss of object information, rather to the reduction of high-frequency noise.

In the case of image stitching, where the objects are stochastically distributed within the sample and the dataset is big, the disadvantages of the AME method are sufficiently reduced. Also, the additional benefit of reducing high-frequency noise has been presented. For those reasons the AME method has been chosen for the first step of phase images preprocessing before stitching.

(A) (B)

(C) (D)

1.0

0.5

0.0

–0.5

–1.0

–1.5

–2.0

1.0

0.5

0.0

–0.5

–1.0

–1.5

–2.0

0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350

1.0

0.5

0.0

–0.5

–1.0

–1.5

–2.0

0 50 100 150 200 250 300 350

1.0

0.5

0.0

–0.5

–1.0

–1.5

–2.0

0 50 100 150 200 250 300 350

DE AME SF PCA EFEMD

DE AME SF PCA EFEMD DE

AME SF PCA EFEMD

DE AME SF PCA EFEMD

F I G U R E 2 Cross-sections of obtained aberrations for different phase images using all compared methods. Comparison shows object- dependence and changeability of all purely numerical methods. Embedded images show phase images, before aberration removal. Sub-figures (A), (B), (C) and (D) are respectively phase images 1–4 from Table 2

(7)

4 | RESIDUAL RANDOM P HASE ABERRATIONS AND OFFSET CORRECTION

After removing the systematic phase aberrations present in raw images, we obtain a set of images that still contain some residual aberrations that have not been removed.

Those errors are most often the the result of inaccuracies in mechanical translation of the motorized table and mechanical vibrations of the sample and optical system arising due to the movement of translation stage, throughout the acquisition process.

Furthermore, the problem of the offset value between single FoVs is present nonetheless. One of the intrinsic properties of DHM is the relative nature of the measurements. This means that each single frame might deliver an unwrapped phase with arbitrary offset, which has to be corrected to perform proper stitching.

4.1 | Strategy for the removal of residual aberrations

The removal of residual aberrations is mostly necessary for either AME or DE methods. Their working principle assumes only the occurrence of systematic phase aberrations and thus removed aberrations are constant between different FoVs (see Table 1). However, since the rest of the methods perform with different biases due to variable object properties (see Table 1) it is beneficial to perform this processing step without considering the first aberration removal method. Because the majority of the aberrations are already removed at this point, segmentation techniques can be deployed effortlessly. This enables the reduction of the residual aberrations by using one of the variable

aberration removal methods. All of the variable methods are suitable for this task, however, due to easiness of deal- ing with the missing data we prefer to use SF method.

4.2 | Strategy for phase offset correction

To mitigate the problem we use the fact that the overlapping regions of neighboring images should in principle represent the same phase shift, but with different offset. We start by knowing the nominal overlap between the images, and crop out those regions. Then, the image phase correlation technique is used in order to find the additional lateral shift between the images. Once the overlapping regions have been found, we subtract the overlapping pixels and calculate mean value of the difference, that will indicate the offset between the frames. In the case of non-boundary frames the estimated offset is a mean of a difference between current image and three other images that have already been corrected. Proposed procedure is a sequential process—we iterate through the images and adjust them to the neighboring images that have already been corrected in previous iterations.

4.3 | Experimental evaluation

The experimental evaluation has been performed on the images treated with the AME method, as it has been shown to perform better than other methods in our tests (see, Sec- tion 3.2). In the Figure 4 we show the contribution of the strategies proposed in Sections 4.1 and 4.2. In Figure 4A it is shown that the sub-images display both erroneous tilt and are offset to different levels. Applying offset correction procedure (Figure 4B) it is shown that despite the overall improvement residual aberrations may cause accumulation of errors (top row) and obscure the result. If instead of offset correction the residual aberrations correction is applied we observe a dramatic improvement. In Figure 4C) it is shown that by applying the residual aberration correction, even the problem of offset value is mitigated to some extent. The final result can be improved even further, as shown in Figure 4, where the residual aberrations correction has been followed by the offset correction displays better contrast between the cells and the background.

T A B L E 3 Comparison between the phase image obtained using double-exposure method (reference method) and other tested methods

No. Metric AME SF PCA EFEMD

1 SSIM 0.7297 0.1442 0.1653 0.1695

MSE 0.0173 0.4657 0.4848 0.4227

PSNR 31.0726 16.7647 16.5901 17.1855

2 SSIM 0.7406 0.3895 0.3684 0.3592

MSE 0.0173 0.095 0.1327 0.1216

PSNR 32.8732 25.9795 25.5268 25.3385

3 SSIM 0.7247 0.2585 0.2402 0.2262

MSE 0.0173 0.169 0.2187 0.2079

PSNR 30.0456 20.1225 19.9032 19.7454

4 SSIM 0.7154 0.4033 0.4131 0.3801

MSE 0.0173 0.0764 0.1407 0.091

PSNR 29.1526 23.9011 23.0554 22.9173

T A B L E 4 Comparison of the TV of the processed images

Number DE AME SF PCA EFEMD

1 10 348.0 8486.0 13 798.0 13 799.0 13 786.0 2 12 171.0 11 647.0 16 047.0 16 051.0 16 046.0 3 12 132.0 9408.0 14 635.0 14 642.0 14 649.0 4 11 757.0 9949.0 14 777.0 14 775.0 14 781.0

(8)

Figure 5 illustrates better the influence of offset correction on the data with removed main and residual aberrations. The local cross-sections of phase maps shown in Figure 4C, D show that the offset correction removes even a small offset difference which in this case equals approx. 0.15 rad. In the presented example the offset correction has small values, however even as such it will influence the quantitative analysis of the sample. In the case of bigger object-background phase difference the offset influence is more significant and therefore this correction must be taken into account.

5 | STITCHING

The stitching procedure that we used is a globally opti- mized Grid/Collection stitching Fiji plugin [14]. The algorithm first performs pairwise image registration using phase correlation method and secondly runs a global optimization procedure on the previously found image displacements.

Additionally, the algorithm blends the overlapping regions of the images, either by averaging, linear blending or nonlinear blending, based on the location of each pixel in the overlapping region. In the previous Sections (3, 4.1, 4.2)

processing path design has been justified by theoretical analysis as well as experimental analysis. In this subsection examples of utilizing whole processing path are presented and final conclusions are drawn.

Figure 6 shows the results of proposed phase map stitching procedure. The white regions visible in the images are the regions with the lack of data, that occur due to one of two reasons. First is the relative rotation around optical axis between motorized stage and the CCD camera, which leads to the apparent sloped but not rotated movement of the object on the CDD plane. Second reason is due to the presence of gross unwrapping errors which are caused by rapid phase change in the object or the local absorption in the sample. The errors are masked in order to protect the phase preprocessing from the faulty influence of the corrupted data. They are detected by simple thresholding in the gradient domain.

6 | CONCLUSIONS AND FUTURE WORKS

In this work, we have proposed a complete processing path that enables stitching of phase images in order to obtain

DE AME SF PCA EFEMD

0 100

200

0 100

200

0 100

200

0 100

200

2 1 0 –1 –2 –3

1.0 0.50.0 –0.5–1.0 –1.5–2.0 –2.5

1.00.5 0.0 –0.5–1.0 –1.5–2.0 –2.5

1.0 0.5 0.0 –0.5–1.0 –1.5–2.0 –2.5

2.0 1.51.0 0.50.0 –0.5 –1.0

2.0 1.5 1.0 0.50.0 –0.5 –1.0

0.30 0.25 0.20 0.15 0.10 0.05

0 200 0 200 0 200 0 200 0 200

Raw imageAberrationCleared imageVariation

F I G U R E 3 Comparison of different aberration compensation methods at the example of image 4 (Table 2). Each column represents different method. In the top row the same aberrated (raw) phase image is presented multiple times for the visual clarity. In the second row calculated aberrations maps are presented. Third row displays the phase image with removed aberrations components. Fourth row visualizes per- pixel variation in the processed image. All images within each of the first three rows have been offset to the same phase level. For each row the displayed values range has been set to the value range of the first image on the left (DE method)

(9)

image with FoV many times bigger than the single FoV of a microscope. It has been shown that presented processing path leads to effortless automatic stitching of phase images that does not require the selection of object-free region.

This has been achieved by the novel method for aberrations correction, namely averaged multiple exposure (AME). We showed that AME is the most suitable method in the case of phase images stitching with randomly distributed cells.

Additionally, two steps supporting AME have been introduced - residual aberrations removal through surface fitting

methods aided by segmentation, as well as offset correction method by comparison of overlapping regions of the neighboring phase images. Our work has shown high effective- ness of the proposed processing path and its high applicability for analysis of extended field of view cell cultures investigation. The further works should be focused on testing the applicability of the method for quantitative phase analysis of histopathological samples as the basis for future development of digital phase pathology and virtual microscopy of transparent large area samples.

(μm) 200

(μm)

200 400 600 800

(μm)

200 400 600 800

400

600

800

7

6

5

4

3

2

1

0

6

5

4

3

2

1

0 200

(μm) 400

600

800

0 0.5 1.0 1.5 2.0

0 0.5 1.0 1.5 2.0 200

(μm)

400

600

800

200

(μm)

400

600

800

(μm)

200 400 600 800

(μm)

200 400 600 800

(A) (B)

(C) (D)

F I G U R E 4 Stitched images obtained with different combinations of methods addressing problems of residual phase aberrations and phase offset. In all images the main aberrations are removed by averaged multiple exposure (AME) method. (A) no further processing after main aberration removal, (B) with offset correction, (C) with residual aberration correction, (D) with both offset and residual aberration correction.

Green lines in (C) and (D) indicate the cross-section visualized in Figure 5

(10)

A C K N O W L E D G M E N T S

The research leading to the described results was carried out within the program TEAM TECH/2016-1/4 of Founda- tion for Polish Science, co-financed by the European Union under the European Regional Development Fund. The authors would like to acknowledge the support from the statutory funds and the grants of the Dean of Mechatronics Faculty, Warsaw University of Technology. We would like to thank the researchers from the Laboratory of Imaging Tissue Structure and Function, Nencki Institute of Experi- mental Biology, Polish Academy of Science for the prepa- ration of the sample that we demonstrated throughout the study.

F I N A N C I A L D I S C L O S U R E None reported.

C O N F L I C T O F I N T E R E S T

The authors declare no potential conflict of interests.

A U T H O R C O N T R I B U T I O N S

Piotr Stępień programmed the phase retrieval path, designed and programmed the processing path described in this article, including the AME method as well as drafted the paper.

Damian Korbuszewski designed and programmed the software with GUI for holograms acquisition and hardware con- trol. Małgorzata Kujawińska, as a head of the group, provided the conception for the work, reviewed it multiple times throughout the working process, including hardware and software decisions and the final editing process.

O R C I D

Piotr Stępień http://orcid.org/0000-0002-7604-4953 Damian Korbuszewski https://orcid.org/0000-0003-0213- 4988

Małgorzata Kujawińska https://orcid.org/0000-0001- 6521-6951

R E F E R E N C E S

1. M. K. Kim, Principles and techniques of digital holographic microscopy, SPIE Rev. 1 (2010), no. 1, 018005.

2. B. Kemper and G. von Bally, Digital holographic microscopy for live cell applications and technical inspection, Appli.Opt. 47 (2008), no. 4, A52–A61.

0 50 100 150 200

μm

rad

2.5

2.0

1.5

1.0

0.5

0

With offset correction Without offset correction

F I G U R E 5 Vertical cross section through the region influenced by the offset correction procedure. Cross-section line has been indicated in the Figure 4C, D

200

(μm)

200

400 600 800

400

600

800

2.00

1.75

1.50

1.25

1.00

0.75

0.50

0.25

0

(A)

200

(μm)

400 600 800

(B)

1,000 0

0

200

400

600

800

1,000

(μm)

0.0 0.5 1.0 1.5 2.0

F I G U R E 6 Stitched images using full processing path.

(A) Comprises of images with indices in range x 3-8 and y 13-20.

(B) Comprises of images with indices in range x 8-15 and y 10-19

(11)

3. S. Al-Janabi, A. Huisman, and P. J. Van Diest, Digital pathology:

current status and future perspectives, Histopathology 61 (2012), no. 1, 1–9.

4. O. Ordi et al., Virtual microscopy in the undergraduate teaching of pathology, J. Pathol. Inform. 6 (2015), 1.

5. A. Greenbaum et al., Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy, Sci. Rep. 3 (2013), 1717.

6. G. Zheng et al., Sub-pixel resolving optofluidic microscope for on-chip cell imaging, Lab Chip 10 (2010), no. 22, 3125– 3129.

7. J. M. Rodenburg and R. H. T. Bates, The theory of super- resolution electron microscopy via wigner-distribution deconvo- lution, Phil. Trans. R. Soc. Lond. A 339 (1992), no. 1655, 521–553.

8. G. Zheng, R. Horstmeyer, and C. Yang, Wide-field, high-resolu- tion fourier ptychographic microscopy, Nat. Photonics 7 (2013), no. 9, 739.

9. Philips intellisite ultra fast scanner, available at https://www.philips.co.uk/healthcare/product/hcnoctn442/intellisite-ultra-fast-scanner.

10. P. L. Makowski et al., Synthetic aperture fourier holography for wide-angle holographic display of real scenes, Appl. Opt. 54 (2015), no. 12, 3658–3665.

11. P. Sidorenko and O. Cohen Single-shot ptychography, Optica 3 (2016), no. 1, 9–14.

12. W. Zhou et al., Study on phase stitching technique on digital holography, Digital Holography and Three-Dimensional Imaging, Optical Society of America, 2009, pp. DWB23.

13. Y. Wen et al., Further investigation on the phase stitching and system errors in digital holography, Appl. Opt. 54 (2015), no. 2, 266–276.

14. S. Preibisch, S. Saalfeld, and P. Tomancak, Globally optimal stitching of tiled 3d microscopic image acquisitions, Bioinformat- ics 25 (2009), no. 11, 1463–1465.

15. M. Takeda, H. Ina, and S. Kobayashi, Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry, J. Opt. Soc. Ame. 72 (1982), no. 1, 156–160.

16. M. A. Herráez et al., Fast two-dimensional phase-unwrap- ping algorithm based on sorting by reliability following a noncontinuous path, Appl. Opt. 41 (2002), no. 35, 7437– 7444.

17. P. Stępień et al., Multi-modal quantitative analysis of hela cells using digital holographic microscopy and confocal laser scanning microscopy, Int. Conf. Speckle Metrology, Janów Podlaski, Poland, Sept. 10–12, 2018, pp. 108341V:1–7.

18. P. Ferraro et al., Compensation of the inherent wave front curva- ture in digital holographic coherent microscopy for quantitative phase-contrast imaging, Appl. Opt. 42 (2003), no. 11, 1938– 1946.

19. Y. Liu, Z. Wang, and J. Huang, Recent progress on aberration compensation and coherent noise suppression in digital hologra- phy, Appl. Sci. 8 (2018), no. 3, 444.

20. J. Di et al., Phase aberration compensation of digital holographic microscopy based on least squares surface fitting, Optics Com- mun. 282 (2009), no. 19, 3873–3877.

21. C. Zuo et al., Phase aberration compensation in digital holo- graphic microscopy based on principal component analysis, Opt.

Lett. 38 (2013), no. 10, 1724–1726.

22. M. Trusiak, M. Wielgus, and K. Patorski, Advanced processing of optical fringe patterns by automated selective reconstruction and enhanced fast empirical mode decomposition, Opt. Lasers Eng. 52 (2014), 230–240.

23. Z. Wang et al., Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. 13 (2004), no.

4, 600–612.

A U T H O R B I O G R A P H I E S

Piotr Stępień is a PhD student and a member of “BiOpTo”

project team. His scientific interests are digital holographic microscopy and optical diffrac- tion tomography with applications in biomedicine.

Damian Korbuszewski is an undergraduate student and a member of “BiOpTo” project team. His focus is front-end design and software implementation.

Małgorzata Kujawińska PhD, DSc., is a full professor of applied optics at Warsaw University of Technology.

Expert in full-field optical metrology, optonumerical methods in mechanics, image processing, automatic data analysis, and design of innovative photonic systems. She has been involved in optical metrology topics since 1980, including development of interferometric, holographic, grating, digital image correlation, and structured light-based methods. She is an author of one monograph, several book chapters, and more than 200 papers in interna- tional scientific journals. She is an SPIE fellow.