Tayyaba Naeem,1) Shamweel Qamar,3) Peom Park1),2),3),4)*
1) System Engineering, Ajou University 2) Industrial Engineering, Ajou University
3) System Biomedical Informatics 4) Humintec, Co. Ltd.
Abstract : Pathology is the motor that drives healthcare to understand diseases. The way pathologists diagnose diseases, which involves manual observation of images under a microscope has been used for the last 150 years, it’s time to change. This paper is specifically based on tumor detection using deep learning techniques. Pathologist examine the specimen slides from the specific portion of body (e-g liver, breast ,prostate region) and then examine it under the microscope to identify the effected cells among all the normal cells. This process is time consuming and not sufficiently accurate. So, there is a need of a system that can detect tumor automatically in less time. Solution to this problem is computational pathology: an approach to examine tissue data obtained through whole slide imaging using modern image analysis algorithms and to analyze clinically relevant information from these data. Artificial Intelligence models like machine learning and deep learning are used at the molecular levels to generate diagnostic inferences and predictions; and presents this clinically actionable knowledge to pathologist through dynamic and integrated reports. Which enables physicians, laboratory personnel, and other health care system to make the best possible medical decisions.
I will discuss the techniques for the automated tumor detection system within the new discipline of computational pathology, which will be useful for the future practice of pathology and, more broadly, medical practice in general.
Key Words : Artificial Intelligence, Machine Learning, Deep Learning Techniques, Medical Image Segmentation, CNNs
1. Introduction
Computational pathology is a term that starting to become more common in the pathology field as computer processing power is being applied to managing, analyzing, and interpreting slide images. Once a slide image is in the digital realm, it becomes possible to apply machine learning and artificial intelligence techniques to the digital pathology images. Automation involves computer-assisted software helping to speed up workflow and making it easier for a pathologist to work with slide images see Figure 1. Yet the bulk of practitioners in the field of path- ology are just now starting to catch up with regard to automation. This paper is based on an automated image recognition system that identify slides with precancerous or cancerous cells enable pathologists to spend more of their time on the slides whose diagnosis requires greater attention.
Medical image segmentation, identifying the pixels of cancerous cells from background images such pathological slides. It is one of the most challenging tasks in medical image
analysis to deliver critical information about the shapes and volumes of these tumors. Then, machine learning approaches extracting useful features(cell type, effected cells, healthy cells) have become a dominant technique for a long period. The promising ability of deep learning approaches has put them as a primary option for image segmentation, and in particular for medical image segmentation. In this review, the detailed process of deep learning–based pathology image segmentation is described, including data preparation, model selection and construction, post-processing, and feature extraction and association with disease.
1.1 Network Structure
Convolutional Neural Networks (CNNs):
The application of deep learning algorithms in pathology image analysis is the focus of this paper. Convolutional neural networks (CNNs) are introduced, which have been widely used for image classification and pathology image analysis, such as tumor region. A CNN is a branch of neural networks and consists of a stack of layers see Figure 2, each layer performing a specific operation, e.g., convolution, pooling, loss calculation, etc. Each intermediate layer receives the output of the previous layer as its input .The beginning layer is an input layer, which is directly connected to an input image with the number of neurons equal to the number of pixels in the input image.
The next set of layers are convolutional layers that present the results of convolving a certain number of filters with the input data and perform as a feature extractor. The filters, commonly known as kernels, are of arbitrary sizes, defined by designers, and depending on
[Figure 1] Flow chart of pathology image analysis
the kernel size. The output of each convolution layer is considered as an activation map, which highlights the effect of applying a specific filter on the input. The next layer can be a pooling layer depending on the design and it helps to
reduce the dimensionality of the convolution’s output. Lastly, high-level abstractions are extracted by fully connected layers. The weights of neural connections and the kernels are continuously optimized during the procedure of a back propagation in the training phase.
2. Data Preparation
Pathology images are usually as large as giga-pixels. Because the memory associated with a central processing unit (CPU) is limited, the pathology images need to be chopped into small patches see Figure 2. Common image patch sizes range from 256 × 256 to 512 × 512 pixels. The size is often chosen to be divisible by 2 to avoid the trouble of padding for pooling layers. Here, padding means adding pixels to the upper, bottom, left, and right
[Figure 2] Structure of CNN
[Figure 3] The structure of an example segmentation neural network. The input is a 512 × 512 × 3 hematoxylin and eosin stained lung cancer pathology image from the National Lung Screening Trial (NLST)
sides of the image, respectively. The value of extra pixel is set to be zero, so it is also called zero padding. In image segmentation neural networks do not require input images to be of the same size, it is preferable for the images to be cropped into the same size to speed up the algorithm and to fully utilize parallel computations on CPU. When the input image patches are padded, simply removing the padding region from the segmentation output can yield a result of the same size as the original image. Data preparation consist of two steps; first is padding input image to make it prepare for neural network training show with green line then second step is training of neural network show with an orange line in Figure 3. Final blue line shows the application and processing of an image.
3. Model Selection
3.1 Software Selection
Python is the most approachable way to implement deep learning algorithms. There are several open-source Python libraries to choose from: Scikit, Caffe, 29 TensorFlow, Keras and some software, such as Aperio GENIE which incorporate a machine learning–based segmentation function. Scikit is the most advance inbuilt library for image segmentation. The skimage data module contains some inbuilt example data sets which are generally stored in jpeg or png format has also a function to import images. These software tools greatly decrease the coding effort, but the cost is that the models are less flexible in regard to both model structure and training phase. We have an idea about scikit-image. After choosing the
software model the next step is finding suitable technique for Image Segmentation. Image Seg- mentation is essentially the process of partitioning a digital image into multiple segments to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze.
3.2 Image Segmentation Technique Selection Training a neural network for image segmen- tation is a supervised learning process. Thus, to construct a training set for segmentation, the next step is to manually label the ground truth see Figure-1. Pathology expertise is essential in this step. Several tools are available for annotating images, and their features as described in software selection we will prefer scikit for this purpose. On the basis of the design, these models can be divided into two main categories: semantic segmentation and instance segmentation Figure 4. It is worth noting that either semantic or instance segmen- tation can be converted into each other through twisting the models, including changing the prediction target and adding post-processing steps, so the models are classified here according to their original implementation. Semantic seg- mentation will be more suitable for tumor detection because it give label to each cell which helps in identifying tumor cells. The goal of semantic segmentation is to segment image parts with different meanings. The first end-to-end and pixel-to-pixel semantic seg- mentation neural network is the Fully Convolutional Network (FCN). In FCN, the last fully con- nected layer in CNN is replaced with a deconvolutional layer to efficiently classify each pixel. The summation of deconvolutional
layers and pooling layers enables FCN to do fine structure segmentation with respect to coarse structure information. Different modifications have been made to FCN to further improve the segmentation performance. For example U-net because greatly increases the number of deconvolutional layers to propagate information to higher resolutions.
4. Encoder Backbone Selection
In addition to the overall model structure design, another important step of model selection is to determine the encoder backbone structure (Figure 5). An encoder generally refers to the part of a neural network that encodes the large inputs to a feature map of much smaller size. Several problems can arise when a network goes deep, such as gradient vanishing and degradation of training accuracy. To efficiently use limited parameters out of many encoders InceptionV3 and ResNet have been reported as useful for pathology image analysis.
4.1 Loss Function
The loss function of a deep learning network quantifies the difference between the neural
network output and desired behavior. The training phase is essentially a process to minimize loss by adjusting the network parameters iteratively.
For semantic segmentation, the most common loss function is pixel-wise cross-entropy between the network outputs and the true segmentation annotations. To further improve the segmentation for adjacent objects (e.g. touching cells), Ronneberger proposed to upweight the cross- entropy loss for the pixels on the separation border.
5. Training Phase
The training phase is a process to update model parameters and is composed of alternating forward and backward propagations Figure 3.
[Figure 4] Example of nuclei segmentation in a pathology image. Input is a hematoxylin and eosin stained image patch (left panel) from the National Lung Screening Trial (NLST)
[Figure 5] Illustration of encoder and decoder network for semantic segmentation
Forward propagation uses network parameters to compute model outputs ,whereas backward propagation updates the network parameters according to their partial derivatives of the loss function. One forward and one backward propagation construct one training step.
5.1 Algorithm of Neural Network Training Phase
1. Input: initial model parameters W(0), loss function L, input I, ground truth GT, gradient descent algorithm A, batch size B, stopping criteria C, training step t = 1.
2. while C is not meet do
3. Randomly sample B samples, get IB and GTB
4. Forward propagation using W(t−1), yield prediction PB(t)
5. Calculate loss l(t) = L(PB(t), GTB) 6. Update W(t) = A(W(t−1), l(t))
7. (Optional: update hyperparameters, A and B) 8. t = t + 1
9. end while
6. Conclusion
Pathology image segmentation using AI tech- nology results can facilitate pathologists in locating Region of Interest ROI quickly by extracting features from segmented regions and correlating these features with pathologic, genomic, and clinical variables. Systematic an- alysis on these variables can provide additional ways to discover clinically meaningful features and to aid in diagnosis and treatment planning of tumor.
6.1 Future direction and challenges
My direction is to improve the segmentation performance of pathological image slide. As we know that pathological data is in large size which cause delay in segmentation process.
Now it’s a challenge to accelerate the giga- pixel image processing speed.
Acknowledgement
This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number : HI18C0316).”
References
1. Shidan Wang, Donghan M.Yang, RuichenRong, XiaoweiZhan, GuanghuaXiao, Pathology Image Analysis Using Segmentation Deep Learning Algorithms, The American Journal of Path- ology, Volume 189, Issue 9, September 2019, Pages 1686-1698.
2. Mohammad Hesam Hesamian, Deep Learning Techniques for Medical Image Segmentation:
Achievements and Challenges, Journal of Digital Imaging, August 2019, Volume 32, Issue 4, pp 582–596.
3. Hye Yoon Chang, Chan Kwon Jung, Junwoo Isaac Woo, Sanghun Lee, Joonyoung Cho, Sun Woo Kim, and Tae-Yeong Kwak, Artificial Intelligence in Pathology, J Pathol Med. 2019 Jan; Published online 2018 Dec 28.
4. Dr. David N. Louis, Computational Pathology, Arch Pathol Lab Med. 2016 Jan; 140(1): 41–
50. Published online 2015 Jun 22.
5. HowardLee, Yi-Ping Phoebe Chen, Image based computer aided diagnosis system for cancer detection, Expert Systems with Applications Volume 42, Issue 12, 15 July 2015, Pages 5356-5365.
6. Aurelle Tchagna Kouanou, An optimal big data workflow for biomedical image analysis,
Informatics in Medicine Unlocked Volume 11, 2018, Pages 68-74.
7. David West Jr, Digital Pathology Gives Rise to Computational Pathology, Published on October 2, 2017.
8. The Digital Future of Pathology, The Medical Futurist,10 May 2018.