The algorithm stitching for medical imaging

E. Semenishchev^a, V. Marchuk^a, V. Voronin^a, M. Pismenskova^a, I. Tolstova^a, I. Svirin^b
aDept. of Radio-Electronics Systems, Don State Technical University, Gagarina 1, Rostov-on-Don,
Russian Federation, 344010,
^bCJSC Nordavind, Moscov, Russian Federation.

ABSTRACT

In this paper we propose a stitching algorithm of medical images into one. The algorithm is designed to stitching the medical x-ray imaging, biological particles in microscopic images, medical microscopic images and other. Such image can improve the diagnosis accuracy and quality for minimally invasive studies (e.g., laparoscopy, ophthalmology and other). The proposed algorithm is based on the following steps: the searching and selection areas with overlap bomdaries; the keypoint and feature detection; the preliminary stitching images and transformation to reduce the visible distortion; the search a single unified borders in overlap area; brightness, contrast and white balance converting; the superimposition into a one image. Experimental results demonstrate the effectiveness of the proposed method in the task of image stitching.

Keywords: medical image, image stitching.

1. INTRODUCTION

Algorithm for aligning images and stitching them into seamless photo-mosaic are among the oldest and most widely used in computer vision. Image stitching or photo stitching is the process of combining multiple photographic images with overlapping fields of view to produce a segmented panorama or high-resolution image. The medical applications such task are: radiology (combining X-ray images), MRI and KT tomography, microbiology (images of microorganisms), genetics (micro images nucleic acid), clinical diagnostic biology (the study of cells and substances in the studied biological material), pulmonology (pictures lung), hematology (blood tests), and many other areas of medical imaging.

In the research literature methods for automatic image alignment and stitching fall broadly into two categories - direct and feature based. Direct methods have the advantage that they use all of the available image data and hence can provide very accurate registration, but they require a close initialisation. Feature based registration does not require initialisation, but traditional feature matching methods (e.g., correlation of image patches around Harris comers lack the invariance properties needed to enable reliable matching of arbitrary panoramic image sequences.

For medical image stitching there are different approaches [1-7]. These approaches are quite effective if the object is temporarily immobile or its rate of motion less the shooting speed of the entire space. Also in the case of filming with automated systems that do not allow mark the exact position of the fixing of the matrix with respect to the object under study. In this case the quality of the united image falls sharply, and in some cases require the participation of the operator to solve this problem. Additional restriction for these methods is also a difference in focal lengths, degree of illumination, established white balance, a small overlap area and other. In the case of studies X-ray images the keypoints are absent. Figure 1 shows an example of X-ray images, as well as an example of image capturing using a microscope.

As seen from the examples of frequently changing position fixing matrix is a change in distances for passing reference light (x-ray), which leads to the appearance of geometric distortions. At the same time, it can change the shutter speed and the degree of exposure. These changes affect the result of the stitching.

In this paper we describe an invariant feature based approach to fully automatic medical image stitching with geometric transformations and imperceptible transition to the overlap area. The rest of the paper is organized as follows. The proposed image stitching method is described in section 2. Section 3 presents experimental results and conclusions are given in section 4.

Figure 1. Some troubles of medical images.

2. THE PROPOSED METHOD

2.1 The image model
A graphical representation of the image stitching can be represented on the figure 2 as follows: is the first image with the stitching area; is the first image; is the second image; is the stitching area; is the second image with the stitching area; is the object on image [8].

Figure 2. The image stitching model.

A simplified mathematical model of the stitching image can be represented as:

where: d is the coefficient compensating type of distortion.

The type of distortion compensation coefficient can be both simple and composite, allowing the suppression of the main types of distortion caused by: the difference object distance shooting sensitive matrix; used the properties of lenses and the inaccuracy of their manufacture; dimensions and shapes of the objects, and others.

2.2 Image denoising method using a combined criterion

We summarize the algorithm of medical image stitching in the following scheme (Fig. 3).

Figure 3. The algorithm of medical image stitching.

Below we will describe all building blocks of the proposed algorithm.

a) The first step is loading a pair of images.

b) On the next step, manual or automatic mode. In the first case an operator select the points which should use for stitching. In the second case:

b.1) We make an exception the background image from consideration. As a method for determining the background can be used the approaches based on the analysis of the boundaries of objects and their mutual arrangement [9]. It is also possible to use trained neural network, a threshold method and methods based on human saliency map.

b.2) The next step is search the objects boundaries and borders in the internal structure of these objects. There is stepwise process. We use tiered exception of analysis the found in the previous step the comers of objects and areas of the border. At each step, we define small (3-7 pixels) individual boundaries and their distance to the comers. This information we use for construct a graph tree.

b.3) We find the scalable small areas, for which the distances are constant and the same for the pair of images. The search is conducted on the analysis of the resulting graph tree. According to analysis of the elements and structure of the objects, we will highlight the areas with the close proximity of the borders or comers. Later, for stitching images, elements from these areas will have a maximum weight.

b.4) To increase the accuracy in the choice of keypoints we make search of correspondences between the elements of the base area using the correlation analysis. As a point of analysis the center of mass within the base area for each of the respective pairs is used. We consider only the area with the maximum correlation.

c) Further above the pair of images we use the projective transformations. The main condition, the maximum

d) In the case of fuzzy boundaries in enclosed object, we use algorithm for centerline search (Fig. 4).

Figure 4. The algorithm for centerline search.

This algorithm is based on the counter-contraction of the external and internal borders blurred object. Some examples of border recovery are shown in Figure 5.

a)	b)	c)	d)

Figure 5. The border recovery.

e) On next step applies a gradient approach to stitching images to eliminate the visible transition between the images in the case of differences in illumination of objects or the difference in exposure. As a curve the transition uses natural boundaries of objects arranged on a pair of frames. The criteria for a given curve include the condition of maximum direction of the curve to the center of stitching region and passing along the borders of the objects. The example of the curve separation is shown in figure 6.

a)

b)

c)

Figure 6. The example of the curve separation.

f) The color correction applied to the selected images. For limitation the area we choose a pair of identical square areas around the reference points and calculate the histogram of brightness for them (Fig. 7). We define the position of the maxima of Mstogmm data and cany out correction of the pair of output images. The connection is made multiplying at a certain constant for each of the channels. For an area without objects apply a gradient smoothing.

a)	b)

Figure 7. The histogram of brightness for color images,

g) On the final step the images combined into a single content.

3. EXPERIMENTAL RESULTS

In Figure 8 examples of image stitching for pairs of medical images are given. The presented images are both monophonic 8 bit (x-ray) and colors 16 bit (microbiological).

a)

b)

c)

Figure 8. The test pairs of medical images.

Figure 9 presents the results of image stitching using proposed method. The algorithm detected connected components of image matches and unmatched images, and output blended panoramas. The proposed method is robust for camera zoom, orientation of images, and changes in illuminations.

a)

b)

c)

Figure 9. The results of the images stitching.


CONCLUSION

This paper has presented a novel algorithm for fully automatic stitching of medical images. Experimental results demonstrate the effectiveness of the proposed method in the task of image stitching. The proposed method is robust to camera zoom, orientation of the input images, and changes in illumination due to flash and exposure/aperture settings.

ACKNOWLEDGMENTS

The reported study was supported by the Russian Foundation for Basic research (RFBR), research project №15-07- 08300U6.

REFERENCES

1. Gooiien, Andre, et al. "A stitching algorithm for automatic registration of digital radiographs." Image analysis and recognition/Springer Berlin Heidelberg, 854-862 (2008)
2. Kumar, Abhinav, et al. "Automatic image alignment and stitching of medical images with seam blending." World Academy of Science/Engineering and Technology, 65 (2012)
3. Singla, Savita, and Reecha Sharma. "Medical Image Stitching Using Hybrid Of Sift & Surf Techniques." ISSN: 2278 - 909X. International Journal of Advanced Research in Electronics and Communication Engineering (IJARECE), Volume 3, 838 - 842 (2014)
4. Mateika, D+, and R. Martavicius. "A Fully Automatic Stitching of 2D X-ray Images." Elektronika ir Elektrotechnika Volume 80.8, 3-8 (2015)
5. Chen, Cheng, et al. "Ruler based automatic C-arm image stitching without overlapping constraint." Journal of digital imaging Volume 28.4, 474-480 (2015)
6. Gramer, Markus, et al. "An algorithm for automatic stitching of CR X-ray images." Advances in Medical Engineering. Springer Berlin Heidelberg, 193-198 (2007)
7. Sedehzadeh, Mahan and Fardad Farokhi. ,’Image Stitching of the Computed Radiology images Using a Pixel- Based Approach." 77-84(2013)
8. Semenishchev, E. A., et al. "Stitching algorithm of the images acquired from different points of fixation." IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, (2015)
9. Semenishchev, E. A., et al. "Investigation of methods to search for the boundaries on the image and their use on lung hardware of methods finding saliency map." SPIE Sensing Technology+ Applications. International Society for Optics and Photonics, (2015)