In biomedical imaging using video microscopy, understanding large tissue structures at cellular and finer resolution poses many image acquisition challenges including limited field-of-view and tissue dynamics during imaging. Automated mosaicing or stitching of live tissue video microscopy enables the visualization and analysis of subtle morphological structures and large scale vessel network architecture in tissues like the mesentery. But mosacing can be challenging if there are deformable, motion-blurred, textureless, feature-poor frames. Feature-based methods perform poorly in such cases for the lack of distinctive keypoints. Standard single block correlation matching strategies might not provide robust registration due to deformable content. In addition, the panorama suffers if there is motion blur present in a sequence. To handle these challenges, we propose a novel algorithm, Deformable Normalized Cross Correlation (DNCC) image matching with RANSAC to establish robust registration. Besides, to produce seamless panorama from motion-blurred frames we present gradient blending method based on image edge information. The DNCC algorithm is applied on Frog Mesentery sequences. Our result is compared with PSS/AutoStitch [1, 2] to establish the efficiency and robustness of the proposed DNCC method.

Ideally, to carry out screening for eye diseases, it is expected to use specialized medical equipment to capture retinal fundus images. However, since this kind of equipment is generally expensive and has low portability, and with the development of technology and the emergence of smartphones, new portable and cheaper screening options have emerged, one of them being the D-Eye device. When compared to specialized equipment, this equipment and other similar devices associated with a smartphone present lower quality and less field-of-view in the retinal video captured, yet with sufficient quality to perform a medical pre-screening. Individuals can be referred for specialized screening to obtain a medical diagnosis if necessary. Two methods were proposed to extract the relevant regions from these lower-quality videos (the retinal zone). The first one is based on classical image processing approaches such as thresholds and Hough Circle transform. The other performs the extraction of the retinal location by applying a neural network, which is one of the methods reported in the literature with good performance for object detection, the YOLO v4, which was demonstrated to be the preferred method to apply. A mosaicing technique was implemented from the relevant retina regions to obtain a more informative single image with a higher field of view. It was divided into two stages: the GLAMpoints neural network was applied to extract relevant points in the first stage. Some homography transformations are carried out to have in the same referential the overlap of common regions of the images. In the second stage, a smoothing process was performed in the transition between images.

Confocal laser scanning enables optical sectioning in fiber bundle endomicroscopy but limits the frame rate. To be able to better explore tissue morphology, it is useful to stitch sequentially acquired frames into a mosaic. However, low frame rates limit the maximum probe translation speed. Line-scanning (LS) confocal endomicroscopy provides higher frame rates, but residual out-of-focus light degrades images. Subtraction-based approaches can suppress this residue at the expense of introducing motion artifacts.
To generate high-frame-rate endomicroscopy images with improved optical sectioning, we develop a high-speed subtraction method that only requires the acquisition of a single camera frame.
The rolling shutter of a CMOS camera acts as both the aligned and offset detector slits required for subtraction-based sectioning enhancement. Two images of the bundle are formed on different regions of the camera, allowing both images to be acquired simultaneously.
We confirm improved optical sectioning compared to conventional LS, particularly far from focus, and show that motion artifacts are not introduced. We demonstrate high-speed mosaicing at frame rates of up to 240 Hz.
High-speed acquisition of optically sectioned images using the new subtraction based-approach leads to improved mosaicing at high frame rates.

OBJECTIVE: The study of small vessels allows for the analysis and diagnosis of diseases with strong vasculopathy. This type of vessels can be observed non-invasively in the retina via fundoscopy. The analysis of these vessels can be facilitated by applications built upon Retinal Image Registration (RIR), such as mosaicing, Super Resolution (SR) or eye shape estimation. RIR is challenging due to possible changes in the retina across time, the utilization of diverse acquisition devices with varying properties, or the curved shape of the retina.
METHODS: We employ the Retinal Image Registration through Eye Modelling and Pose Estimation (REMPE) framework, which simultaneously estimates the cameras\' relative poses, as well as eye shape and orientation to develop RIR applications and to study their effectiveness.
RESULTS: We assess quantitatively the suitability of the REMPE framework towards achieving SR and eye shape estimation. Additionally, we provide indicative results demonstrating qualitatively its usefulness in the context of longitudinal studies, mosaicing, and multiple image registration. Besides the improvement over registration accuracy, demonstrated via registration applications, the most important novelty presented in this work is the eye shape estimation and the generation of 3D point meshes. This has the potential for allowing clinicians to perform measurements on 3D representations of the eye, instead of doing so in 2D images that contain distortions induced because of the projection on the image space.
CONCLUSIONS: RIR is very effective in supporting applications such as SR, eye shape estimation, longitudinal studies, mosaicing and multiple image registration. Its improved registration accuracy compared to the state of the art translates directly in improved performance when supporting the aforementioned applications.

Optical biopsy methods, such as probe-based endomicroscopy, can be used for the identification of early mucosal dysplasia in various gastrointestinal conditions and have potential applications in the screening of early-stage gastric cancer in vivo. However, it is difficult to scan a large area of the gastric mucosa for mosaicing during standard endoscopy. This paper proposes a novel \'snap-on\' robotic scanning device that can integrate distally with a commercial endoscope. A customized low-cost endomicroscopy system is used for obtaining micro imaging. The developed device could scan a large area of gastric tissue during standard endoscopy. The device achieves positioning accuracy that is less than 0.23 mm. Experimental results showed that the device could achieve large area mosaicing (15.8-18.6 mm2) and demonstrated the potential clinical value of the device for real-time gastric tissue identification and margin assessment. This approach presents an important alternative to current histology techniques for gastrointestinal tract diagnosis.

Real-time 3D ultrasound (3DUS) imaging offers improved spatial orientation information relative to 2D ultrasound. However, in order to improve its efficacy in guiding minimally invasive intra-cardiac procedures where real-time visual feedback of an instrument tip location is crucial, 3DUS volume visualization alone is inadequate. This paper presents a set of enhanced visualization functionalities that are able to track the tip of an instrument in slice views at real-time. User study with in vitro porcine heart indicates a speedup of over 30% in task completion time.

Three-dimensional transesophageal echocardiography (TEE) provides real-time soft tissue information, but its use is hampered by its limited field of view. The mosaicing of multiple TEE views makes it possible to visualize a large structure, like the left atrium, in a single volume. To this end, an automatic registration method is required. Similarly to atlas-based segmentation approaches, atlas-based mosaicing (ABM) uses a full volume atlas set to moderate the onerous registration of the individual TEE views. The performance of ABM depends both on the quality of the involved registrations and on the selection of the optimal transformation from the candidate transformations that result from the various atlases. The study described here explored the performance of different selection strategies on multiview TEE data of the left atrium. We found that by incorporating two stages of transformation selection, using the image similarity and the conformity between the candidate transformations as selection criteria, the average registration error dropped below 3 mm with respect to manual registration of these data. Finally, we used this method for the automatic construction of a wide-view TEE volume of the left atrium.

OBJECTIVE: Navigated panretinal photocoagulation is a standard care for proliferative diabetic retinopathy. Slit-lamp-based systems used for this treatment provide a narrow view of the retina. Retinal mosaics are used for view expansion and treatment planning. Mosaicing slit-lamp images is a hard task due to the absence of a physical model of the imaging process, large textureless regions and imaging artifacts, mostly reflections.
METHODS: We present a comparative study of various geometric transformation models applied to retinal image mosaicing in computer-assisted slit-lamp imaging. We propose an efficient point correspondence-based framework for transformation model evaluation in a typical closed-loop motion scenario. We compare the performance of multiple linear and nonlinear models of different complexities and assess the effect of the number of points used for parameter estimation. We use a local fitting error (LFE) metric to estimate the models\' performance in pairwise registration. Because LFE alone is not conclusive regarding the problem of accumulated drift, we propose a loop closure error (LCE) metric to quantify the effect of accumulated local registration errors. We also provide a new normalization procedure for the quadratic transformation model, widely used in retinal image registration.
RESULTS: In total, seven transformation models were evaluated on three datasets of long image sequences. LFE decreases with increasing complexity of the model, while LCE, in contrast, shows superior performance of simple models. Varying the number of point correspondences did not reveal a common trend for the LCE metric, showing an increase in the error for simple models and an unstable behavior of the complex models.
CONCLUSIONS: Our results show that simple models are less sensitive to drift and preferable for sequential mosaicing in slit-lamp imaging, while more complex models are the best choice for short-term registration.

Hyperspectral monitoring of large areas (more than 10 km(2)) can be achieved via the use of a system employing spectrometers and CMOS cameras. A robust and efficient algorithm for automatically combining multiple, overlapping images of a scene to form a single composition (i.e., for the estimation of the point-to-point mapping between views), which uses only the information contained within the images themselves is described here. The algorithm, together with the 2D fast Fourier transform, provides an estimate of the displacement between pairs of images by accounting for rotations and changes of scale. The resulting mosaic was successively georeferenced within the WGS-84 geographic coordinate system. This paper also addresses how this information can be transferred to a push broom type spectral imaging device to build the hyperspectral cube of the area prior to land classification. The performances of the algorithm were evaluated using sample images and image sequences acquired during a proximal sensing field campaign conducted in San Teodoro (Olbia-Tempio-Sardinia). The hyperspectral cube closely corresponds to the mosaic. Mapping allows for the identification of objects within the image and agrees well with ground-truth measurements.