Keypoint Detection, Matching, and Tracking in Images with Non-Linear Distortion: Applications in Medical Endoscopy and Panoramic Vision

摘要

Point correspondences between different views are the input to many computer visionalgorithms with a multitude of purposes that range from camera calibration toimage content retrieval, and pass by structure-from-motion, registration, and mosaicking.Establishing such correspondences is particularly difficult, not only in the case ofwide-baseline and/or strong change in viewpoint, but also when images present significantnon-linear distortions. The thesis addresses this last problem and investigatessolutions for detecting, matching, and tracking points in images acquired by cameraswith unconventional optics such as fish-eye lenses, catadioptric sensors, or medicalendoscopes.We start by studying the impact of radial distortion in keypoint detection and descriptionusing the well known SIFT algorithm. Such study leads to several modificationsto the original method that substantially improve matching performance inimages with wide field-of-view. Our work is conclusive in showing that non-lineardistortion must be implicitly handled by a suitable design of filters and operators, asopposed to being explicitly corrected via image warping. The benefits of such approachare demonstrated in experiments of structure-from-motion, as well as in thedevelopment of a vision-system for indoor localization where perspective images areused to retrieve panoramic views acquired with a catadioptric camera.In a second line of research, we investigate solutions for feature tracking in continuoussequences acquired by cameras with radial distortion. We build on the top of theconventional frameworks for image region alignment and propose specific deformationmodels that simultaneously describe the effect of local image motion and globalimage distortion. It is shown for the first time that image distortion can be calibratedat each frame time instant by tracking a random set of salient points. The result isfurther explored to solve the problem of knowing the intrinsic calibration of cameraswith motorised zoom at all times. This problem is particularly relevant in the contextof medical endoscopy and the solution passes by combining off-line calibration withon-line tracking to update of the camera focal length. The effectiveness of our trackingand calibration approaches are validated in both medical and non-medical videosequences.The last contribution is a pipeline for visual odometry in stereo laparoscopy thatrelies in multi-model fitting for segmenting different rigid motions and implicitly discardingregions of non-rigid deformation. This is complemented by a temporal clusteringscheme that enables to decide which parts of the scene should be used to estimatethe camera motion in a reliable manner.