3D Skeleton reconstruction from images using matching 2D skeletons

摘要

Generating 3D skeletons from 2D images of real-world objects, comprising: determining (a set of one or more) 2D skeletons of the objects in each image of a set of images captured simultaneously by cameras; for pairs of simultaneously captured images, finding one 2D skeleton from the set of 2D skeletons in one of the images that matches with just one of the 2D skeletons in the other image; generating one 3D skeleton from the matched 2D skeletons. Object pose estimation may allow rendering alternative views, from new viewpoints, of animated, articulated objects such as humans, animals, mammals, robots. Images may be scaled, sampled, cropped, and samples clustered, to identify object parts (eg. limbs, joints, head, hands, feet, shoulders, knees, elbows, pelvis, arm, thigh, trunk) and associated probabilities. Distance between corresponding parts of matched 2D skeletons may be determined – by projecting 2D skeleton parts as epipolar lines – and thresholded. Weighted links between nodes (corresponding to 2D skeletons) of a graph may be based on skeleton distances. Weak 3D skeletons may be generated by projecting 2D skeleton parts as lines in 3D space to determine 3D positions of those parts. 3D part locations may be finalised using a random sample consensus algorithm.