Relighting Character Motion for Photoreal Simulations; Conference paper

摘要

We present a fully image-based approach for capturing and modeling real human locomotion under varying illumination and viewpoint that overviews the techniques and results presented by Einarsson et al, 2006. An actor performs repeatable locomotive actions (walking/running) on a rotating treadmill while being filmed from a vertical array of 3 high-speed cameras under controlled rapidly changing lighting conditions. The known rotation of the treadmill, repeatability of the actor's motion, timing of the lighting pattern and capture rate of the cameras are all carefully synchronized so that the actor is imaged in (approximately) the same position in the locomotion at the same point in the lighting pattern but having rotated a known amount due to the known turntable motion. This allows us to effectively multiply the number of cameras from 3 1 in azimuth to 3 36. Small perturbations in the actor's repeating cyclic position are corrected for using optical flow, and optical flow is also used to align images temporally. This leads to a flowed reflectance field data structure. Datasets are compressed using image compression. Image-based relighting and a combination of view morphing and light field rendering implemented on the GPU allow us to render the subject under novel viewpoint and illumination. To composite the person into a scene we derive an alpha matte from retro-reflective material and a back-lit diffuse backdrop, and implement a voxel-based visual hull process to compute how the person should cast shadows on the ground plane. We demonstrate realistic composites of real subjects into real and virtual environments applicable to the area of training simulation.