MIDAS Projection: Markerless and Modelless Dynamic Projection Mapping for Material Representation

摘要

The visual appearance of an object can be disguised by projecting virtualshading as if overwriting the material. However, conventional projectionmappingmethods depend on markers on a target or a model of the targetshape, which limits the types of targets and the visual quality. In this paper,we focus on the fact that the shading of a virtual material in a virtual sceneis mainly characterized by surface normals of the target, and we attemptto realize markerless and modelless projection mapping for material representation.In order to deal with various targets, including static, dynamic,rigid, soft, and fluid objects, without any interference with visible light, wemeasure surface normals in the infrared region in real time and project materialshading with a novel high-speed texturing algorithm in screen space.Our system achieved 500-fps high-speed projection mapping of a uniformmaterial and a tileable-textured material with millisecond-order latency, andit realized dynamic and flexible material representation for unknown objects.We also demonstrated advanced applications and showed the expressiveshading performance of our technique.