Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Images

摘要

Relighting of human images has various applications in image synthesis.For relighting, we must infer albedo, shape, and illumination from a humanportrait. Previous techniques rely on human faces for this inference, basedon spherical harmonics (SH) lighting. However, because they often ignorelight occlusion, inferred shapes are biased and relit images are unnaturallybright particularly at hollowed regions such as armpits, crotches, or garmentwrinkles. This paper introduces the first attempt to infer light occlusion in theSH formulation directly. Based on supervised learning using convolutionalneural networks (CNNs), we infer not only an albedo map, illumination butalso a light transport map that encodes occlusion as nine SH coefficients perpixel. The main difficulty in this inference is the lack of training datasetscompared to unlimited variations of human portraits. Surprisingly, geometricinformation including occlusion can be inferred plausibly even with a smalldataset of synthesized human figures, by carefully preparing the dataset sothat the CNNs can exploit the data coherency. Our method accomplishesmore realistic relighting than the occlusion-ignored formulation.