FocusGAN: Preserving Background in Text-Guided Image Editing

摘要

Text-guided image editing (TIE) seeks to manipulate images by the guidance of language. However, the existing TIE methods always overlook the target-irrelevant pixels and the editing may make the background discolored, distorted, or partially disappear. To overcome this problem, we propose a novel TIE method named FocusGAN, which edits the text-relevant pixels precisely as well as keeps the background invariant. Specifically, we build a network of two stages. In each stage, we first construct a channel-wise subject focusing attention to make the generator focus on the sub-region that best matches each word. Then, the word-level discriminator provides fine-grained feedback by correlating words with image areas, so that the generator can manipulate specific visual attributes without affecting the background. Last, we propose a background-keeping cyclic loss to further improve the invariance of the background and to encourage the edit of the subject that matches the given text. Experiments on CUB and Oxford datasets demonstrate that our approach can effectively keep the background invariant in manipulating images using natural language descriptions.