Human-in-the-Loop Differential Subspace Search in High-Dimensional Latent Space

摘要

Generative models based on deep neural networks often have a high-dimensionallatent space, ranging sometimes to a few hundred dimensions oreven higher, which typically makes them hard for a user to explore directly.We propose differential subspace search to allow efficient iterative userexploration in such a space, without relying on domain- or data-specificassumptions. We develop a general framework to extract low-dimensionalsubspaces based on a local differential analysis of the generative model, suchthat a small change in such a subspace would provide enough change in theresulting data. We do so by applying singular value decomposition to theJacobian of the generative model and forming a subspace with the desireddimensionality spanned by a given number of singular vectors stochasticallyselected on the basis of their singular values, to maintain ergodicity.We use our framework to present 1D subspaces to the user via a 1D sliderinterface. Starting from an initial location, the user finds a new candidatein the presented 1D subspace, which is in turn updated at the new candidatelocation. This process is repeated until no further improvement canbe made. Numerical simulations show that our method can better optimizesynthetic black-box objective functions than the alternatives that we tested. Furthermore, we conducted a user study using complex generative modelsand the results show that our method enables more efficient exploration ofhigh-dimensional latent spaces than the alternatives.