Deep Multispectral Painting Reproduction via Multi-Layer, Custom-Ink Printing

摘要

We propose a workflow for spectral reproduction of paintings, which capturesa painting’s spectral color, invariant to illumination, and reproducesit using multi-material 3D printing. We take advantage of the current 3Dprinters’ capabilities of combining highly concentrated inks with a largenumber of layers, to expand the spectral gamut of a set of inks. We usea data-driven method to both predict the spectrum of a printed ink stackand optimize for the stack layout that best matches a target spectrum. Thisbidirectional mapping is modeled using a pair of neural networks, which are optimized through a problem-specific multi-objective loss function. Ourloss function helps find the best possible ink layout resulting in the balancebetween spectral reproduction and colorimetric accuracy under a multitudeof illuminants. In addition, we introduce a novel spectral vector errordiffusion algorithm based on combining color contoning and halftoning,which simultaneously solves the layout discretization and color quantizationproblems, accurately and efficiently. Our workflow outperforms thestate-of-the-art models for spectral prediction and layout optimization. Wedemonstrate reproduction of a number of real paintings and historically importantpigments using our prototype implementation that uses 10 custominks with varying spectra and a resin-based 3D printer.