Burst photography for high dynamic range and low-light imaging on mobile cameras

摘要

Cell phone cameras have small apertures, which limits the numberrnof photons they can gather, leading to noisy images in low light.rnThey also have small sensor pixels, which limits the number ofrnelectrons each pixel can store, leading to limited dynamic range. Werndescribe a computational photography pipeline that captures, aligns,rnand merges a burst of frames to reduce noise and increase dynamicrnrange. Our system has several key features that help make it robustrnand efficient. First, we do not use bracketed exposures. Instead,rnwe capture frames of constant exposure, which makes alignmentrnmore robust, and we set this exposure low enough to avoid blowingrnout highlights. The resulting merged image has clean shadows andrnhigh bit depth, allowing us to apply standard HDR tone mappingrnmethods. Second, we begin from Bayer raw frames rather thanrnthe demosaicked RGB (or YUV) frames produced by hardwarernImage Signal Processors (ISPs) common on mobile platforms. Thisrngives us more bits per pixel and allows us to circumvent the ISP’srnunwanted tone mapping and spatial denoising. Third, we use a novelrnFFT-based alignment algorithm and a hybrid 2D/3D Wiener filterrnto denoise and merge the frames in a burst. Our implementationrnis built atop Android’s Camera2 API, which provides per-framerncamera control and access to raw imagery, and is written in thernHalide domain-specific language (DSL). It runs in 4 seconds onrndevice (for a 12 Mpix image), requires no user intervention, andrnships on several mass-produced cell phones.