Darkroom: Compiling High-Level Image Processing Code into Hardware Pipelines

摘要

Specialized image signal processors (ISPs) exploit the structure ofrnimage processing pipelines to minimize memory bandwidth usingrnthe architectural pattern of line-buffering, where all intermediate datarnbetween each stage is stored in small on-chip buffers. This providesrnhigh energy efficiency, allowing long pipelines with tera-op/sec. imagernprocessing in battery-powered devices, but traditionally requiresrnpainstaking manual design in hardware. Based on this pattern, wernpresent Darkroom, a language and compiler for image processing.rnThe semantics of the Darkroom language allow it to compile programsrndirectly into line-buffered pipelines, with all intermediaternvalues in local line-buffer storage, eliminating unnecessary communicationrnwith off-chip DRAM. We formulate the problem ofrnoptimally scheduling line-buffered pipelines to minimize bufferingrnas an integer linear program. Finally, given an optimally scheduledrnpipeline, Darkroom synthesizes hardware descriptions for ASIC orrnFPGA, or fast CPU code. We evaluate Darkroom implementationsrnof a range of applications, including a camera pipeline, low-level featurerndetection algorithms, and deblurring. For many applications, werndemonstrate gigapixel/sec. performance in under 0.5mm2 of ASICrnsilicon at 250 mW (simulated on a 45nm foundry process), realtimern1080p/60 video processing using a fraction of the resourcesrnof a modern FPGA, and tens of megapixels/sec. of throughput on arnquad-core x86 processor.