BALANCING MEMORY CONSUMPTION OF MULTIPLE GRAPHICS PROCESSING UNITS IN DEEP LEARNING

摘要

A deep learning system is provided for balancing memory consumption. The deep learning system includes a central processing unit (CPU). The deep learning system further includes a plurality of Graphics Processing Units (GPUs), each having a memory that includes a neural network for neural network training. The memory of a particular one of the plurality of GPUs having a smallest memory consumption includes an additional neural network for neural network validation.