Method for Delivering Knowledge to Light Deep Learning Network from Deep Learning Network

摘要

The present invention relates to a method for transferring knowledge from a deep learning network to a lightweight deep learning network. The method is a softmax ( softmax) a first step of calculating an output value STK T to which the function is applied; a second step of calculating a P model that is an output value obtained by applying a softmax function to a second soft label, which is a distribution of a result value output by receiving data to be learned by the lightweight deep learning network; a third step of calculating a degree of similarity (KDR) between STK T and P models; a fourth step of calculating a ratio (TSTR) of the current learning time step to the total learning time; a fifth step of selecting a variable (TSSR) for determining the total ratio based on the similarity (KDR) and the ratio (TSTR) of the current learning time step; A sixth step of adjusting the ratio of STK T and the ratio of the actual correct answer (EK T ) received by the lightweight deep learning network from the deep learning network based on the variable (TSSR) for determining the percent ratio.