Systems, devices and methods are provided for building and using learning machines to understand and explain learning machines. The present system comprises a reference learning machine and an explainer learning machine being built for explaining and understanding the reference learning machine. A set of input signals is fed through the reference learning machine and the outputs at the different components of the learning machine for each given input signal are recorded. The recorded outputs at the different components of the learning machine for each given input signal, along with the corresponding expected output of the learning machine for each given input signal, are then used to update the parameters of the explainer learning machine. After the parameter update process, the explainer learning machine can then be queried for quantitative insights about the reference learning machine.