Repeatable determinism using non-random weight initialisations in smart city applications of deep learning

摘要

Modern Smart City applications draw on the need for requirements that are safe, reliable and sustainable, as such these applications have a need to utilise machine-learning mechanisms such that they are consistent with public liability. Machine and deep learning networks, therefore, are required to be in a form that is safe and deterministic in their development and also in their deployment. The viability of non-random weight initialisation schemes in neural networks make the network more deterministic in learning sessions which is a desirable property in safety critical systems where deep learning is applied to smart city applications and where public liability is a concern. The paper uses a variety of schemes over number ranges and gradients and achieved a 98.09% accuracy figure, + 0.126% higher than the original random number scheme at 97.964%. The paper highlights that in this case, it is the number range and not the gradient that is affecting the achieved accuracy most dominantly, although there can be a coupling of number range with activation functions used. Unexpectedly in this paper, an effect of numerical instability was discovered from run to run when run on a multi-core CPU. The paper also has shown the enforcement of consistent deterministic results on an multi-core CPU by defining atomic critical code regions, and that aids repeatable information assurance in model fitting (or learning sessions). That enforcement of consistent repeatable determinism has also a benefit to accuracy even for the random schemes, and a highest score of 98.29%, + 0.326% higher than the baseline was achieved. However, also the non-random initialisation scheme causes weight arrangements after learning to be more structured which has benefits for validation in safety critical applications.