SYSTEMS AND METHODS FOR MACHINE NUMERACY

摘要

Described herein is a method for training a neural network. The method includes pre-training the neural network based on at least one of: (i) a magnitude model, whereby pre-training comprises: masking a plurality of said numbers (masked numbers); predicting a magnitude of each masked number; and minimising a total loss over the masked numbers; and (ii) a value model, whereby pre-training comprises: partially masking a plurality of said numbers (partially masked numbers) such that each partially masked number comprises a visible portion; predicting a value of each partially masked number; and minimising a total loss over the partially masked numbers. The method also includes training the pre-trained neural network on a domain-specific corpus and then applying the trained neural network to a document to identify a relationship between one or more numbers in the document and a context of the document.