Cross modal processing method, device, electronic apparatus and computer storage medium

摘要

The training of the lexical relation between the vocabulary data and the corresponding image is done by training the combination of the training vocabulary data and the corresponding image Problem to be solved: to provide a cross modal processing method, an apparatus, an electronic apparatus, and a computer storage medium in which the training effect of the language model performing cross modal processing is improved.A plurality of vocabulary data and a sample set containing a plurality of images are acquired and a plurality of training samples are generated based on the sample set.Each training sample is a combination of at least one vocabulary data and a corresponding one at least one image, and by training a language model using multiple training samples Cross modal processing between lexical data and images is carried out using the spoken language model.Diagram