While cross-lingual word embeddings have been studied extensively in recentyears, the qualitative differences between the different algorithms remainvague. We observe that whether or not an algorithm uses a particular featureset (sentence IDs) accounts for a significant performance gap among thesealgorithms. This feature set is also used by traditional alignment algorithms,such as IBM Model-1, which demonstrate similar performance to state-of-the-artembedding algorithms on a variety of benchmarks. Overall, we observe thatdifferent algorithmic approaches for utilizing the sentence ID feature spaceresult in similar performance. This paper draws both empirical and theoreticalparallels between the embedding and alignment literature, and suggests thatadding additional sources of information, which go beyond the traditionalsignal of bilingual sentence-aligned corpora, may substantially improvecross-lingual word embeddings, and that future baselines should at least takesuch features into account.
展开▼
