Annotating and Learning Morphological Segmentation of Egyptian Colloquial Arabic

摘要

We present an annotation and morphological segmentation scheme for Egyptian Colloquial Arabic (ECA) with which we annotate user-generated content that significantly deviates from the orthographic and grammatical rules of Modem Standard Arabic and thus cannot be processed by the commonly used MSA tools. Using a per letter classification scheme in which each letter is classified as either a segment boundary or not, and using a memory-based classifier, with only word-internal context, prove effective and achieve a 92% exact match accuracy at the word level. The well-known MADA system achieves 81%, while the per letter classification scheme using the ATB achieves 82%. Error analysis shows that the major problem is that of character ambiguity, since the ECA orthography overloads the characters which would otherwise be more specific in MSA, like the differences between y (S) and Y (S) and A (l) ＜ , (l), and ＜ (l) which are collapsed to y (S) and A (l) respectively or even totally confused and interchangeable. While normalization helps alleviate orthographic inconsistencies, it aggravates the problem of ambiguity.