Transforming the arχiv to XML

摘要

We describe an experiment of transforming large collections of LATEX documents to more machine-understandable representations. Concretely, we are translating the collection of scientific publications of the Cornell e-Print Archive (ARXIV) using the LATEX to XML converter which is currently under development. The main technical task of our ARXMLIV project is to supply LaTeXML bindings for the (thousands of) LATEX classes and packages used in the ARXIV collection. For this we have developed a distributed build system that reiteratively runs LaTeXML over the ARXIV collection and collects statistics about e.g. the most sorely missing LaTeXML bindings and clusters common error events. This creates valuable feedback to both the developers of the LaTeXML package and to binding implementers. We have now processed the complete ARXIV collection of more than 400,000 documents from 1993 until 2006 (one run is a processor-year-size undertaking) and have continuously improved our success rate to more than 56% (i.e. over 56% of the documents that are LATEX have been converted by LaTeXML without noticing an error and are available as XHTML+MathML documents).