Mining Mountains of Data: Organizing All Atom Molecular Dynamics Protein Simulation Data into SQL and OLAP Cubes.

摘要

Across scientific disciplines, the ability to generate, collect, and store data has outpaced the ability to make sense of them. Methods and technology exist today for working with extremely large data sets, yet the most common data organization paradigm in science is to create files, in some cases millions of files, and store them in file systems. Despite the best intentions, these repositories quickly become disorganized, fragile, and difficult to manage; hindering mining and exploitation of the data they contain. This is fundamentally an informatics problem, and here I present the design of a very large scale repository to organize and mine molecular dynamics simulation data.