Adaptive caching and dynamic delay scheduling for in-memory data analytics

摘要

In various embodiments, computer-implemented techniques for improving function of in-memory processing systems via adaptively caching datasets include: identifying data stored in a distributed filesystem, the data including data to be processed by an in-memory processing application and data not to be processed by the in-memory processing application; identifying one or more partitions of the data corresponding to the data to be processed; and selectively transferring the one or more partitions from the distributed filesystem to a memory of the in-memory processing application. Corresponding systems and computer-program products are also disclosed.