PREFETCH CHAINING FOR OPTIMIZING DATA READ-AHEAD FOR WORKFLOW AND ANALYTICS

摘要

One embodiment provides a method for optimizing data read-ahead for machine learning applications including obtaining, by a processor, next file information from a workflow scheduler for next input files for a next processing stage that are to be accessed by a machine learning application. Data for the next processing stage for the machine learning application and at least one system job is prefetched. The next input files are prefetched as the prefetching data reaches an end of current inputs. A schedule is caused to chain the next input files for the next processing stage to current input files of a current processing stage.