Systems and methods for allocating jobs in a distributed file system queue are provided. The system may receive a job request and intelligently divide the job request into one or more logical job stages. Each job stage may be assigned a stage weight based on historical data corresponding to each job stage, such as job stage computing requirements, job stage data outputs, and the like. The system may assign each job stage to a job queue n the distributed file system based on the stage weight and the current computing availability of each job queue. The system may execute each job stage in the assigned job queue to complete the job request.
展开▼
