Smart data staging based on scheduling policy

摘要

Moving data to a remote system and performing computations at the remote system. A method includes receiving from one or more users a plurality jobs. The one or more jobs specify both data to be operated on and the operations to be performed on the data at the remote system. At a data service, one of the jobs to upload to the remote system in priority over other jobs in the plurality of jobs is selected. The method further includes beginning to upload the prioritized job including staging data from the job. At the remote system, once a portion of the data has been received, a service in the remote system begins to perform the specified operation on the data for the prioritized job. Beginning to perform the specified operation is performed prior to all of the data from the prioritized job being uploaded to the remote system.