Performance tuning in distributed computing systems

摘要

A method of profiling distributed computing jobs may include receiving, by a profiling orchestrator service, a profiling request initiated by a profiling client. The profiling request may identify a master node and a target application to be executed by the master node and a plurality of worker nodes. The method may further include transmitting, to a worker node of the plurality of worker nodes, a profiling command identifying a parameter reflecting execution of the target application on the worker node. The method may further include receiving profiling data comprising an execution marker associated with execution of the target application on the worker node. The method may further include forwarding the profiling data to the profiling client.