Byzantine-resilient distributed learning under constraints

摘要

We consider a class of convex distributed statistical learning problems with inequality constraints in an adversarial scenario. At each iteration, an $lpha$-fraction of $m$ machines, which are supposed to compute stochastic gradients of the loss function and send them to a master machine, may act adversarially and send faulty gradients. To guard against defective information sharing, we develop a Byzantine primal-dual algorithm. For $lphain[0,0.5)$, we prove that after $T$ iterations the algorithm achieves $ilde{check{O}}(1/T+1/sqrt{mT}-+lpha/sqrt{T})$ statistical error bounds on both the optimality gap and the constraint violation. Our result holds for a class of normed vector spaces and, when specialized to the Euclidean space, it attains the optimal error bound for Byzantine stochastic gradient descent.