DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learning
Jul 24, 2023
Sprecher:innen
Tomoya Murata
Speaker · 0 followers
Taiji Suzuki
Speaker · 1 follower
Über
Differential private optimization for nonconvex smooth objective is considered. In the previous work, the best known utility bound is O(√(d)/(nε_DP)) in terms of the squared full gradient norm, which is achieved by Differential Private Gradient Descent (DP-GD) as an instance, where n is the sample size, d is the problem dimensionality and ε_DP is the differential privacy parameter. To improve the best known utility bound, we propose a new differential private optimization framework called DIFF2 (DIFFerential private optimization via gradient DIFFerences) that constructs a differential private global gradient estimator with possibly quite small variance based on communicated gradient differences rather than gradients themselves. It is shown that DIFF2 with a gradient descent subroutine achieves the utility of O(d^2/3/(nε_DP)^4/3), which can be significantly better than the previous one in terms of the dependence on the sample size n. To the best of our knowledge, this is the first fundamental result to improve the standard utility O(√(d)/(nε_DP)) for nonconvex objectives. Additionally, a more computational and communication efficient subroutine is combined with DIFF2 and its theoretical analysis is also given. Numerical experiments are conducted to validate the superiority of DIFF2 framework.Differential private optimization for nonconvex smooth objective is considered. In the previous work, the best known utility bound is O(√(d)/(nε_DP)) in terms of the squared full gradient norm, which is achieved by Differential Private Gradient Descent (DP-GD) as an instance, where n is the sample size, d is the problem dimensionality and ε_DP is the differential privacy parameter. To improve the best known utility bound, we propose a new differential private optimization framework called DIFF2…
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Watch later
Shirley Wu, …
Addressing Technology Mediated Social Harms
Watch later
Privacy Leakage in Generative Al for Vision
Watch later
The Melting Pot of Neural Algorithmic Reasoning
Watch later
On The Complicate Romance Between Sparsity and Bobustness
Watch later
