On Differentially Private Stochastic Convex Optimization with Heavy-tailed Data
Jul 12, 2020
Sprecher:innen
Hanshen Xiao
Sprecher:in · 0 Follower:innen
Srini Devadas
Sprecher:in · 0 Follower:innen
Über
In this paper, we consider the problem of designing Differentially Private (DP) algorithms for Stochastic Convex Optimization (SCO) on heavy-tailed data. The irregularity of such data violates some key assumptions used in almost all existing DP-SCO and DP-ERM methods, resulting in failure to provide the DP guarantees. To better understand this type of challenges, we provide in this paper a comprehensive study of DP-SCO under various settings. First, we consider the case where the loss function is strongly convex and smooth. For this case, we propose a method based on the sample-and-aggregate framework, which has an excess population risk of Õ(d^3/nϵ^4) (after omitting other factors), where n is the sample size and d is the dimensionality of the data. Then, we show that with some additional assumptions on the loss functions, it is possible to reduce the expected excess population risk to Õ( d^2/ nϵ^2 ). To lift these additional conditions, we also provide a gradient smoothing and trimming based scheme to achieve excess population risks of Õ( d^2/nϵ^2) and Õ(d^2/3/(nϵ^2)^1/3) for strongly convex and general convex loss functions, respectively, with high probability. Experiments on both synthetic and real-world datasets suggest that our algorithms can effectively deal with the challenges caused by data irregularity.In this paper, we consider the problem of designing Differentially Private (DP) algorithms for Stochastic Convex Optimization (SCO) on heavy-tailed data. The irregularity of such data violates some key assumptions used in almost all existing DP-SCO and DP-ERM methods, resulting in failure to provide the DP guarantees. To better understand this type of challenges, we provide in this paper a comprehensive study of DP-SCO under various settings. First, we consider the case where the loss function i…
Kategorien
Mathematik
Kategorie · 2,4k Präsentationen
KI und Datenwissenschaft
Kategorie · 10,8k Präsentationen
Über ICML 2020
The International Conference on Machine Learning (ICML) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known as machine learning. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. ICML is one of the fastest growing artificial intelligence conferences in the world. Participants at ICML span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Decentralised Learning with Random Features and Distributed Gradient Descent
Später ansehen
Scalable Identification of Partially Observed Systems with Certainty-Equivalent EM
Später ansehen
Kunal Menda, …
Auto-GAN-Distiller: Searching to Compress Generative Adversarial Networks
Später ansehen
Yonggan Fu, …
Continuous Graph Flow
Später ansehen
Zhiwei Deng, …
