Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent
Dec 6, 2021
Speakers
About
What is the information leakage of an iterative learning algorithm about its training data, when the internal state of the algorithm is not observable? How much is the contribution of each specific training epoch to the final leakage? We study this problem for noisy gradient descent algorithms, and model the dynamics of Rényi differential privacy loss throughout the training process. Our analysis traces a provably tight bound on the Rényi divergence between the pair of probability distributions over parameters of models with neighboring datasets. We prove that the privacy loss converges exponentially fast, for smooth and strongly convex loss functions, which is a significant improvement over composition theorems. For Lipschitz, smooth, and strongly convex loss functions, we prove optimal utility for differential privacy algorithms with a small gradient complexity.What is the information leakage of an iterative learning algorithm about its training data, when the internal state of the algorithm is not observable? How much is the contribution of each specific training epoch to the final leakage? We study this problem for noisy gradient descent algorithms, and model the dynamics of Rényi differential privacy loss throughout the training process. Our analysis traces a provably tight bound on the Rényi divergence between the pair of probability distributions…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Learning Optimal Predictive Checklists
Watch later
Haoran Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On Optimal Interpolation in Linear Regression
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
OmniPrint: A Configurable Printed Character Synthesizer
Watch later
Haozhe Sun, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Methods: Self-prediction and Contrastive learning
Watch later
Total of 2 viewers voted for saving the presentation to eternal vault which is 0.2%
Estimating the Long-Term Effects of Novel Treatments
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%