Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
Jul 25, 2023
Speakers
Christopher Choquette-Choo
Speaker · 0 followers
Hugh Brendan McMahan
Speaker · 0 followers
Keith Rush
Speaker · 0 followers
Abhradeep Guha Thakurta
Speaker · 1 follower
About
We introduce new differentially private (DP) mechanisms for gradient-based machine learning (ML) with multiple passes (epochs) over a dataset, substantially improving the achievable privacy-utility-computation tradeoffs. We formalize the problem of DP mechanisms for adaptive streams with multiple participations and introduce a non-trivial extension of online matrix factorization DP mechanisms to our setting. This includes establishing the necessary theory for sensitivity calculations and efficient computation of optimal matrices. For some applications like $>\!\! 10,000$ SGD steps, applying these optimal techniques becomes computationally expensive. We thus design an efficient Fourier-transform-based mechanism with only a minor utility loss. Extensive empirical evaluation on both example-level DP for image classification and user-level DP for language modeling demonstrate substantial improvements over all previous methods, including the widely-used DP-SGD. Though our primary application is to ML, our main DP results are applicable to arbitrary linear queries and hence may have much broader applicability.We introduce new differentially private (DP) mechanisms for gradient-based machine learning (ML) with multiple passes (epochs) over a dataset, substantially improving the achievable privacy-utility-computation tradeoffs. We formalize the problem of DP mechanisms for adaptive streams with multiple participations and introduce a non-trivial extension of online matrix factorization DP mechanisms to our setting. This includes establishing the necessary theory for sensitivity calculations and effici…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption
Watch later
Seewoo Lee, …
Fourmer: An Efficient Global Modeling Paradigm for Image Restoration
Watch later
Man Zhou, …
Deceptive Alignment Monitoring
Watch later
Settling the Reward Hypothesis
Watch later
