Paper 2021/687
Towards Understanding Practical Randomness Beyond Noise: Differential Privacy and Mixup
Hanshen Xiao and Srinivas Devadas
Abstract
Information-theoretical privacy relies on randomness. Representatively, Differential Privacy (DP) has emerged as the gold standard to quantify the individual privacy preservation provided by given randomness. However, almost all randomness in existing differentially private optimization and learning algorithms is restricted to noise perturbation. In this paper, we set out to provide a privacy analysis framework to understand the privacy guarantee produced by other randomness commonly used in optimization and learning algorithms (e.g., parameter randomness). We take mixup: a random linear aggregation of inputs, as a concrete example. Our contributions are twofold. First, we develop a rigorous analysis on the privacy amplification provided by mixup either on samples or updates, where we find the hybrid structure of mixup and the Laplace Mechanism produces a new type of DP guarantee lying between Pure DP and Approximate DP. Such an average-case privacy amplification can produce tighter composition bounds. Second, both empirically and theoretically, we show that proper mixup comes almost free of utility compromise.
Metadata
- Available format(s)
-
PDF
- Category
- Foundations
- Publication info
- Preprint. MINOR revision.
- Keywords
- Differential PrivacyPractical RandomnessConvex OptimizationStatistical Divergence
- Contact author(s)
-
hsxiao @ mit edu
devadas @ mit edu - History
- 2021-05-28: received
- Short URL
- https://ia.cr/2021/687
- License
-
CC BY
BibTeX
@misc{cryptoeprint:2021/687, author = {Hanshen Xiao and Srinivas Devadas}, title = {Towards Understanding Practical Randomness Beyond Noise: Differential Privacy and Mixup}, howpublished = {Cryptology {ePrint} Archive, Paper 2021/687}, year = {2021}, url = {https://eprint.iacr.org/2021/687} }