Seonglae ChoDifferential privacy, as discussed by Google, ensures that no individual's data has a significant impact on the model's predictions.
The goal is to measure and minimize changes in output results based on the inclusion or exclusion of individual data items, so that data analysis results remain nearly identical whether a specific individual's data is included or excluded.
VaultGemma
Provides mathematically defined privacy limits that prevent data leakage.
DP can prevent reconstruction attacks, and its guarantee is stronger when viewed through RDP compared to traditional (ε,δ)-DP interpretation. Previously, DP guarantees were mainly interpreted in terms of membership inference, but this paper provides direct guarantees against reconstruction/extraction attacks. In particular, using the probability preservation property of RDP (Rényi Differential Privacy), the paper argues that even with the same DP-SGD mechanism, one can obtain tighter leakage bounds than the ε interpretation in existing literature.
Even with a privacy budget that cannot prevent membership inference, it can still make full reconstruction of rare secrets difficult. In other words, if an attacker already knows a secret, they can somewhat determine "was this in the training set?" However, extracting a secret from the model without prior knowledge is much more difficult.
The smaller the prior probability p₀ of a secret (i.e., the rarer and higher entropy it is), the more limited the overall leakage is, even when the posterior p₁ increases due to training. Therefore, the longer and rarer a secret is, the more protected it is. As the paper states, "secret length itself serves as protection."
Opacus
The goal is to enable DP-SGD to be applied by adding only about 2 lines to existing training code. Much faster than micro-batch methods like PyVacy