Awesome
BlurNN - Machine Learning with Differential Privacy
BlurNN is a pytorch-based model privacy-preserving module. Two kinds of mechanisms are included, optimizers with differential privacy and model sanitizers, and differs from the timing of perturb adjunction.
Install
python setup.py install
Usage
Optimizers
Check here
for torch.optim
usage.
This package extends optimizers in torch.optim
by extra parameters for differential privacy.
DPSGD
Extended optimizer from torch.optim.SGD
.
Implements sanitizer in Deep Learning with Differential Privacy.
blurnn.optim.DPSGD(params, lr=<required parameter>, momentum=0, dampening=0, weight_decay=0, nesterov=False, noise_scale=0, norm_bound=0)
Extra Parameters
- norm_bound (float, optional) - gradient cliping bound (
gradient * argmax(norm_bound, norm(gradient))
) - noise_scale (float, optional) - gaussian noise with standard deviation
noise_scale * norm_bound
is added to each clipped gradient
Usage
>>> optimizer = dp_optim.SGD(model.parameters(), lr=0.1, momentum=0.9, noise_scale=0.01, norm_bound=1.5)
>>> optimizer.zero_grad()
>>> loss_fn(model(input), target).backward()
>>> optimizer.step()
Performance
Following graphs are comparison of loss and accuracy during epochs using different optimizers over MNIST.
DPAdam
blurnn.optim.DPAdam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False, noise_scale=0, norm_bound=0)
Usage
>>> optimizer = dp_optim.DPAdam(model.parameters(), noise_scale=0.01, norm_bound=1.5)
>>> optimizer.zero_grad()
>>> loss_fn(model(input), target).backward()
>>> optimizer.step()
Performance
Following graphs are comparison of loss and accuracy during epochs using different optimizers over MNIST.
Model Sanitizer
ModelSanitizer
Implement Sparse Vector Technique(SVT) to obtain (eps1 + eps2 + eps3)-differential privacy
Reference Privacy-preserving Federated Brain Tumour Segmentation
Parameters
- epochs (int):
- eps1 (float):
- eps2 (float, optional): default:
(2 * release_proportion * sensitivity) ^ (2/3) * eps1
- eps3 (float):
- release_proportion (float, optional): the proportion of model to be shared. release_proportion should be in range [0,1), default: 0.4
- sensitivity (float, optional): default: 1
- gamma (float, optional): the bound default: 0.5
- tau (float, optional): default: 0.1
Usage
>>> sanitizer = ModelSanitizer(epochs=N, eps1=0.001, eps3=0.001)
>>> sanitizer.base_model = model
>>> for epoch in range(N):
>>> training...
>>> testing...
>>> step...
>>> sanitizer.sanitize(model)
License
The MIT License