Technical task: Differential Privacy & Synthetic Data Generation
Overview
Welcome to the HCAIM Hackathon! This document contains all the information needed to get you started for the first challenge of this hackathon. This challenge is focused on differential privacy and synthetic data generation. These are two very important areas of privacy-preserving machine learning. You will see in this challenge how these technologies can help us train models and extract insights from sensitive data which we cannot access directly.
Differential privacy is the most mathematically rigorous definition of privacy, and hence the most secure. You won’t have to understand all the details for this challenge. The following three points should be sufficient.
- Think of differential privacy as being based on your ability to guess whether or not a person/entity/record is in a dataset or not based on the information shared. The more precise the information shared, the easier it is to guess.
- The amount of information shared in any output is quantified by ? (epsilon). If ?=0 then you will have absolutely no ability to guess whether a person/entity/record exists in the database. When it is ∞ you would know with 100% confidence that the person/entity/record is or is not in the database.
- Typically noise is intentionally added to results to guarantee an upper bound on the ? of privacy is lost. So your differentially private questions are going to receive noisy answers. The more ? you use, the more accurate the result. In real applications, we would usually want ? to be low (eg 1-10), but there is no magic rule to follow for this, unfortunately.
The basic technique for making data differentially private is by adding noise to a dataset as is shown in the diagram below. This is exactly what the SmartNoise library does.
This challenge was developed in collaboration with Oblivious.
Scenario
Two parties (you and a partner) have two joinable sets of data with a shared ID. You both want to create a machine learning/statistical model to make predictions, you have the inputs, and they have the outputs. Alas, you don’t trust one another enough to share your data directly, so you agree to only share your data through an enclave so neither party can see one another’s inputs.
But of course, even enclaves will need to output something to be useful. To ensure maximum security you agree that any outputs from the enclave must be differentially private. These outputs can be either the results of SQL queries with noise added to make it differentially private or differentially private synthetic data which should reflect some of the statistics and trends of the real data. We have implemented API functions to perform these tasks for you. These use the SmartNoise library in the backend.
In the first section of this challenge, we provide you with some sandbox data and a sandbox enclave. You can use these to practice different kinds of queries and synthetic data generation methods before experimenting on the full dataset.
Goal
The goal of this challenge is to train a machine learning model on the differentially private outputs of the enclave. The performance of this model will be evaluated on a test set stored in the enclave so you will not have direct access to it. You should try and train your model while minimising the amount of potential data leakage from the data stored in the enclave.
Starter Code
The Google Colab notebook for this challenge can be found here. You should make a copy which you can edit yourself. Much of the information in this document is outlined there as well. There is also some sample code showing you how to access the enclave and train a model.
You will be provided with a codename for each participant/team that you will use for accessing the enclave, submission and the leaderboard. This will be provided at the start of the hackathon by the mentors.
Evaluation
You will be scored both on the accuracy of your resulting model, as well as how securely you have treated the data. Every time you access the enclave to make an SQL query or to generate some synthetic data the epsilon value you use will be added to a running total. You can consider this running total as representing the potential data leakage of the training process. Your final score will be calculated based on the formula score = best_accuracy – total_epsilon/500.
Reading Material
There are several links to educational resources in the notebook linked above. Below we provide some supplemental material.