Microsoft Invents A Fast CryptoNet
Written by Mike James   
Monday, 15 February 2016

A CryptoNet is a neural network that works like a standard neural network but with encrypted data. It can make a prediction based on the data without revealing it, or the prediction, to a third party. A CryptoNet is where a neural network meets homomorphic encryption. 

key

The problem that Microsoft Research is trying to solve is how to provide the facilities of a neural network as a hosted service - Machine Learning As  a Service MLAS - in a secure way. A potential user might well value the result of applying a neural network to their data but be much less happier about making it visible to a remote server. 

There are a few possible ways of keeping the data secure but the most desirable is to use homomorphic encryption. Suppose you want to send two numbers to an "addition" server, it adds them and sends back the result. You could use standard encryption to send the two items of data, but then the server would have to have a decryption key to decrypt the values, add them and then encrypt the result. 

This is workable but clearly potentially leaky. 

Homomorphic encryption allows the user to send the two values encrypted, for the server to add the encrypted values and send the result back to the user. The user can then decrypt the result to get the sum of the original values.

In other words in homomorphic encryption the sum of the encrypted values is the encryption of the sum of the original values. 

Homomorphic encryption is almost magic but some schemes have been invented for addition and multiplication but they are very slow. 

A CryptoNet is a neural network that works with homomorphically encoded data to produce an encoded result. As we can create homomorphic codes that preserve addition and multiplication and we can arrange for a neural network to work with just these operations. The problem is that training and using a neural network is already time consuming. 

The breakthrough is to carefully match the characteristics of the homomorphic encryption to the neural network and vice versa. The homomorphic encryption actually works with polynomials - that is, it encrypts a polynomial - and as long as the operation of the neural network only involves polynomials than we can obtain a speed up using SIMD instructions.

In this case polynomials of degree up to 8191 were used to pack images into batches of 8192. The MNIST database of hand written digits was used to train the network. A training set of 50,000 images was used and the network achieved a 99% accuracy on the remaining 10,000 test cases.  Each image is 28x28 and 8191 images can be encoded in one 122-second operation. The network can be applied in 570 seconds on a PC with 16GB of RAM and a Xeon E5 running at 3.5GHz. The decryption of the result takes 5 seconds. This gives a total throughput of 77236 images per hour which is practical enough for a commercial service. 

The idea of mixing the computational complexity of neural networks and homomorphic encryption seems like a non-starter, but apparently it can be done. You can train and use a neural network with encrypted data without decrypting it first.

At the moment it is difficult to say if this technology is going anywhere. It is unclear how big the demand for machine learning as a service is, or could be. It is also unclear how often issues of data security would make this approach desirable.

 synaptic1

 

More Information

CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy

Related Articles

Microsoft Wins ImageNet Using Extremely Deep Neural Networks

Google Gears Up To Sell AI Vision

Project Oxford Extends Reach

Azure Machine Learning Service Goes Live

Open Source Homomorphic Cryptography 

DARPA spends $20 million on homomorphic encryption 

Modifiable encryption 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin

 

Banner


Remembering Grace Hopper On Her 114th Anniversary
09/12/2024

Today sees the start of Computer Science Education Week and  the 2024 Hour of Code. These educational event are timed to coincide with Grace Hopper's birthday on January 9th, 1906 due to her conc [ ... ]



Prompt Engineering Techniques To Make You An Expert
18/11/2024

Introducing a GitHub repository full of hot tips and instructions on how to build the perfect prompt presented in a collection of Jupiter Notebooks.


More News

 

espbook

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Tuesday, 16 February 2016 )