A programming technique that works on the same principle as disease-preventing vaccinations could safeguard machine learning systems from malicious cyber-attacks.
The technique was developed by the digital specialist arm of Australia’s national science agency, the CSIRO, and presented recently at an international conference on machine learning, held in Long Beach, California, US.
Machine learning systems, or neural networks, are becoming increasingly prevalent in modern society, where they are pressed into service across a wide range of areas, including traffic management, medical diagnosis, and agriculture. They are also critical components in autonomous vehicles.
They operate from an initial training phase, in which they are fed tens of thousands of possible iterations of a given task. The resulting algorithm then has the capability to learn – to add to its own repertoire of possibilities and act accordingly, without the need for further human input.
As efficient as they are, however, machine leaning systems – like any computer-driven mechanism – remain vulnerable to hacking attacks. The primary way in which this is happens involves the introduction of “noise” – additional data-points that interfere with and distort input signals – such that external elements are misclassified.
The approach is known in the jargon of the business as introducing “adversarial examples” into the system. By adding noises (and often, not very much noise) a machine learning algorithm can be misled into classifying an image of a panda as an image of a gibbon.
More pertinently, given the rise of autonomous vehicles, they can be hacked and persuaded to classify a stop sign as a green traffic light.
Making machine-learning systems thus hack-resistant is a lively research field, spurred by recent research that found that real-world examples could be easily fooled with the use of nothing more sophisticated than a smartphone camera.
The latest approach, by researchers led by Richard Nock, takes its cue from public health.
In medicine, the practice of vaccination rests on the sound idea that exposing the body’s immune system to weak or dead versions of a pathogen – the ones that cause influenza or polio, for instance – prompts the development of specific antibodies. The immune system then “remembers” the pathogen so that the next time it encounters it – at full strength and for real – it will recognise it and eliminate it straight away.
Nock and colleagues approached their task in the same manner.
“Our new techniques prevent adversarial attacks using a process similar to vaccination,” he explains.
“We implement a weak version of an adversary, such as small modifications or distortion to a collection of images, to create a more ‘difficult’ training data set. When the algorithm is trained on data exposed to a small dose of distortion, the resulting model is more robust and immune to adversarial attacks.”
The approach is still at an early stage and has yet to be tested in a real-world situation against genuine malicious incursion attempts, but the results are promising. They are described in detail in a preprint paper available here.
And although more testing is clearly required, there is at this stage no evidence that vaccination may turn computers autistic.