How Can Neuroscience Protect Artificial Intelligence from Cyberattacks?
Deep learning has come a long way since the days when it could only recognise handwritten characters on cheques and envelopes. Today, deep neural networks have become a core component of many computer vision applications, from photo and video editors to medical software and self-driving cars. Shaped broadly on the structure of the brain, neural networks have come closer to seeing the world as we humans do. But they still have a long way to go, and they make mistakes in situations where humans would never err.
These situations, generally known as adversarial examples, change the behaviour of an AI model in surprising ways. Adversarial machine learning is one of the greatest challenges of current artificial intelligence systems. They can lead to machine learning models that fail in unpredictable ways or become vulnerable to cyberattacks.
Building AI systems that are resilient against adversarial attacks has become an active area of research and a hot topic of discussion at AI conferences. In computer vision , one interesting method of protecting deep learning systems against adversarial attacks is to apply findings in neuroscience to close the gap between neural networks and the mammalian vision system.
Using this approach, researchers at MIT and the MIT-IBM Watson AI Lab discovered that directly mapping the features of the mammalian visual cortex onto deep neural networks creates AI systems that are more predictable in their behaviour and more robust to adversarial perturbations. In a paper published on the bioRxiv preprint server, the researchers introduced VOneNet, an architecture that combines current deep learning techniques with neuroscience-inspired neural networks.
The work, carried out with the help of scientists at the University of Munich, Ludwig Maximilian University and the University of Augsburg, was accepted at NeurIPS 2020, one of the major annual AI conferences, which is to be held virtually this year.
Convolutional neural networks “Convolutional neural network”
The main architecture used in computer vision today is convolutional neural networks (CNN). When stacked on top of one another, multiple convolutional layers can be trained to learn and extract hierarchical features from images. The lower layers find general patterns such as corners and edges, and the higher layers gradually become adept at finding more specific things such as objects and people. Compared with traditional fully connected networks, ConvNets have proved to be both more robust and computationally efficient. Nevertheless, there are fundamental differences between the way CNNs and the human visual system process information.
DEEP, EVER DEEPER
“Deep neural networks (and convolutional neural networks in particular) have emerged as surprisingly good models of the visual cortex – surprisingly, they tend to fit experimental data collected from the brain better than computational models specially tailored to explain the neuroscience data. David Cox, IBM Director of the MIT-IBM Watson AI Lab, told TechTalks. “But not every deep neural network matches the brain data equally well, and there are some persistent gaps where the brain and DNNs differ.”
The most prominent of these gaps are adversarial examples, in which subtle perturbations such as a small patch or an imperceptible layer of noise can cause neural networks to misclassify their inputs. These changes mostly go unnoticed by the human eye.
“It is certainly the case that the images that fool DNNs would never fool our own visual systems,” says Cox. “It is also the case that DNNs are surprisingly brittle against natural degradations of images (e.g. the addition of noise), so robustness in general appears to be an open problem for DNNs. With this in mind, we thought this was a good place to look for differences between the brain and DNNs that might be useful.
Cox has been exploring the intersection of neuroscience and artificial intelligence since the early 2000s, when he was a student of James DiCarlo, professor of neuroscience at MIT. The two have continued to work together ever since.
“The brain is an incredibly powerful and effective information-processing machine, and it is tantalising to ask whether we can learn new tricks from it that can be used for practical purposes. At the same time, we can use what we know about artificial systems to provide guiding theories and hypotheses that can suggest experiments to help us understand the brain,” says Cox.
Brain-like neural networks
For the new research, Cox and DiCarlo joined Joel Dapello and Tiago Marques, the lead authors of the paper, to see whether neural networks become more robust to adversarial attacks when their activations are similar to brain activity. The AI researchers tested several popular CNN architectures trained on the ImageNet data set, including AlexNet, VGG and different variations of ResNet. They also included some deep learning models that had undergone “adversarial training”, a process in which a neural network is trained on adversarial examples in order to avoid misclassifying them.
The scientist evaluated the AI models using the “BrainScore” metric, which compares activations in deep neural networks with neural responses in the brain . They then measured the robustness of each model by testing it against white-box adversarial attacks, in which an attacker has full knowledge of the structure and parameters of the target neural networks. “Surprisingly, the more brain-like a model was, the more robust the system was against adversarial attacks,” says Cox.
To go down into the depths, and deeper still, to stay there and think a little, takes a little fearlessness, a little resolve to be able to give up what one knows, and the courage to face, when struck with wonder, the new and wholly unknown KNOWLEDGE one may encounter in the depths. In order to make headway on ethical and moral values in the light of science, it is “exactly the time to think”!
Translated from the Turkish original.