Securing AI Systems: Preventing Manipulation and Emerging Threats
How can we prevent AI attacks?
The more we rely on AI systems, the higher the risk of manipulation becomes. The race to develop appropriate protective measures has begun.
Artificial intelligence is becoming an increasingly integral part of our everyday lives. But what if the algorithms used to control driverless cars, critical infrastructure, healthcare and much more are manipulated?
Currently, such attacks are still a rarity – but experts believe that the frequency will increase significantly as AI systems become more widespread. If we are to continue to rely on such automated systems, we need to ensure that AI systems cannot be tricked into making poor or even dangerous decisions.
Manipulation of AI systems
The concern that AI could be manipulated is, of course, not new. However, there is now a growing understanding of how deep learning algorithms can be tricked by minor – but imperceptible – changes. This, in turn, leads to misclassification of what the algorithm is studying.
Several years ago, researchers already showed how they could create adversarial 3D objects that would trick a neural network into thinking a turtle was a rifle. Professor Dawn Song (University of California, Berkeley) also showed how stickers on certain parts of a stop sign can trick AI into interpreting it as a speed limit sign instead.
When a human is still involved, such errors can be noticed in time. But if automation takes over more and more, there may soon be no one left to check the AI’s work.
Fight against the misuse of AI
Help could come from the U.S. Defense Advanced Research Projects Agency’s (DARPA) multi-million dollar GARD project, which has three main goals around fighting AI abuse:
- Develop algorithms that already protect machine learning from vulnerabilities and glitches
- Develop theories on how to ensure that AI algorithms are still protected against attacks as the technology becomes more advanced and more freely available
- Develop and share tools that can protect against attacks on AI systems and assess whether AI is well protected
To provide platforms, libraries, datasets, and training materials to the GARD program, DARPA partners with a number of technology companies, including IBM and Google. This allows the robustness of AI models and their defenses against current and future attacks to be evaluated.
A key component of GARD is the Armory virtual platform, which is available on GitHub. It serves as a testing environment for researchers who need repeatable, scalable, and robust assessments of defenses developed by others.
In the fight against AI misuse, building platforms and tools to assess and protect against today’s threats is already difficult enough. Figuring out what hackers will do against these systems tomorrow is even more difficult.
The risk of data poisoning
In addition to direct attacks on AI algorithms, so-called data poisoning also poses an enormous risk. This involves attackers altering the training data used to create the AI in order to influence the AI’s decisions from the outset. This risk is particularly prevalent when an AI is trained on a dataset that originates in the public domain – precisely when the public knows that this is the case.
Microsoft’s AI bot Tay represents an example of this. Microsoft sent him out on Twitter to interact with humans so he could learn to use natural language and speak like humans. Within a few hours, people had misled Tay into saying offensive things, so Microsoft eventually took him offline again – Securing AI Systems: Preventing Manipulation and Emerging Threats.
Source: ZDNET
