How hackers can 'poison' open-source code

guest · Aug 15, 2021

How hackers can 'poison' open-source code
August 13, 2021
https://techxplore.com/news/2021-08-hackers-poison-open-source-code.html

Cornell Tech researchers have discovered a new type of online attack that can manipulate natural-language modeling systems and evade any known defense—with possible consequences ranging from modifying movie reviews to manipulating investment banks' machine-learning models to ignore negative news coverage that would affect a specific company's stock.

In a new paper, researchers found the implications of these types of hacks—which they call "code poisoning"—to be wide-reaching for everything from algorithmic trading to fake news and propaganda.

"With many companies and programmers using models and codes from open-source sites on the internet, this research shows how important it is to review and verify these materials before integrating them into your current system," said Eugene Bagdasaryan, a doctoral candidate at Cornell Tech and lead author of "Blind Backdoors in Deep Learning Models," which was presented Aug. 12 at the virtual USENIX Security '21 conference. The co-author is Vitaly Shmatikov, professor of computer science at Cornell and Cornell Tech.
Click to expand...

"With previous attacks, the attacker must access the model or data during training or deployment, which requires penetrating the victim's machine learning infrastructure," Shmatikov said. "With this new attack, the attack can be done in advance, before the model even exists or before the data is even collected—and a single attack can actually target multiple victims."

How can the "poisoners" be stopped? The research team proposed a defense against backdoor attacks based on detecting deviations from the model's original code. But even then, the defense can still be evaded.
Click to expand...

arXiv: Blind Backdoors in Deep Learning Models

We investigate a new method for injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code. We use it to demonstrate new classes of backdoors strictly more powerful than those in the prior literature: single-pixel and physical backdoors in ImageNet models, backdoors that switch the model to a covert, privacy-violating task, and backdoors that do not require inference-time input modifications.

Our attack is blind: the attacker cannot modify the training data, nor observe the execution of his code, nor access the resulting model. The attack code creates poisoned training inputs "on the fly," as the model is training, and uses multi-objective optimization to achieve high accuracy on both the main and backdoor tasks. We show how a blind attack can evade any known defense and propose new ones.
Click to expand...

Log in or Sign up

How hackers can 'poison' open-source code

guest Guest

Log in or Sign up

How hackers can 'poison' open-source code

guest Guest

Useful Searches