Malware in Deep Neural Networks
Deep learning systems are becoming increasingly popular, but their security aspects are often overlooked by developers. You will learn how deep neural networks can be used as an attack vector, and either explore how these attacks can be mitigated, or perhaps develop even more effective attacks.
Recent work [1] has shown that it is possible to embed malware in neural network-based models at negligible cost to the model’s performance. The increasing popularity of neural networks in both commercial and personal applications makes this a significant security issue. A malicious actor may embed malware in a pretrained network, distribute it on TorchHub, HuggingFace or other hosting services, and infect whomever downloads the weights with an arbitrary payload given a trigger condition. Though malware can be embedded in PDF files and image files in much the same way - often referred to as stegomalware - neural networks pose a novel threat in that the neural network’s weight data do not follow a distribution that is as easily analyzed as other types of files, and is as a result harder to detect. As there is very scarce work on this matter, the objective of this project is to develop methods of mitigating and understanding this threat.
Goal
For this project there are two possible research directions:
- Developing defenses - e.g, detection methods, payload corruption, etc.
- Developing attacks - e.g. novel embedding methods or countermeasures to existing defenses
Of particular interest to us is the latter. We have developed a decent defense already based on randomly modifying the order the weights appear in memory at runtime, but we theorize it is possible to circumvent, for instance through modifying the payload to include indices or by using error-correction algorithms.
Given a successful development, it will be possible to submit the work to an academic conference or journal.
Learning outcomes
- Machine learning
- Security/malware
- Research
Qualifications
- Python proficiency
- Motivated, creative
- Some experience with ML frameworks like pytorch and tensorflow is an advantage but not required
Supervisors
- Birk Torpmann-Hagen
- Michael A. Riegler
References
[1] Dorjan Hitaj et al. “MaleficNet: Hiding Malware into Deep Neural Networks Using Spread-Spectrum Channel Coding”. In: European Symposium on Re-
search in Computer Security. Springer. 2022, pp. 425–444.
