Paper 2023/1526

Polynomial Time Cryptanalytic Extraction of Neural Network Models

Isaac A. Canales-Martínez, Technology Innovation Institute
Jorge Chavez-Saab, Technology Innovation Institute
Anna Hambitzer, Technology Innovation Institute
Francisco Rodríguez-Henríquez, Technology Innovation Institute
Nitin Satpute, Technology Innovation Institute
Adi Shamir, Weizmann Institute of Science
Abstract

Billions of dollars and countless GPU hours are currently spent on training Deep Neural Networks (DNNs) for a variety of tasks. Thus, it is essential to determine the difficulty of extracting all the parameters of such neural networks when given access to their black-box implementations. Many versions of this problem have been studied over the last 30 years, and the best current attack on ReLU-based deep neural networks was presented at Crypto’20 by Carlini, Jagielski, and Mironov. It resembles a differential chosen plaintext attack on a cryptosystem, which has a secret key embedded in its black-box implementation and requires a polynomial number of queries but an exponential amount of time (as a function of the number of neurons). In this paper, we improve this attack by developing several new techniques that enable us to extract with arbitrarily high precision all the real-valued parameters of a ReLU-based DNN using a polynomial number of queries and a polynomial amount of time. We demonstrate its practical efficiency by applying it to a full-sized neural network for classifying the CIFAR10 dataset, which has 3072 inputs, 8 hidden layers with 256 neurons each, and about 1.2 million neuronal parameters. An attack following the approach by Carlini et al. requires an exhaustive search over 2^256 possibilities. Our attack replaces this with our new techniques, which require only 30 minutes on a 256-core computer.

Metadata
Available format(s)
PDF
Category
Attacks and cryptanalysis
Publication info
Published by the IACR in EUROCRYPT 2024
Keywords
ReLU-Based Deep Neural NetworksNeural Network ExtractionPolynomial QueryPolynomial Time Differential Attack
Contact author(s)
isaac canales @ tii ae
jorge saab @ tii ae
anna hambitzer @ tii ae
francisco rodriguez @ tii ae
nitin satpute @ tii ae
adi shamir @ weizmann ac il
History
2024-06-11: last of 8 revisions
2023-10-06: received
See all versions
Short URL
https://ia.cr/2023/1526
License
Creative Commons Attribution
CC BY

BibTeX

@misc{cryptoeprint:2023/1526,
      author = {Isaac A. Canales-Martínez and Jorge Chavez-Saab and Anna Hambitzer and Francisco Rodríguez-Henríquez and Nitin Satpute and Adi Shamir},
      title = {Polynomial Time Cryptanalytic Extraction of Neural Network Models},
      howpublished = {Cryptology {ePrint} Archive, Paper 2023/1526},
      year = {2023},
      url = {https://eprint.iacr.org/2023/1526}
}
Note: In order to protect the privacy of readers, eprint.iacr.org does not use cookies or embedded third party content.