Exploring the expected quantizing scheme with suitable mixed-precision policy is the key to compress deep neural networks(DNNs)in high efficiency and accuracy.This exploration implies heavy workloads for domain expert...Exploring the expected quantizing scheme with suitable mixed-precision policy is the key to compress deep neural networks(DNNs)in high efficiency and accuracy.This exploration implies heavy workloads for domain experts,and an automatic compression method is needed.However,the huge search space of the automatic method introduces plenty of computing budgets that make the automatic process challenging to be applied in real scenarios.In this paper,we propose an end-to-end framework named AutoQNN,for automatically quantizing different layers utilizing different schemes and bitwidths without any human labor.AutoQNN can seek desirable quantizing schemes and mixed-precision policies for mainstream DNN models efficiently by involving three techniques:quantizing scheme search(QSS),quantizing precision learning(QPL),and quantized architecture generation(QAG).QSS introduces five quantizing schemes and defines three new schemes as a candidate set for scheme search,and then uses the Differentiable Neural Architecture Search(DNAS)algorithm to seek the layer-or model-desired scheme from the set.QPL is the first method to learn mixed-precision policies by reparameterizing the bitwidths of quantizing schemes,to the best of our knowledge.QPL optimizes both classification loss and precision loss of DNNs efficiently and obtains the relatively optimal mixed-precision model within limited model size and memory footprint.QAG is designed to convert arbitrary architectures into corresponding quantized ones without manual intervention,to facilitate end-to-end neural network quantization.We have implemented AutoQNN and integrated it into Keras.Extensive experiments demonstrate that AutoQNN can consistently outperform state-of-the-art quantization.For 2-bit weight and activation of AlexNet and ResNet18,AutoQNN can achieve the accuracy results of 59.75%and 68.86%,respectively,and obtain accuracy improvements by up to 1.65%and 1.74%,respectively,compared with state-of-the-art methods.Especially,compared with the full-precision AlexNet and ResN展开更多
In lightweight cryptographic primitives, round functions with only simple operations XOR, modular addition and rotation are widely used nowadays. This kind of ciphers is called ARX ciphers. For ARX ciphers, impossible...In lightweight cryptographic primitives, round functions with only simple operations XOR, modular addition and rotation are widely used nowadays. This kind of ciphers is called ARX ciphers. For ARX ciphers, impossible differential cryptanalysis and zero-correlation linear cryptanalysis are among the most powerful attacks, and the key problems for these two attacks are discovering more and longer impossible differentials(IDs) and zero-correlation linear hulls(ZCLHs). However, finding new IDs and ZCLHs for ARX ciphers has been a manual work for a long time, which has been an obstacle in improving these two attacks. This paper proposes an automatic search method to improve the efficiency of finding new IDs and ZCLHs for ARX ciphers. In order to prove the efficiency of this new tool, we take HIGHT, LEA, SPECK three typical ARX algorithms as examples to explore their longer and new impossible differentials and zero-correlation linear hulls. To the best of our knowledge, this is the first application of automatic search method for ARX ciphers on finding new IDs and ZCLHs. For HIGHT, we find more 17 round IDs and multiple 17 round ZCLHs. This is the first discovery of 17 round ZCLHs for HIGHT. For LEA, we find extra four 10 round IDs and several 9 round ZCLHs. In the specification of LEA, the designers just identified three 10 round IDs and one 7round ZCLH. For SPECK, we find thousands of 6 round IDs and forty-four 6 round ZCLHs. Neither IDs nor ZCLHs of SPECK has been proposed before. The successful application of our new tool shows great potential in improving the impossible differential cryptanalysis and zero-correlation linear cryptanalysis on ARX ciphers..展开更多
基金supported by the China Postdoctoral Science Foundation under Grant No.2022M721707the National Natural Science Foundation of China under Grant Nos.62002175 and 62272248+1 种基金the Special Funding for Excellent Enterprise Technology Correspondent of Tianjin under Grant No.21YDTPJC00380the Open Project Foundation of Information Security Evaluation Center of Civil Aviation,Civil Aviation University of China,under Grant No.ISECCA-202102.
文摘Exploring the expected quantizing scheme with suitable mixed-precision policy is the key to compress deep neural networks(DNNs)in high efficiency and accuracy.This exploration implies heavy workloads for domain experts,and an automatic compression method is needed.However,the huge search space of the automatic method introduces plenty of computing budgets that make the automatic process challenging to be applied in real scenarios.In this paper,we propose an end-to-end framework named AutoQNN,for automatically quantizing different layers utilizing different schemes and bitwidths without any human labor.AutoQNN can seek desirable quantizing schemes and mixed-precision policies for mainstream DNN models efficiently by involving three techniques:quantizing scheme search(QSS),quantizing precision learning(QPL),and quantized architecture generation(QAG).QSS introduces five quantizing schemes and defines three new schemes as a candidate set for scheme search,and then uses the Differentiable Neural Architecture Search(DNAS)algorithm to seek the layer-or model-desired scheme from the set.QPL is the first method to learn mixed-precision policies by reparameterizing the bitwidths of quantizing schemes,to the best of our knowledge.QPL optimizes both classification loss and precision loss of DNNs efficiently and obtains the relatively optimal mixed-precision model within limited model size and memory footprint.QAG is designed to convert arbitrary architectures into corresponding quantized ones without manual intervention,to facilitate end-to-end neural network quantization.We have implemented AutoQNN and integrated it into Keras.Extensive experiments demonstrate that AutoQNN can consistently outperform state-of-the-art quantization.For 2-bit weight and activation of AlexNet and ResNet18,AutoQNN can achieve the accuracy results of 59.75%and 68.86%,respectively,and obtain accuracy improvements by up to 1.65%and 1.74%,respectively,compared with state-of-the-art methods.Especially,compared with the full-precision AlexNet and ResN
基金supported by the National Natural Science Foundation of China under Grant No. 61572516, 61402523, 61202491, 61272041 and 61272488
文摘In lightweight cryptographic primitives, round functions with only simple operations XOR, modular addition and rotation are widely used nowadays. This kind of ciphers is called ARX ciphers. For ARX ciphers, impossible differential cryptanalysis and zero-correlation linear cryptanalysis are among the most powerful attacks, and the key problems for these two attacks are discovering more and longer impossible differentials(IDs) and zero-correlation linear hulls(ZCLHs). However, finding new IDs and ZCLHs for ARX ciphers has been a manual work for a long time, which has been an obstacle in improving these two attacks. This paper proposes an automatic search method to improve the efficiency of finding new IDs and ZCLHs for ARX ciphers. In order to prove the efficiency of this new tool, we take HIGHT, LEA, SPECK three typical ARX algorithms as examples to explore their longer and new impossible differentials and zero-correlation linear hulls. To the best of our knowledge, this is the first application of automatic search method for ARX ciphers on finding new IDs and ZCLHs. For HIGHT, we find more 17 round IDs and multiple 17 round ZCLHs. This is the first discovery of 17 round ZCLHs for HIGHT. For LEA, we find extra four 10 round IDs and several 9 round ZCLHs. In the specification of LEA, the designers just identified three 10 round IDs and one 7round ZCLH. For SPECK, we find thousands of 6 round IDs and forty-four 6 round ZCLHs. Neither IDs nor ZCLHs of SPECK has been proposed before. The successful application of our new tool shows great potential in improving the impossible differential cryptanalysis and zero-correlation linear cryptanalysis on ARX ciphers..
