Please use this identifier to cite or link to this item:
http://hdl.handle.net/1942/36647
Title: | Structured precision skipping: Accelerating convolutional neural networks with budget-aware dynamic precision selection | Authors: | Huang, Kai CHEN, Siang LI, Bowen CLAESEN, Luc Yao, Hao Chen, Junjian Jiang, Xiaowen Liu, Zhili Xiong, Dongliang |
Issue Date: | 2022 | Publisher: | Source: | JOURNAL OF SYSTEMS ARCHITECTURE, 124 (Art N° 102403) | Abstract: | Despite the remarkable advancement in various intelligence tasks achieved by Convolutional Neural Networks, the massive computation and storage consumption limit applications on resource-constrained devices. Existing works explore to reduce computation cost by leveraging the input-dependent redundancy at runtime. The irregular dynamic sparsity distribution, however, limits the real speedup for dynamic models deployed in traditional neural network accelerators. To solve this problem, we propose an algorithm-architecture co-design, named structured precision skipping (SPS), to exploit the dynamic precision redundancy in statically quantized models. SPS computes most neurons in a lower precision and only a small portion of important neurons in a higher precision to preserve performance. Specifically, we first propose the structured dynamic block to exploit the dynamic sparsity in a structured manner. Based on the block, we then apply a budget-aware training method by inducing a budget regularization to learn the precision skipping under a target resource constraint. Finally, we present an architecture design based on the bit-serial architecture with support for SPS models, where only a predict controller module with small overhead is introduced. Extensive evaluation results demonstrate that SPS can achieve up to 1.5× speedup and 1.4× energy saving on various models and datasets with marginal accuracy loss. | Keywords: | Convolutional neural networks;Algorithm-architecture co-design;Model compression and acceleration;Dynamic quantization | Document URI: | http://hdl.handle.net/1942/36647 | ISSN: | 1383-7621 | e-ISSN: | 1873-6165 | DOI: | 10.1016/j.sysarc.2022.102403 | ISI #: | 000782573200004 | Category: | A1 | Type: | Journal Contribution | Validations: | ecoom 2023 |
Appears in Collections: | Research publications |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
1-s2.0-S1383762122000078-main.pdf Restricted Access | Published version | 1.45 MB | Adobe PDF | View/Open Request a copy |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.