NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps
Ríos Navarro, José Antonio
Tapiador Morales, Ricardo
Milde, Moritz B.
Linares Barranco, Alejandro
|Department||Universidad de Sevilla. Departamento de Arquitectura y Tecnología de Computadores|
|Abstract||Convolutional neural networks (CNNs) have become
the dominant neural network architecture for solving many stateof-
the-art (SOA) visual processing tasks. Even though Graphical
Processing Units (GPUs) are most often ...
Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving many stateof- the-art (SOA) visual processing tasks. Even though Graphical Processing Units (GPUs) are most often used in training and deploying CNNs, their power efficiency is less than 10 GOp/s/W for single-frame runtime inference.We propose a flexible and efficient CNN accelerator architecture called NullHop that implements SOA CNNs useful for low-power and low-latency application scenarios. NullHop exploits the sparsity of neuron activations in CNNs to accelerate the computation and reduce memory requirements. The flexible architecture allows high utilization of available computing resources across kernel sizes ranging from 1x1 to 7x7. NullHop can process up to 128 input and 128 output feature maps per layer in a single pass. We implemented the proposed architecture on a Xilinx Zynq FPGA platform and present results showing how our implementation reduces external memory transfers and compute time in five different CNNs ranging from small ones up to the widely known large VGG16 and VGG19 CNNs. Post-synthesis simulations using Mentor Modelsim in a 28nm process with a clock frequency of 500MHz show that the VGG19 network achieves over 450GOp/s. By exploiting sparsity, NullHop achieves an efficiency of 368%, maintains over 98% utilization of the MAC units, and achieves a power efficiency of over 3TOp/s/W in a core area of 6.3mm2. As further proof of NullHop’s usability, we interfaced its FPGA implementation with a neuromorphic event camera for real time interactive demonstrations.
|Citation||Aimar, A., Mostafa, H., Calabrese, E., Ríos Navarro, J.A., Tapiador Morales, R., Lungu, I.,...,Delbruck, T. (2019). NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps. IEEE Transactions on Neural Networks and Learning Systems, 30 (3), 644-656.|