Article
A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM
Author/s | Velasco Montero, Delia
Goossens, Bart Fernández Berni, Jorge ![]() ![]() ![]() ![]() ![]() ![]() ![]() Rodríguez Vázquez, Ángel Benito ![]() ![]() ![]() ![]() ![]() ![]() ![]() Philips, Wilfried |
Department | Universidad de Sevilla. Departamento de Electrónica y Electromagnetismo |
Publication Date | 2023 |
Deposit Date | 2024-02-16 |
Published in |
|
Abstract | Extracting information of interest from continuous video streams is a strongly demanded computer vision task. For the realization of this task at the edge using the current de-facto standard approach, i.e., deep learning, ... Extracting information of interest from continuous video streams is a strongly demanded computer vision task. For the realization of this task at the edge using the current de-facto standard approach, i.e., deep learning, it is critical to optimize key performance metrics such as throughput and energy consumption according to prescribed application requirements. This allows achieving timely decision-making while extending the battery lifetime as much as possible. In this context, we propose a method to boost neural-network performance based on a co-execution strategy that exploits hardware heterogeneity on edge platforms. The enabling tool is Apache TVM, a highly efficient machine-learning compiler compatible with a diversity of hardware back-ends. The proposed approach solves the problem of network partitioning and distributes the workloads to make concurrent use of all the processors available on the board following a pipeline scheme. We conducted experiments on various popular CNNs compiled with TVM on the Jetson TX2 platform. The experimental results based on measurements show a significant improvement in throughput with respect to a single-processor execution, ranging from 14% to 150% over all tested networks. Power-efficient configurations were also identified, accomplishing energy reductions above 10%. |
Funding agencies | Flemish Government (Belgium) Ministerio de Ciencia e Innovación (MICIN). España Office of Naval Research (ONR). United States |
Project ID. | PID2021-128009OB-C31
![]() TED2021-131835B-I00 ![]() N00014-19-1-2156 ![]() |
Citation | Velasco Montero, D., Goossens, B., Fernández Berni, J., Rodríguez Vázquez, Á.B. y Philips, W. (2023). A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM. IEEE Access, 11, 35007-35021. https://doi.org/10.1109/ACCESS.2023.3264828. |
Files | Size | Format | View | Description |
---|---|---|---|---|
A Pipelining-Based Heterogeneo ... | 2.406Mb | ![]() | View/ | |