Artículo
A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM
Autor/es | Velasco Montero, Delia
Goossens, Bart Fernández Berni, Jorge Rodríguez Vázquez, Ángel Benito Philips, Wilfried |
Departamento | Universidad de Sevilla. Departamento de Electrónica y Electromagnetismo |
Fecha de publicación | 2023 |
Fecha de depósito | 2024-02-16 |
Publicado en |
|
Resumen | Extracting information of interest from continuous video streams is a strongly demanded computer vision task. For the realization of this task at the edge using the current de-facto standard approach, i.e., deep learning, ... Extracting information of interest from continuous video streams is a strongly demanded computer vision task. For the realization of this task at the edge using the current de-facto standard approach, i.e., deep learning, it is critical to optimize key performance metrics such as throughput and energy consumption according to prescribed application requirements. This allows achieving timely decision-making while extending the battery lifetime as much as possible. In this context, we propose a method to boost neural-network performance based on a co-execution strategy that exploits hardware heterogeneity on edge platforms. The enabling tool is Apache TVM, a highly efficient machine-learning compiler compatible with a diversity of hardware back-ends. The proposed approach solves the problem of network partitioning and distributes the workloads to make concurrent use of all the processors available on the board following a pipeline scheme. We conducted experiments on various popular CNNs compiled with TVM on the Jetson TX2 platform. The experimental results based on measurements show a significant improvement in throughput with respect to a single-processor execution, ranging from 14% to 150% over all tested networks. Power-efficient configurations were also identified, accomplishing energy reductions above 10%. |
Agencias financiadoras | Flemish Government (Belgium) Ministerio de Ciencia e Innovación (MICIN). España Office of Naval Research (ONR). United States |
Identificador del proyecto | PID2021-128009OB-C31
TED2021-131835B-I00 N00014-19-1-2156 |
Cita | Velasco Montero, D., Goossens, B., Fernández Berni, J., Rodríguez Vázquez, Á.B. y Philips, W. (2023). A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM. IEEE Access, 11, 35007-35021. https://doi.org/10.1109/ACCESS.2023.3264828. |
Ficheros | Tamaño | Formato | Ver | Descripción |
---|---|---|---|---|
A Pipelining-Based Heterogeneo ... | 2.406Mb | [PDF] | Ver/ | |