Velasco Montero, DeliaGoossens, BartFernández Berni, JorgeRodríguez Vázquez, Ángel BenitoPhilips, Wilfried2024-02-162024-02-162023Velasco Montero, D., Goossens, B., Fernández Berni, J., Rodríguez Vázquez, Á.B. y Philips, W. (2023). A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM. IEEE Access, 11, 35007-35021. https://doi.org/10.1109/ACCESS.2023.3264828.2169-3536https://hdl.handle.net/11441/155304Extracting information of interest from continuous video streams is a strongly demanded computer vision task. For the realization of this task at the edge using the current de-facto standard approach, i.e., deep learning, it is critical to optimize key performance metrics such as throughput and energy consumption according to prescribed application requirements. This allows achieving timely decision-making while extending the battery lifetime as much as possible. In this context, we propose a method to boost neural-network performance based on a co-execution strategy that exploits hardware heterogeneity on edge platforms. The enabling tool is Apache TVM, a highly efficient machine-learning compiler compatible with a diversity of hardware back-ends. The proposed approach solves the problem of network partitioning and distributes the workloads to make concurrent use of all the processors available on the board following a pipeline scheme. We conducted experiments on various popular CNNs compiled with TVM on the Jetson TX2 platform. The experimental results based on measurements show a significant improvement in throughput with respect to a single-processor execution, ranging from 14% to 150% over all tested networks. Power-efficient configurations were also identified, accomplishing energy reductions above 10%.application/pdf15 p.engAttribution-NonCommercial-NoDerivatives 4.0 Internacionalhttp://creativecommons.org/licenses/by-nc-nd/4.0/Apache TVMContinuous inferenceConvolutional neural networksEdge visionHeterogeneous processingJetson TX2Performance optimizationinfo:eu-repo/semantics/articleinfo:eu-repo/semantics/openAccesshttps://doi.org/10.1109/ACCESS.2023.3264828