Contributions to the realization of DNN-based visual inference on embedded systems

Abstract

This thesis comprises a set of contributions to the state of the art of embedded computer vision systems. CNNs constitute an accurate and flexible approach for artificial vision. They significantly outperform traditional algorithms based on prescribed features. This has prompted the development of a myriad of specific hardware and software components tailored for these neural networks. However, CNNs are memory-hungry and computationally heavy, which notably hinders their integration in embedded devices for field deployments. Therefore, a primary goal of this thesis was to explore system architectures and configurations optimized in terms of power consumption, frame rate, compactness, and cost. In addition, flexibility and programmability have also been two design principles that we have kept in mind throughout the research conducted in this doctoral dissertation. This is why we employed widespread software libraries to endow low-cost low-power embedded commercial platforms with visual inference capabilities. The active development of these libraries will continuously improve the resulting performance from the underlying hardware. The implementation of visual inference on edge devices has been addressed from different perspectives, and a vast set of experimental results have been collected to validate the methodologies introduced. This has been done on diverse embedded hardware platforms (RPi 3B/4B, Odroid XU4, Jetson TX2, etc.), software frameworks (Caffe, TF, OpenCV, TVM, etc.), and CNN models (GoogLeNet, MobileNet, ResNet, etc.). We have also introduced FoMs adapted to the nature of the targeted evaluation in order to support application-level decisions on the basis of meaningful system parameters. A variety of tools and lab equipment have been employed for the comprehensive characterizations performed. From all this work, a major conclusion that can be drawn is that low-cost DNN embedding under real-time operation conditions with moderate-to-high accuracy is currently possible, but the implementation must be thoroughly planned in advanced, system components must be carefully selected, and long battery lifetime should not be expected yet. The procedures proposed in this thesis assist in these tasks and constitute guidelines for future enhanced realizations of embedded vision. Another relevant conclusion is that all abstraction levels, i.e., application, algorithm, software, and hardware, must be jointly considered, and the corresponding performance metrics vertically conveyed during the design, in order to accomplish competitive systems useful for real scenarios.