Development of a ROS environment for researching machine learning techniques applied to drones

Abstract

The first part of this dissertation presents ROS-MAGNA, a general framework for the definition and management of cooperative missions for multiple Unmanned Aircraft Systems (UAS) based on the Robot Operating System (ROS) [42]. This framework makes transparent the type of autopilot on-board and creates the state machines that control the behaviour of the different UAS from the specification of the multi-UAS mission. In addition, it integrates a virtual world generation tool to manage the information of the environment and visualize the geometrical objects of interest to properly follow the progress of the mission. The framework supports the coexistence of software-in-the-loop, hardware-in-the-loop and real UAS cooperating in the same arena, being a very useful testing tool for the developer of UAS advanced functionalities. To the best of our knowledge, it is the first framework which endows all these capabilities. The document also includes simulations and real experiments which show the main features of the framework. ROS-MAGNA is used to develop and test a machine learning tool. The information generated during a mission is used to train neural networks of different architecture for navigation purposes. The data treatment and training processes are accomplished in a testbench to select the best solution from different datasets. Tensorflow is the framework selected to implement every deep learning algorithm along with its Tensorboard tool for training understanding.Furthermore, an API with the pre-trained is used during a real mission in real time. The third part of this dissertation is the design and integration of a voice control assistant inside ROSMAGNA. Employing diverse online and offline tools, oral commands are processed to perform changes to the mission state and performance and to retrieve information.