Generalizing Fuzzy SARSA Learning for Real-Time Operation of Irrigation Canals

Abstract

Recently, a continuous reinforcement learning model called fuzzy SARSA (state, action, reward, state, action) learning (FSL) was proposed for irrigation canals. The main problem related to FSL is its convergence and generalization in environments with many variables such as large irrigation canals and situations beyond training. Furthermore, due to its architecture, FSL may require high computation demands during its learning process. To deal with these issues, this work proposes a computationally lighter generalizing learned Q-function (GLQ) model, which benefits from the FSL-learned Q-function, to provide operators with a faster and simpler mechanism to obtain operational instructions. The proposed approach is tested for di erent water requests in the East Aghili Canal, located in the southwest of Iran. Several performance indicators are used for evaluating the GLQ model results, showing convergence in all the investigated cases and the ability to estimate operational instructions (actions) in situations beyond training, delivering water with high accuracy regarding several performance indicators. Hence, the use of the GLQ model is recommended for determining the operational patterns in irrigation canals.