Application of Reinforcement Learning for continuous stirred tank reactor (CSTR) temperature control

Abstract

The Center for Industrial Process Technology (CIPT) is a research group of KU Leuven carrying out research about applications of Artificial Intelligence (AI) for chemical process control. In recent years, there has been a strong increase in using machine learning in terms of Reinforcement Learning (RL) for automation tasks. In RL, an agent learns what action to take based on a given state in order to reach the best performance of the task by interacting with its environment. This principle is already successfully applied in chemical process control for maximizing the concentration of a product. A state-of-the-art RL algorithm is Soft Actor-Critic (SAC) that has been proven to outperform other RL algorithms. Based on this, it is hypothesized that SAC will also outperform the previously used RL algorithms in chemical process control. In this Master’s thesis, a SAC RL algorithm is used for controlling the reactor temperature and concentration of cyclopentenol of the Van de Vusse reaction taking place inside a CSTR. The control inputs/actions for SAC are the heat removal value and the cooling jacket temperature. The performance of SAC is analyzed based on training steps, different time steps and stability. After training the SAC, the control structure was able to control the process within a temperature range of 375 ± 0.05 Kelvin or a concentration range of 1.10 ± 0.05 kmol/m3. In the end, recommendations for future work are described.