Implementing an Improved Reinforcement Learning Algorithm for the Simulation of Weekly Activity-Travel Sequences

Abstract

Recently, within the area of activity-based travel demand modeling there is a general tendency to enhance the realism of these models by incorporating dynamics based on learning and adaptation processes. The research presented here attempts at contributing to the current state of the art by formulating a framework for the simulation of individual activity-travel patterns. To this end, the current research redesigns an existing reinforcement learning technique by adding a regression-tree function approximator. This artifice enables the Q-learning algorithm not only to consider more explanatory and decision variables, but also to handle a larger granularity of these dimensions. In addition, the reward function underlying the Q-learning process is drawn up carefully based on activity attributes rather than activity type. For the purpose of testing the applicability of the proposed improvements, a prototype model is implemented and applied to real-world data. The prototype model proves to learn weekly activity-travel patterns rather quickly, requiring only a limited amount of memory. Additionally, in order to validate the suggested approach, the simulated weekly activity-travel sequences are compared to the observed ones by assessing the dissimilarity based on a number of distance measures.