The usefulness of the Sequence Alignment Methods in validating rule-based activity-based forecasting models

Abstract

This research paper aims at achieving a better understanding of rule-based activity-based models, by proposing a new level of validation at the process model level in the A Learning-based Transportation Oriented Simulation System (ALBATROSS) model. To that effect, the work activity process model, which includes six decision steps, has been investigated. Each decision step is evaluated during the prediction of the individuals' schedules. There are specific decision steps that affect the execution pattern of the work activity process model. So, the comportment of execution in the process model contains activation dependency. This branches the execution and evaluation of each agent under examination. Sequence Alignment Methods (SAM) can be used to evaluate how similar/dissimilar the predicted and observed decision sequences are on an agent level. The original Chi-squared Automatic Interaction Detector decision trees at each decision step utilized in ALBATROSS are compared with other well known induction methods chosen to appraise the purpose of the analyses. The models are validated at four levels: the classifier or decision step level whereby confusion matrix statistics are used; The work activity trips Origin-Destination matrix level; the time of day work activity start time level, using a correlation coefficient; and the process model level, using SAM. The results of validation on the proposed process model level show conformity to all validation levels. In addition, the results provide additional information in better understanding the process model's behavior. Hence, introducing a new level of validation incur new knowledge and assess the predictive performance of rule-based activity-based models. And assist in identifying critical decision steps in the work activity process model.