Alpha Precision: Estimating the Significant System Behavior in a Model

Abstract

One of the goals of process discovery is to construct, from a given event log, a process model which correctly represents the underlying system. As with any abstraction, one does not necessarily want to represent all possible behavior, but only the significant behavior. While various discovery algorithms support this use case of discovering the significant process behavior, proper evaluation measures for this use case appear to be missing. Therefore, this paper presents a new precision metric that quantifies to what extent the discovered model contains significant system behavior. Besides being a metric with a clear and intuitive interpretation, the metric distinguishes itself in two other areas. Firstly, it introduces the concept of a-significance, which only measures precision with respect to significant behavior. Secondly, it is designed as a system measure and estimates the precision with respect to the underlying system rather than the observed log. This work introduces a new precision measure and a statistical estimation method. Additionally, an empirical demonstration and evaluation of the metric are provided, which creates initial insights and knowledge about the performance and characteristics of the new measure. The results show that the a-precision measure provides a solid foundation for future work on developing quality measures for this particular use case.