Generating and specializing declare ground truth models to support process discovery evaluation under behavioral change

Abstract

Evaluating process discovery algorithms requires ground truth models against which discovered models can be compared. A critical but underexplored dimension of such evaluations concerns how algorithms perform when the underlying process model is not static but evolving. For example, processes can become more or less restrictive over time due to regulatory, organizational, or operational changes. Addressing this requires the ability to systematically construct sets of process models that are hierarchically related through specialization and generalization. While procedural ground truth model generation is well established, declarative process mining lacks comparable support, and the ability to systematically specialize or generalize generated models is largely absent. This gap is consequential given the growing importance of declarative models in dynamic business environments, where behavior is defined through constraints rather than explicit execution paths. This paper addresses this gap by presenting algorithms for the synthetic generation and specialization of Declare ground truth models, following the methodology of algorithm engineering. The proposed specialization algorithm is grounded in a theorem that characterizes when one declarative model constitutes a specialization of another, enabling controlled reduction of allowable behavior through constraint modification. Together, these techniques support the construction of hierarchically related model sets for use in process discovery and conformance checking evaluations across a spectrum of behavioral flexibility. We provide knowledge about the design of the algorithms by evaluating effectiveness and efficiency. Results show that model generation succeeds in over 80% of cases when models contain between 1 and 25 constraints and involve 11 to 35 activities. Increasing the number of constraints negatively impacts success rates and execution time, with the generator exhibiting exponential and the specializer linear time trends. Template choice also affects performance, with Response and Precedence improving success rates while Exclusive Choice reduces them. These findings offer practical guidance for generating declarative ground truth models that are both realistic and systematically varied, supporting more rigorous evaluation of declarative process mining algorithms.