Evaluation of Algorithmic Contributions to Business Process Management

Abstract

Algorithms play an important role for business process management. Indeed, all of the 27 papers accepted for BPM 2023 refer to algorithms in one way or the other. Some of these research papers propose new algorithms, such as the paper on the Agent Miner [8], they compose analytical pipelines, such as the paper on process mining video data [5], or they discuss the evaluation of algorithms, such as the paper on sustainability dimensions [3]. So far, however, the BPM community has not developed explicit guidelines for evaluating algorithmic contributions.

This tutorial describes an overarching methodological framework of how algorithms can be researched. In turn, we discuss its foundations in three dimensions. First, we describe four ontological entities including real-world problem, algorithmic task, algorithm design, and algorithm implementations. Second, we distinguish knowledge contributions along two dimensions, that is a) knowledge relating to tasks and to designs and b) knowledge of and knowledge about. Third, we discuss how methodologically new or better knowledge is established. The objective of this tutorial is to present specific strategies how algorithms can be systematically researched for early career researchers. These are illustrated by good examples of BPM papers.