Reclaiming Knowledge Structuring from Automation: Human-AI Co-Engineering of KC Models

Abstract

Adaptive learning environments use knowledge tracing models to estimate learners' acquisition of different skills and concepts. These models rely on mappings between problem items and their associated Knowledge Components (KCs), which are often encoded in a Q-matrix. Consequently, the Q-matrix strongly influences the effectiveness of these learning environments. Existing Q-matrix construction approaches span expert-driven, automated, and human-in-the-loop methods, but typically treat development as a one-directional process. We identified key limitations in static representations and limited traceability in tightly coupled pipelines. As these limitations hinder validation and iterative refinement, we propose a framework for interactive human-AI co-engineering of Q-matrices. This framework enables continuous collaboration between human experts and AI systems, supporting both initial construction and ongoing refinement. It is based on a dynamic graph representation, explicit roles for human and AI contributors, and considers both the KCs and mappings as evolving data. Interactors can rely on probes for inspection, explanation, and simulation for iterative refinement. We argue that this approach enables more consistent, transparent, and controllable Q-matrix engineering.

Adaptive learning environments use knowledge tracing models to estimate learners' acquisition of different skills and concepts. These models rely on mappings between problem items and their associated Knowledge Components (KCs), which are often encoded in a Q-matrix. Consequently, the Q-matrix strongly influences the effectiveness of these learning environments. Existing Q-matrix construction approaches span expert-driven, automated, and human-in-the-loop methods, but typically treat development as a one-directional process. We identified key limitations in static representations and limited traceability in tightly coupled pipelines. As these limitations hinder validation and iterative refinement, we propose a framework for interactive human-AI co-engineering of Q-matrices. This framework enables continuous collaboration between human experts and AI systems, supporting both initial construction and ongoing refinement. It is based on a dynamic graph representation, explicit roles for human and AI contributors, and considers both the KCs and mappings as evolving data. Interactors can rely on probes for inspection , explanation, and simulation for iterative refinement. We argue that this approach enables more consistent, transparent, and controllable Q-matrix engineering.