Please use this identifier to cite or link to this item:
http://hdl.handle.net/1942/49320Full metadata record
| DC Field | Value | Language |
|---|---|---|
| dc.contributor.author | THYS, Jarne | - |
| dc.contributor.author | VANACKEN, Davy | - |
| dc.contributor.author | ROVELO RUIZ, Gustavo | - |
| dc.date.accessioned | 2026-06-16T11:30:56Z | - |
| dc.date.available | 2026-06-16T11:30:56Z | - |
| dc.date.issued | 2026 | - |
| dc.date.submitted | 2026-06-04T08:15:07Z | - |
| dc.identifier.citation | Fayolas, Camille; Van Gorp, Pieter; Baghdadi, Mahmoud; Ebert, Achim; Hu, Jun; Humayoun, Shah Rukh; Jaidka, Sapna; Luyten, Kris; Mentler, Tilo; Palanque, Philippe; Parvin, Parvaneh; Spano, Lucio Davide; Stumpf, Simone; Van Der Veer, Gerrit; Zaina, Luciana; Ziegler, Jürgen (Ed.). Lecture Notes in Computer Science, Springer Nature Switzerland, p. 105 -122 (Art N° 9) | - |
| dc.identifier.isbn | 978-3-032-26050-5 | - |
| dc.identifier.issn | 0302-9743 | - |
| dc.identifier.issn | 1611-3349 | - |
| dc.identifier.uri | http://hdl.handle.net/1942/49320 | - |
| dc.description.abstract | AutoML systems targeting novices often prioritize algorithmic automation over usability, leaving gaps in users' understanding, trust, and end-to-end workflow support. To address these issues, we propose an abstract pipeline that covers data intake, guided configuration, training, evaluation, and inference. To examine the abstract pipeline, we report a user study where we assess trust, understandability, and UX of a prototype implementation. In a 24-participant study, all participants successfully built their own models, UEQ ratings were positive, yet experienced users reported higher trust and understanding than novices. Based on this study, we propose four design principles to improve the design of AutoML systems targeting novices: (P1) support first-model success to enhance user self-efficacy, (P2) provide explanations to help users form correct mental models and develop appropriate levels of reliance, (P3) provide abstractions and context-aware assistance to keep users in their zone of proximal development, and (P4) ensure predictability and safeguards to strengthen users' sense of control. | - |
| dc.description.sponsorship | This work was supported by the Special Research Fund (BOF) of Hasselt University (BOF24OWB28). This research was made possible with support from the MAXVR-INFRA project, a scalable and flexible infrastructure that facilitates the transition to digital-physical work environments. The MAXVR-INFRA project is funded by the European Union - NextGenerationEU and the Flemish Government. | - |
| dc.language.iso | en | - |
| dc.publisher | Springer Nature Switzerland | - |
| dc.rights | © 2026 The Author(s), under exclusive license to Springer Nature Switzerland AG | - |
| dc.subject.other | AutoML | - |
| dc.subject.other | Large Language Models | - |
| dc.subject.other | Transformers | - |
| dc.subject.other | Text Classification | - |
| dc.subject.other | Conversational Assistant | - |
| dc.title | Engineering Trustworthy Automation: Design Principles and Evaluation for AutoML Tools for Novices | - |
| dc.type | Proceedings Paper | - |
| dc.relation.edition | 16511 | - |
| local.bibliographicCitation.authors | Fayolas, Camille | - |
| local.bibliographicCitation.authors | Van Gorp, Pieter | - |
| local.bibliographicCitation.authors | Baghdadi, Mahmoud | - |
| local.bibliographicCitation.authors | Ebert, Achim | - |
| local.bibliographicCitation.authors | Hu, Jun | - |
| local.bibliographicCitation.authors | Humayoun, Shah Rukh | - |
| local.bibliographicCitation.authors | Jaidka, Sapna | - |
| local.bibliographicCitation.authors | Luyten, Kris | - |
| local.bibliographicCitation.authors | Mentler, Tilo | - |
| local.bibliographicCitation.authors | Palanque, Philippe | - |
| local.bibliographicCitation.authors | Parvin, Parvaneh | - |
| local.bibliographicCitation.authors | Spano, Lucio Davide | - |
| local.bibliographicCitation.authors | Stumpf, Simone | - |
| local.bibliographicCitation.authors | Van Der Veer, Gerrit | - |
| local.bibliographicCitation.authors | Zaina, Luciana | - |
| local.bibliographicCitation.authors | Ziegler, Jürgen | - |
| local.bibliographicCitation.conferencedate | 2025, June 23-27 | - |
| local.bibliographicCitation.conferencename | The 17th ACM SIGCHI Symposium on Engineering Interactive Computing Systems | - |
| local.bibliographicCitation.conferenceplace | Trier, Germany | - |
| dc.identifier.epage | 122 | - |
| dc.identifier.spage | 105 | - |
| local.format.pages | 18 | - |
| local.bibliographicCitation.jcat | C1 | - |
| dc.relation.references | Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In Action control: From cognition to behavior (pp. 11–39). Springer. Allen, J., Galescu, L., Teng, C. M., & Perera, I. (2020). Conversational Agents for Complex Collaborative Tasks. AI Magazine, 41(4), 54–78. https://doi.org/10.1609/aimag.v41i4.7384 Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191 Brawner, K., Wang, N., & Nye, B. (2023). Teaching Artificial Intelligence (AI) with AI for AI applications. The International FLAIRS Conference Proceedings, 36. https://doi.org/10.32473/flairs.36.133388 Chakraborti, T., Agarwal, S., Khazaeni, Y., Rizk, Y., & Isahagian, V. (2020). D3BA: A Tool for Optimizing Business Processes Using Non-deterministic Planning. In A. Del Río Ortega, H. Leopold, & F. M. Santoro (Eds.), Business Process Management Workshops (pp. 181–193). Springer International Publishing. https://doi.org/10.1007/978-3-030-66498-5_14 Choi, J. I., Ahmadvand, A., & Agichtein, E. (2019). Offline and Online Satisfaction Prediction in Open-Domain Conversational Systems. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, 1281–1290. https://doi.org/10.1145/3357384.3358047 Crisan, A., & Fiore-Gartland, B. (2021). Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, 1–15. https://doi.org/10.1145/3411764.3445775 Ding, B., Qin, C., Zhao, R., Luo, T., Li, X., Chen, G., Xia, W., Hu, J., Luu, A. T., & Joty, S. (2024). Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and Challenges. Findings of the Association for Computational Linguistics ACL 2024, 1679–1705. https://doi.org/10.18653/v1/2024.findings-acl.97 Drori, I., Krishnamurthy, Y., Rampin, R., DE PAULA LOURENCO, R., Piazentin Ono, J., Cho, K., Silva, C., & Freire, J. (2021). AlphaD3M: Machine Learning Pipeline Synthesis. ICML AutoML Workshop. https://hdl.handle.net/10993/57634 Drozdal, J., Weisz, J., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M., Ju, L., & Su, H. (2020). Trust in AutoML: Exploring information needs for establishing trust in automated machine learning systems. Proceedings of the 25th International Conference on Intelligent User Interfaces, IUI ’20, 297–307. https://doi.org/10.1145/3377325.3377501 Eerlings, G., Vanbrabant, S., Liesenborgs, J., Rovelo Ruiz, G., Vanacken, D., & Luyten, K. (2025). AI-Spectra: A Visual Dashboard for Model Multiplicity to Enhance Informed and Transparent Decision-Making. In L. Zaina, J. C. Campos, D. Spano, K. Luyten, P. Palanque, G. van der Veer, A. Ebert, S. R. Humayoun, & V. Memmesheimer (Eds.), Engineering Interactive Computer Systems. EICS 2024 International Workshops (pp. 55–73). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-91760-8_5 Egelé, R., Guyon, I., Vishwanath, V., & Balaprakash, P. (2023). Asynchronous Decentralized Bayesian Optimization for Large Scale Hyperparameter Optimization. 2023 IEEE 19th International Conference on E-Science (e-Science), 1–10. https://doi.org/10.1109/e-Science58273.2023.10254839 Feng, K. J. K., & Mcdonald, D. W. (2023). Addressing UX Practitioners’ Challenges in Designing ML Applications: An Interactive Machine Learning Approach. Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI ’23, 337–352. https://doi.org/10.1145/3581641.3584064 Grassini, S. (2024). A Psychometric Validation of the PAILQ-6: Perceived Artificial Intelligence Literacy Questionnaire. Nordic Conference on Human-Computer Interaction, 1–10. https://doi.org/10.1145/3679318.3685359 Grudin, J., & Jacques, R. (2019). Chatbots, Humbots, and the Quest for Artificial General Intelligence. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, 1–11. https://doi.org/10.1145/3290605.3300439 He, J., Piorkowski, D., Muller, M., Brimijoin, K., Houde, S., & Weisz, J. (2023). Rebalancing Worker Initiative and AI Initiative in Future Work: Four Task Dimensions. Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work, CHIWORK ’23, 1–16. https://doi.org/10.1145/3596671.3598572 Hemmer, P., Schemmer, M., Kühl, N., Vössing, M., & Satzger, G. (2022). On the Effect of Information Asymmetry in Human-AI Teams. CHI Conference on Human Factors in Computing Systems (CHI ’22), Workshop on Human-Centered Explainable AI (HCXAI). CHI Conference on Human Factors in Computing Systems (CHI ’22), Workshop on Human-Centered Explainable AI (HCXAI). https://doi.org/10.48550/ARXIV.2205.01467 Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015). Principles of Explanatory Debugging to Personalize Interactive Machine Learning. Proceedings of the 20th International Conference on Intelligent User Interfaces, 126–137. https://doi.org/10.1145/2678025.2701399 Laugwitz, B., Held, T., & Schrepp, M. (2008). Construction and Evaluation of a User Experience Questionnaire. In A. Holzinger (Ed.), HCI and Usability for Education and Work (Vol. 5298, pp. 63–76). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-89350-9_6 Lee, J. D., & See, K. A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392 Lindauer, M., Karl, F., Klier, A., Moosbauer, J., Tornede, A., Mueller, A., Hutter, F., Feurer, M., & Bischl, B. (2024). Position: A call to action for a human-centered AutoML paradigm. Proceedings of the 41st International Conference on Machine Learning, ICML’24, 235, 30566–30584. Luo, D., Feng, C., Nong, Y., & Shen, Y. (2024). AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Models. Proceedings of the 32nd ACM International Conference on Multimedia, MM ’24, 8586–8594. https://doi.org/10.1145/3664647.3680665 Mumuni, A., & Mumuni, F. (2025). Automated data processing and feature engineering for deep learning and big data applications: A survey. Journal of Information and Intelligence, 3(2), 113–153. https://doi.org/10.1016/j.jiixd.2024.01.002 Olson, R. S., Bartley, N., Urbanowicz, R. J., & Moore, J. H. (2016). Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science. Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO ’16, 485–492. https://doi.org/10.1145/2908812.2908918 Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2022). Challenges in Deploying Machine Learning: A Survey of Case Studies. ACM Comput. Surv., 55(6), 114:1-114:29. https://doi.org/10.1145/3533378 Robino, G. (2025). Conversation Routines: A Prompt Engineering Framework for Task-Oriented Dialog Systems (arXiv:2501.11613). arXiv. https://doi.org/10.48550/arXiv.2501.11613 Ross, S. I., Martinez, F., Houde, S., Muller, M., & Weisz, J. D. (2023). The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development. Proceedings of the 28th International Conference on Intelligent User Interfaces, 491–514. https://doi.org/10.1145/3581641.3584037 Smith, M. J., Sala, C., Kanter, J. M., & Veeramachaneni, K. (2020). The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development. Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, SIGMOD ’20, 785–800. https://doi.org/10.1145/3318464.3386146 Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. Advances in Neural Information Processing Systems, 25. https://proceedings.neurips.cc/paper/2012/hash/05311655a15b75fab86956663e1819cd-Abstract.html Spitzer, P., Kühl, N., & Goutier, M. (2022). Training Novices: The Role of Human-AI Collaboration and Knowledge Transfer. Workshop on Human-Machine Collaboration and Teaming (HM-CaT 2022), The 39th International Conference on Machine Learning. Workshop on Human-Machine Collaboration and Teaming (HM-CaT 2022), The 39th International Conference on Machine Learning. https://doi.org/10.48550/ARXIV.2207.00497 Tayebi Arasteh, S., Han, T., Lotfinia, M., Kuhl, C., Kather, J. N., Truhn, D., & Nebelung, S. (2024). Large language models streamline automated machine learning for clinical studies. Nature Communications, 15(1), 1603. https://doi.org/10.1038/s41467-024-45879-8 Thys, J., Vanacken, D., & Rovelo Ruiz, G. (2025). Improving AI Text Classification: A Cascaded Approach. 3rd Workshop on Engineering Interactive Systems Embedding AI Technologies. http://hdl.handle.net/1942/46328 Vanbrabant, S., Eerlings, G., Rovelo Ruiz, G. A., & Vanacken, D. (2025). ECHO: Enhancing Conversational Explainable AI through Tool-Augmented Language Models. Proc. ACM Hum.-Comput. Interact., 9(4), EICS014:1-EICS014:33. https://doi.org/10.1145/3734191 Wertsch, J. V. (1979). From Social Interaction to Higher Psychological Processes A Clarification and Application of Vygotsky’s Theory. Human Development, 22(1), 1–22. https://doi.org/10.1159/000272425 Xiao, Z., Zhou, M. X., Liao, Q. V., Mark, G., Chi, C., Chen, W., & Yang, H. (2020). Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. ACM Trans. Comput.-Hum. Interact., 27(3), 15:1-15:37. https://doi.org/10.1145/3381804 Xin, D., Wu, E. Y., Lee, D. J.-L., Salehi, N., & Parameswaran, A. (2021). Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, 1–16. https://doi.org/10.1145/3411764.3445306 Yao, J., Zhang, L., & Huang, J. (2025). Evaluation of large language model-driven AutoML in data and model management from human-centered perspective. Frontiers in Artificial Intelligence, 8. https://doi.org/10.3389/frai.2025.1590105 You, J., Park, D., Song, J.-Y., & Suh, B. (2022). A Labeling Task Design for Supporting Recent Algorithmic Needs. 2022 IEEE International Conference on Big Data (Big Data), 2689–2698. https://doi.org/10.1109/bigdata55660.2022.10020415 | - |
| local.type.refereed | Refereed | - |
| local.type.specified | Proceedings Paper | - |
| local.bibliographicCitation.artnr | 9 | - |
| dc.identifier.doi | 10.1007/978-3-032-26051-2_9 | - |
| local.provider.type | CrossRef | - |
| local.bibliographicCitation.btitle | Lecture Notes in Computer Science | - |
| local.uhasselt.international | no | - |
| item.fulltext | With Fulltext | - |
| item.embargoEndDate | 2027-07-02 | - |
| item.fullcitation | THYS, Jarne; VANACKEN, Davy & ROVELO RUIZ, Gustavo (2026) Engineering Trustworthy Automation: Design Principles and Evaluation for AutoML Tools for Novices. In: Fayolas, Camille; Van Gorp, Pieter; Baghdadi, Mahmoud; Ebert, Achim; Hu, Jun; Humayoun, Shah Rukh; Jaidka, Sapna; Luyten, Kris; Mentler, Tilo; Palanque, Philippe; Parvin, Parvaneh; Spano, Lucio Davide; Stumpf, Simone; Van Der Veer, Gerrit; Zaina, Luciana; Ziegler, Jürgen (Ed.). Lecture Notes in Computer Science, Springer Nature Switzerland, p. 105 -122 (Art N° 9). | - |
| item.contributor | THYS, Jarne | - |
| item.contributor | VANACKEN, Davy | - |
| item.contributor | ROVELO RUIZ, Gustavo | - |
| item.accessRights | Embargoed Access | - |
| Appears in Collections: | Research publications | |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| accepted_manuscript.pdf Until 2027-07-02 | Peer-reviewed author version | 880.94 kB | Adobe PDF | View/Open Request a copy |
| published.pdf Restricted Access | Published version | 1.98 MB | Adobe PDF | View/Open Request a copy |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.