Beyond the Flag: Preliminary Learner-Centered Design Considerations for AI-Supported Assessment in ODeL Physics Education

Abstract

Artificial intelligence (AI)-supported online proctoring is increasingly used in online and distance education, although its implementation continues to raise questions about anxiety, privacy, fairness, and technical reliability. This exploratory mixed-method case study examined learners’ experiences during the first implementation of an AI-supported online proctored examination in postgraduate physics education within an Open and Distance e-Learning (ODeL) environment. Ten learners participated in the examination, which required interpreting physics concepts and applying them to instructional tasks. Quantitative ratings and written responses were collected concurrently through an anonymous post-examination survey and integrated during interpretation. Eight of the ten learners perceived the system as useful in discouraging academic dishonesty, while six indicated that they would be comfortable using it again. Their responses also showed concerns about monitoring anxiety, technical interruptions, privacy, and the interpretation of AI-generated flags. The findings suggested that learners’ acceptance depended not only on the perceived usefulness of proctoring but also on how the assessment was explained, prepared for, supported and reviewed. Four preliminary design considerations were developed from the integrated findings: Purposeful Assessment, Transparent Procedures, Learner Preparation, and Technical Resilience. These principles offer preliminary guidance for designing AI-supported assessment in ODeL while keeping academic decisions under informed human judgment.