ICUE (Interactivity Centered Usability Evaluation) is an enhanced usability testing protocol created by the researcher. ICUE augments the facilitator’s role for usability testing, and offers strategies in developing and presenting usability tasks during a testing session. ICUE was designed to address weaknesses found in the usability evaluation of course management systems (CMSs) with traditional usability testing protocol. The critical interaction between two or more CMS users (e.g., learners, classmates, and instructors) are insufficiently examined with traditional usability testing protocol, as the facilitator is a passive observer and minimizes their interactions with the participant while tasks are attempted. When interactive eLearning activities are not observed during user tests, traditional usability testing protocol can be limited to a technical review of structural and functional elements in a CMS (e.g., interface design, debugging code, and navigation design), resulting in an incomplete evaluation.

ICUE suggests that CMS usability tests should incorporate authentic and interactive eLearning activities as task scenarios. In order to augment traditional usability testing protocol with the ability to observe social interactions in eLearning, the facilitator engages the user through the CMS tools during tasks designed to mimic eLearning activities. The facilitator also strategically modifies the CMS environment during the usability test, which simulates asynchronous activities of a fellow classmate and instructor in an eLearning scenario. In addition to modifying the facilitator’s role in ICUE, usability tasks are designed to be attempted by the user in multiple stages, which also mimics a user’s genuine eLearning experience. ICUE modifications afford an opportunity for evaluators to observe more authentic behaviors during CMS usability tests.

The goal of this inquiry was to offer a proof of concept for the ICUE protocol, in addition to introducing the benefits from ICUE that may inform the development of a prototype to evaluate CMSs. The inquiry’s case study encompassed a usability test, with the application of ICUE on an eLearning program’s CMS. Findings from this authentic testing environment for ICUE, illustrated that traditional usability testing protocol was insufficient in evaluating eLearning activities. Ten out of 34 usability problems from the usability test were attributable to ICUE, and further analysis of these problems, indicated meaningful impacts on CMS design.

Analysis of the usability test report revealed that the application of ICUE resulted in the identification of usability problems that might have gone undetected with traditional usability testing protocol. Although the implementation of ICUE increased the responsibilities of a usability test facilitator, the modifications did not appear to impact the usability test participant’s experience during testing. The benefits of ICUE identified in this instance of CMS usability testing, suggests that additional studies may inform the development of a smarter prototype for evaluation CMS evaluations.

As eLearning approaches evolve from advanced developments in CMSs, or shift to less defined personal learning environments, learners will continue to seek knowledge through technologies that facilitate social interaction. Usability testing techniques should adapt to meet the needs of increasingly complex and interactive eLearning technologies, as user testing can continue to help determine the success or failure of computer systems.