Would online participants in GCC score better on exams?

1- Study Overview and Uniqueness

This study discusses the relationship between a training program format and the exam scores of participants. We would like to know whether attending a training program online or in class would be associated with how well a participant would score on a multiple-choice exam by the end of the training. Our sample consists of 1,722 participants in almost 130 programs. They are professionals who work in the public and private sectors in Saudi Arabia, United Arab Emirates, Oman, Bahrain, and Kuwait. The training programs were offered between 2021 and 2022; 986 participants attended in the classroom, while 736 attended online using a video conferencing platform. All participants sat for a multiple-choice exam by the 5th day (the last day) consisting of 40 questions with a passing mark of 70 out of 100.

The importance of this study lies first in its uniqueness of topic and sample. Several studies tackled the effectiveness of online instructor-led learning, but they were mainly focused on academic students. This study, however, targets online learning for working professionals whose aspiration for standard learning majorly differs from university students. More importantly, the scarcity of studies in this area focusing on the GCC population makes us believe that the results are very insightful and representative of learning and development professionals in the region. Furthermore, we hope this will be one of many studies that specifically consider the uniqueness of GCC learners.

2- Hunches and Hypothesis

Being in the corporate training industry for more than 20 years drives you sometimes to highlight observations about participants, but one would not dare to state them as proven facts. Nevertheless, like any other researcher, it all starts with a scientific intuition that urges validation. In this case, hunches about online learners are stated forth:

• To learn online, the learner must have different skills from face-to-face training.
• Online participants must be at higher risk of interruptions and losing focus.
• Some participants had to attend online against their will or preference.
• Gender seems to be a predictor of learning outcomes.

The hypothesis that we wish to test in this study is below:

Learning acquisition – measured through exam scores – is predicted by training format (online or face-to-face).

Examining this hypothesis aims to inform learning and development professionals, although partially, on some factors that drive the choice of whether a training program should be offered in class or online, optimizing the effectiveness of learning activities in organizations.

3- Results

It has been remarked that 82% of classroom learners passed their exams, while 75% of online learners did. Additionally, the average exam score in the classroom was 80%, while it was 76% for online learners. Considering exam scores as indicators of knowledge acquisition, the results thus far favor classroom format for training delivery. Nevertheless, these observations are insufficient to claim that the score difference is influenced by the class format (Virtual Learning or Classroom).

Further testing on the data is required to validate the causality claim of our hypothesis. Hence, a logistic regression test is performed on the data set to explore whether the class format predicts passing the exam and thus is a driver of learning acquisition. The output of the logistic regression test is shown below.

Although this table might not be your regular data visualization tool, reading it is simpler than you think. The value in the red box tests whether our hypothesis claiming that learning acquisition is predicted by training format is accurate. Since the value there is less than 0.05, we are confident that our hypothesis is valid because this value measures the probability of our observations being due to chance. The output in the blue box compares the possibility of passing with the likelihood of failure for a classroom attendee. Consequently, the data shows that a classroom attendee is 1.55 times more likely to pass the exam than an online attendee. Finally, the value in the orange box tells us how much variation in the exam result is explained by the training format. In simpler words, many factors influence a training participant's test results, and each factor contributes to this result with a certain percentage. In our case, the training format explains 1.2% of the variation in exam results. Before you conclude that this number needs to be revised, remember that hoping the class format would be a significant predictor of knowledge acquisition is too ambitious.

4- Insights and Recommendations

Learning acquisition in training can be influenced by how interested participants are in the topic and how relevant it is to their jobs. It could be a function of the trainer's style and the course instructional design strategy. Acquiring new learning depends as well on the group dynamics in class. This is only to name a few factors. All the same, this study invites us to add the class format to the list when we are considering a suitable training program for our employees as learning and development professionals. Looking at the difference in average scores, the 4% increase in favor of classroom training does not recommend ultimately moving away from online training. It invites us to be more diligent in deciding wisely on whom to invite to online training and considering the learner preference while choosing learning delivery channels. Additionally, the causation relationship between passing or failing an exam and the class format invites learning and development professionals to consider classroom formats for training programs tied to certifications where participants must sit for an exam. A final recommendation is to be more careful when designing and delivering online learning classes to ensure that the engagement level of learners during the training will compensate for the natural disturbance that online learners might experience, given the fact that electronic devices such as phones and laptops invite people to move their attention away from the training to other work or personal activities – a dynamic that is more controlled usually in classroom training.