Why We Build Our Own Machine Learning Models for Proctoring

Machine learning, especially computer vision, forms the backbone of modern automated proctoring systems. It plays a crucial role in analyzing webcam footage and screen recordings to assist educators in identifying potential exam violations. For instance, if a student leaves their workstation during an online exam, the computer vision system flags that section of the video for the instructor’s review.

At Respondus Monitor, we take a unique approach to machine learning that sets us apart from other proctoring solutions. Rather than relying on generic, off-the-shelf machine learning models, we develop and train our models in-house using data specific to online proctoring environments. While general-purpose models may be suitable for applications like security cameras or access control systems, they are not optimized for the nuances of exam proctoring.

The Advantages of Custom-Built Machine Learning Models

Our decision to develop custom models is rooted in the need to address the specific challenges of online testing environments, which include factors like indoor lighting, webcam quality, limited movement, and face-level camera angles. This tailored approach offers several key advantages:

1. Significant Reduction in False Positives

False positives occur when the software incorrectly flags an event as suspicious. In the context of proctoring, a false positive might involve flagging a legitimate behavior as a potential violation. Over the past year, our machine learning efforts have reduced Respondus Monitor’s false positive rate by 80%. This improvement means that when our system flags an event today, it is more likely to be truly relevant, enhancing the overall accuracy of the proctoring process.

2. Optimized Performance for Low-Bandwidth and Older Devices

Generic computer vision models are designed to detect a wide range of objects and behaviors, from animals to vehicles to facial expressions. However, these models are not optimized for the specific needs of exam proctoring. By focusing our machine learning efforts on the elements most relevant to proctoring, we keep our models small and efficient. This ensures that Respondus Monitor performs well even on low-bandwidth networks and older devices, such as outdated Chromebooks.

3. Commitment to Algorithm Fairness

Algorithm fairness is a critical consideration in machine learning, particularly in proctoring, where biases can lead to unfair treatment of certain groups. At Respondus Monitor, we rigorously test each new model across various demographic factors, including age, gender, skin tone, and other characteristics that could influence proctoring outcomes, such as hairstyles, glasses, and head coverings. This targeted testing, conducted with data from actual online testing environments, helps ensure that our models are fair and unbiased.

4. Foundation for Innovative Features

Building our own models allows us to focus on challenges unique to online proctoring. For example, it’s common for students to slouch or adjust their screens during an exam, resulting in only a partial view of their face in the webcam. To address this, we recently introduced a feature that prompts students to adjust their webcam angle when this issue arises, reducing the likelihood of unnecessary flags by the system.

Another feature we developed detects poor lighting conditions and gently prompts students to adjust their lighting, improving both the quality of the proctoring video and the student’s overall experience. These enhancements are possible because we control the entire machine learning process, allowing us to fine-tune our models for the specific conditions of online testing.

Conclusion

In the rapidly evolving landscape of online education, the importance of reliable and accurate proctoring cannot be overstated. By building and training our own machine learning models, Respondus Monitor ensures that our proctoring system is not only more accurate but also fairer and more adaptable to the unique challenges of online testing. This commitment to innovation and excellence sets us apart, providing educators with the tools they need to maintain the integrity of their exams while offering students a seamless and equitable testing experience.