Using a Supervised Method and AI for Fixation Detection in Eye-Tracking Data in Immersive Virtual Reality

By Lorena Ferreira Carpes

Abstract
Eye tracking (ET) is an established method to explore human cognition. ET studies in Immersive Virtual Reality (IVR) utilize head-mounted displays (HMDs), which enable users to achieve a high degree of immersion. This immersion is beneficial for performing tasks that would be unfeasible or too difficult to accomplish in real life, providing valuable insights into individuals' behavior. Fixation classification is one of the most useful methods to model gaze-behavior patterns. However, there are few studies investigating fixation detection from ET data collected in IVR. This study evaluated fixation detection using the F1 score for two methods: Random Forest (RF) with features extracted from the literature (RF-literature) and Random Forest with an autoencoder (RF-autoencoder) for feature extraction. The performance of these methods was compared with an RF model without feature extraction and with an unsupervised clustering method. The results show that the unsupervised clustering method outperformed RF-literature and RF-autoencoder (F1=0.925), and that RF-literature (F1 = 0.891) outperformed RF-autoencoder (F1 = 0.773) and RF with no extracted features, which had the worst performance (F1 = 0.753). RF-autoencoder struggled with undefined events, frequently misclassifying them as fixations. This may be due to the similar characteristics between fixations and smooth pursuit movement, which the autoencoder failed to capture with the given feature set. This study concludes that RF-literature, besides the unsupervised clustering method developed by Robben(2024), has potential for fixation detection in ET data collected in IVR and contributes to the analysis of human behavior in IVR.

Keywords: eye-tracking; fixation identification; virtual reality; head-mounted display; random forest; AI