Evaluating ROCKET and Catch22 features for calf behaviour classification from accelerometer data using Machine Learning models

Summary

Monitoring calf behaviour continuously would be beneficial to identify routine practices (e.g., weaning, transport, dehorning, etc.) that impact calf welfare in dairy farms. In that regard, accelerometer data collected from neck collars can be used along with Machine Learning models to classify calf behaviour automatically but further development is needed to classify a broad spectrum of behaviours with good genericity. While the performance of Machine Learning models has been extensively studied, the accelerometer features used as inputs have received limited attention. In this paper, we explored the performance of ROCKET and Catch22 features that were specifically designed for time-series classification. For that purpose, 30 Irish Holstein Friesian and Jersey pre-weaned calves were filmed while equipped with an accelerometer sensor. Behaviours were annotated, allowing for 27.4 h of accelerometer time-series aligned with the corresponding behaviour. ROCKET and Catch22 features were extracted from raw and additional accelerometer time-series, along with commonly used features in the field, referred to as Hand-Crafted features. Each set of features was used to train three Machine Learning models (Random Forest, eXtreme Gradient Boosting, and RidgeClassifierCV) to classify six behaviours (drinking milk, grooming, lying, running, walking and other). 10 iterations from a validation set was used to tune each model with ROCKET, Catch22 and Hand-Crafted features extracted from various window sizes [3, 5 seconds] and overlap percentages [0, 25, 50%] between windows. For each feature set, the model achieving the best performance in the validation process was tested with its respective optimal window size and overlap using a test set composed of calves not used for model training. The best results were obtained with RidgeClassifierCV regardless of the features set. The highest performance was achieved with ROCKET (Balanced Accuracy: 0.81), followed by Catch22 (Balanced Accuracy: 0.74), both outperforming Hand-Crafted features (Balanced Accuracy: 0.66). These results highlight that the choice of accelerometer features must be considered as carefully as the models themselves. In particular, ROCKET features could help overcome current limitations, enabling the classification models to be used on farms to improve calf welfare.