The Influence of Hierarchical Labeling on Crop Type Classification using Hierarchical Deep Learning Architectures

By Tamara Stoof

Abstract
This paper studies the effect of introducing a hierarchical labeling structure to multi-spectral, multi-temporal satellite images (SITS) for crop type classification of 104 different crops using three hierarchical deep learning architectures. Two different label hierarchies have been used for this: 1) a manual expert-based hierarchy, and 2) a hierarchy based on KMEANS clustering of the spectral data. Three hierarchical models are compared to a reference non-hierarchical Multi-Scale-1D-ResNet model. The major difficulty with the SITS dataset is a severe class-imbalance, making the classification of the minority classes (92 classes) difficult. Two techniques -hierarchical labeling and model architectures, and Synthetic Minority Oversampling Technique (SMOTE), and a combination of the two methods- were used to handle the class imbalance. An ablation study using the class balanced CIFAR100 dataset was used to see if similar results would be produced as with the SITS dataset. The results of this study show that the introduction of hierarchies in the dataset and model architecture improves the classification of the rare, minority classes in a class-imbalanced dataset -if the hierarchy is appropriately chosen based on the spectral data. However, the influence of hierarchical models on class balanced datasets seems to be of little significance.