Limitations of Class Probability-Based Methods in Feature Representation Evaluation

While methods that rely on predicted class likelihoods offer insights into model performance, they suffer from a critical limitation: they disregard the intrinsic worth of the feature representation per se. This means that even if a method predicts the correct class with high confidence, it might be doing so based on a poor feature representation, leading to inaccurate evaluations. Consequently, focusing solely on class probabilities can be misleading and may not fully capture the effectiveness of the feature representation.