A Simplified Transition Matrix Method for Learning with Noisy Labels

Introduction

Learning with noisy labels is a pervasive challenge in machine learning, as inaccurate labels can severely hamper the performance of classification models. To tackle this issue, various methods have been proposed, among which the transition matrix method has gained significant traction due to its simplicity and statistical consistency. The transition matrix effectively captures the probability of label transitions from true labels to noisy labels, providing crucial insights for accurate classification.

However, estimating the transition matrix for each sample can be problematic, particularly in scenarios where label noise is instance-dependent or when dealing with large real-world datasets. This process can be computationally expensive and may lead to unidentifiable solutions. To address these limitations, we introduce a simplified model that only requires estimating a global transition matrix, complemented by implicit regularization.

By estimating a global transition matrix, we can effectively capture the overall probability transfer from correct labels to noisy labels, thereby enhancing classification accuracy. Furthermore, we leverage implicit regularization techniques to fine-tune the sparse form representation of the difference between the estimated posterior probability distribution and the noisy label distribution. This approach proves to be adaptable to diverse types of label noise and effectively mitigates the issue of inaccurate posterior probability estimation.

To rigorously evaluate the effectiveness and consistency of our proposed method, we conducted theoretical analysis and extensive experiments on synthetic and real datasets with various types of label noise. Our experimental findings demonstrate that our method consistently outperforms previous transition matrix methods and exhibits a broader range of applicability. Moreover, our method achieves competitive results without relying on additional auxiliary techniques, surpassing other state-of-the-art methods.

To foster reproducibility and facilitate further research in this field, we will make the code of our method publicly available as an open-source project on Github. This will allow the research community to readily access and utilize our method, promoting collaboration and advancing the state-of-the-art in learning with noisy labels.