Measuring Distribution Difference: Kullback-Leibler Divergence for Noise-Dependent Samples

The previous section highlighted the limitations of the transfer matrix approach when applied to samples affected by noise, especially when the posterior estimation of noise probability is inaccurate. This challenge arises primarily due to the discrepancy between the probability distributions of A and B. To address this, we propose a metric to quantify the difference between probability distributions. This metric is known as the Kullback-Leibler divergence, which measures the discrepancy between two probability distributions. In our approach, we utilize the Kullback-Leibler divergence to gauge the disparity between the posterior estimation of noise probability and its true probability. By minimizing the Kullback-Leibler divergence, we can enhance our model's performance in scenarios where samples are influenced by noise.