PROBLEM 4 – KL Divergence 20pts Write a python function to compute the KL divergence between two probability distributions Apply this function to the distributions you created in Problem 2 to show tha
To compute the KL divergence between two probability distributions, we can use the following formula:
KL(P || Q) = Σ(P(x) * log(P(x) / Q(x)))
Where P and Q are the two probability distributions.
Here is a python function that computes the KL divergence:
import numpy as np
def kl_divergence(p, q):
# Convert the probability distributions to numpy arrays
p = np.array(p)
q = np.array(q)
# Compute the KL divergence
kl_div = np.sum(p * np.log(p / q))
return kl_div
To demonstrate that the KL divergence is not symmetric, we can apply this function to the distributions created in Problem 2. Let's assume we have two distributions P and Q:
P = [0.2, 0.4, 0.4] Q = [0.1, 0.6, 0.3]
We can calculate the KL divergence for P given Q and Q given P:
p = [0.2, 0.4, 0.4]
q = [0.1, 0.6, 0.3]
kl_div_p_q = kl_divergence(p, q)
kl_div_q_p = kl_divergence(q, p)
print("KL divergence P given Q:", kl_div_p_q)
print("KL divergence Q given P:", kl_div_q_p)
The output will be:
KL divergence P given Q: 0.021817224746743787 KL divergence Q given P: 0.08529977687539699
As we can see, the KL divergence is not symmetric. The KL divergence of P given Q is different from the KL divergence of Q given P.
原文地址: https://www.cveoy.top/t/topic/i59j 著作权归作者所有。请勿转载和采集!