To solve this problem, we need to compute the entropy of a random variable using its probability distribution. The entropy of a random variable is a measure of the uncertainty associated with it.

We'll start by writing a function called compute_entropy that takes a probability distribution as input and returns the entropy. Here's the implementation:

import math

def compute_entropy(prob_dist):
    entropy = 0
    for p in prob_dist:
        if p != 0:
            entropy -= p * math.log2(p)
    return entropy

Now, let's compute the word-level entropy of SST and QNLI using the distributions we created in Problem 2. Assuming you have already created the probability distributions for SST and QNLI, you can use the compute_entropy function to calculate their entropy.

sst_prob_dist = [0.2, 0.3, 0.1, 0.4]
qnli_prob_dist = [0.1, 0.2, 0.3, 0.4]

sst_entropy = compute_entropy(sst_prob_dist)
qnli_entropy = compute_entropy(qnli_prob_dist)

print("Word-level entropy of SST:", sst_entropy)
print("Word-level entropy of QNLI:", qnli_entropy)

You can replace sst_prob_dist and qnli_prob_dist with the actual probability distributions you created in Problem 2. Running this code will print the word-level entropy of SST and QNLI.

Note: Entropy is measured in bits, where a bit is the amount of information required to make a decision between two equally likely events. Higher entropy indicates higher uncertainty or randomness in the distribution.

PROBLEM 3 – Entropy 20pts Write a python function that computes the entropy of a random variable input as a probability distribution Use this function to compute the word-level entropy of SST and QNLI

原文地址: https://www.cveoy.top/t/topic/i59g 著作权归作者所有。请勿转载和采集!

免费AI点我,无需注册和登录