神经网络结构图分析:正向输出计算和误差反向传播
(1) 输入层到H1层:
$z_1 = w_1x_1+w_2x_2+b_1 = 1 \times 1 + (-2) \times (-1) + 1 \times 1 + 0 = 4$
$z_2 = w_3x_1+w_4x_2+b_2 = (-1) \times 1 + 1 \times (-1) + 0 = -1$
$h_1 = \sigma(z_1) = \frac{1}{1+e^{-4}} \approx 0.982$
$h_2 = \sigma(z_2) = \frac{1}{1+e^{1}} \approx 0.268$
H1层到H2层:
$z_1 = w_1h_1+w_2h_2+b_1 = 2 \times 0.982 + (-1) \times 0.268 + 0 = 1.696$
$z_2 = w_3h_1+w_4h_2+b_2 = (-2) \times 0.982 + (-1) \times 0.268 + 0 = -2.218$
$h_3 = \sigma(z_1) = \frac{1}{1+e^{-1.696}} \approx 0.845$
$h_4 = \sigma(z_2) = \frac{1}{1+e^{2.218}} \approx 0.098$
H2层到输出层:
$z_1 = w_1h_3+w_2h_4+b_1 = 3 \times 0.845 + (-1) \times 0.098 + (-2) = 0.437$
$z_2 = w_3h_3+w_4h_4+b_2 = (-1) \times 0.845 + 4 \times 0.098 + 2 = 2.422$
$o_1 = \sigma(z_1) = \frac{1}{1+e^{-0.437}} \approx 0.607$
$o_2 = \sigma(z_2) = \frac{1}{1+e^{-2.422}} \approx 0.918$
(2) 真实输出为$(0.5,1)$,则误差为:
$E = \frac{1}{2}\sum_{i=1}^{2}(y_i-o_i)^2 = \frac{1}{2}[(0.5-0.607)^2+(1-0.918)^2] \approx 0.067$
根据误差反向传播算法,可以计算输出层和H2层的误差信号:
$\delta_{o_1} = o_1(1-o_1)(y_1-o_1) \approx -0.031$
$\delta_{o_2} = o_2(1-o_2)(y_2-o_2) \approx 0.057$
$\delta_{h_3} = h_3(1-h_3)(w_1\delta_{o_1}+w_3\delta_{o_2}) \approx 0.016$
$\delta_{h_4} = h_4(1-h_4)(w_2\delta_{o_1}+w_4\delta_{o_2}) \approx -0.002$
然后根据误差信号和梯度下降算法,可以更新H2层到输出层的权重和偏置:
$w_1 = w_1 - \eta\delta_{o_1}h_3 \approx 2.975$
$w_2 = w_2 - \eta\delta_{o_1}h_4 \approx -1.001$
$w_3 = w_3 - \eta\delta_{o_2}h_3 \approx -1.015$
$w_4 = w_4 - \eta\delta_{o_2}h_4 \approx 4.054$
$b_1 = b_1 - \eta\delta_{o_1} \approx -2.001$
$b_2 = b_2 - \eta\delta_{o_2} \approx 2.057$
其中$\eta$为学习率,可以根据实际情况进行调整。
最后,根据误差信号和梯度下降算法,可以更新H1层到H2层的权重和偏置:
$w_1 = w_1 - \eta\delta_{h_3}h_1 \approx 2.010$
$w_2 = w_2 - \eta\delta_{h_3}h_2 \approx -0.989$
$w_3 = w_3 - \eta\delta_{h_4}h_1 \approx -1.998$
$w_4 = w_4 - \eta\delta_{h_4}h_2 \approx -0.999$
$b_1 = b_1 - \eta\delta_{h_3} \approx 0.999$
$b_2 = b_2 - \eta\delta_{h_4} \approx -0.001$
同样地,学习率$\eta$可以根据实际情况进行调整。
原文地址: https://www.cveoy.top/t/topic/kwjj 著作权归作者所有。请勿转载和采集!