LeNet 改进：激活函数、池化方式、卷积核大小、正则化、网络结构优化

本文以 LeNet 为基础，分别对激活函数、池化方式、卷积核大小、正则化方法和网络结构进行改进，并比较改进前后的模型性能。

1. 激活函数的改进 将 LeNet 中的激活函数替换为 ReLU。

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.pool2 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.pool1(self.relu(self.conv1(x)))
        x = self.pool2(self.relu(self.conv2(x)))
        x = x.view(-1, 16 * 4 * 4)
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x)
        return x

2. 池化方式 平均池化改为最大池化。

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.pool2 = nn.MaxPool2d(2) # 改为最大池化
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.pool1(self.relu(self.conv1(x)))
        x = self.pool2(self.relu(self.conv2(x)))
        x = x.view(-1, 16 * 4 * 4)
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x)
        return x

3. 卷积核大小 将其中一个 55 的卷积核修改为 77。

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 7) # 修改卷积核大小为 7*7
        self.pool2 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.pool1(self.relu(self.conv1(x)))
        x = self.pool2(self.relu(self.conv2(x)))
        x = x.view(-1, 16 * 4 * 4)
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x)
        return x

4. 正则化方法

方法 1：在全连接层后加入 Dropout 层

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.pool2 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.dropout1 = nn.Dropout(p=0.5) # 新增 Dropout 层
        self.fc2 = nn.Linear(120, 84)
        self.dropout2 = nn.Dropout(p=0.5) # 新增 Dropout 层
        self.fc3 = nn.Linear(84, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.pool1(self.relu(self.conv1(x)))
        x = self.pool2(self.relu(self.conv2(x)))
        x = x.view(-1, 16 * 4 * 4)
        x = self.dropout1(self.relu(self.fc1(x))) # 加入 Dropout 层
        x = self.dropout2(self.relu(self.fc2(x))) # 加入 Dropout 层
        x = self.fc3(x)
        return x

方法 2：卷积层后加入 BatchNorm 层

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.bn1 = nn.BatchNorm2d(6) # 新增 BatchNorm 层
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.bn2 = nn.BatchNorm2d(16) # 新增 BatchNorm 层
        self.pool2 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.pool1(self.relu(self.bn1(self.conv1(x)))) # 加入 BatchNorm 层
        x = self.pool2(self.relu(self.bn2(self.conv2(x)))) # 加入 BatchNorm 层
        x = x.view(-1, 16 * 4 * 4)
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x)
        return x

5. 网络结构优化 将卷积核从 55 修改为 33，但增加网络的层数（注意调整步长）。

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 3, stride=1, padding=1) # 将卷积核从 5*5 修改为 3*3
        self.bn1 = nn.BatchNorm2d(6)
        self.conv2 = nn.Conv2d(6, 6, 3, stride=1, padding=1) # 新增一层卷积层
        self.bn2 = nn.BatchNorm2d(6)
        self.pool1 = nn.MaxPool2d(2)
        self.conv3 = nn.Conv2d(6, 16, 3, stride=1, padding=1) # 将卷积核从 5*5 修改为 3*3
        self.bn3 = nn.BatchNorm2d(16)
        self.conv4 = nn.Conv2d(16, 16, 3, stride=1, padding=1) # 新增一层卷积层
        self.bn4 = nn.BatchNorm2d(16)
        self.pool2 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(16 * 7 * 7, 120) # 调整全连接层的输入维度
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.pool1(self.relu(self.bn2(self.conv2(self.bn1(self.conv1(x)))))) # 新增一层卷积层和 BatchNorm 层
        x = self.pool2(self.relu(self.bn4(self.conv4(self.bn3(self.conv3(x)))))) # 新增一层卷积层和 BatchNorm 层
        x = x.view(-1, 16 * 7 * 7) # 调整全连接层的输入维度
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x)
        return x

6. 扩展任务

使用扩充后的数据集（如旋转后的图像）测试模型性能
对模型进行可视化分析，例如特征图可视化

总结本文通过对 LeNet 模型进行一系列改进，提升了模型的性能。改进方法包括：使用 ReLU 作为激活函数，采用最大池化，修改卷积核大小，使用 Dropout 或 BatchNorm 进行正则化，以及调整网络结构。这些改进方法可以有效地提高模型的准确率和泛化能力。