Spatial Attention Module in Deep Learning: Explanation and Implementation

The input features are denoted by F; AvgPool and MaxPool represent global average pooling and global max pooling, respectively; MLP denotes a multilayer perceptron consisting of two fully connected layers, where ReLU is used as the activation function for the first layer; σ represents the sigmoid activation function; MC is the channel attention module, and F' denotes the feature obtained after passing through the channel attention module. The spatial attention module processes the input features through the English Channel. The average and max pooling are used to process the input features with spatial attention, while compressing the channel dimension. After the pooling process, two 2D spatial matrices are formed, and the channel dimension is concatenated. Then, a convolutional layer is employed with a single output channel and the sigmoid activation function, as shown in Equations (3) and (4).