Multi-View Information Fusion Methods for Medical Imaging: A Comprehensive Overview

The combination of multi-view information is crucial in both CEM imaging and mammography. As far as we know, there is currently no systematic and complete method for combining multi-view information in CEM imaging. In mammography, there are many methods for combining multi-view information, but most of them use a simple strategy: feature concatenation. For example, Sun et al. used two independent CNN networks to extract features from the CC and MLO views, and then concatenated the extracted features and input them into the subsequent network. Nasir et al. used four independent CNN networks to extract features from four views, and then concatenated the features after the global pooling layer. They then input the fused features into a fully connected (fc) layer to obtain the final result. Wu et al. also concatenated features after global pooling, but their work has two differences compared to Nasir et al.: (1) they used two consecutive fc layers after feature concatenation to allow the model to better learn the relationship between different view features; (2) all CC views share one feature extraction network, and all MLO views share another feature extraction network. In addition, they also compared the performance when using different fusion strategies. Li et al. compared the performance of extracting features from different views using either independent model parameters or shared model parameters, and the conclusion was that the network can achieve better results when using shared model parameters to extract features from different views. Other networks also use feature concatenation strategies at different positions in the model to fuse multi-view information (e.g. Liu et al., Yan et al., Yang et al.). Since different views may not be aligned, most concatenation-based methods fuse multi-view information after the global pooling layer. As a result, they lose a large amount of local information and cannot fully exploit the multi-view information.