STFMCNN Performance Evaluation: A Comparative Study on Real Images

This research evaluates the effectiveness of the proposed STFMCNN method by conducting two experiments using real images. The objective of the first experiment is to fuse Landsat-8 images as CR images and Sentinel-2 images as FR images. Conversely, the second experiment aims to blend MODIS images as CR images and Landsat-5 images as FR images. Notably, the first experiment exhibits obvious gradual changes with few abrupt changes, while the second experiment showcases prominent abrupt changes with a limited number of gradual changes. The spatial resolution ratios of FR and CR images in the first experiment are 3, whereas the ratio is 16 in the second experiment. These two experiments are employed to thoroughly examine the impact of different spatial resolution ratios on the performance of the four STF methods, recognizing the crucial role of this factor in STF [41].

The surface reflectance products (i.e., Level 2 products) of Landsat images were obtained from the United States Geological Survey (USGS) (http://earthexplorer.usgs.gov/). Similarly, the MCD43A4 surface reflectance products of MODIS images were downloaded from USGS, with MCD43A4 products representing daily surface reflectance images adjusted by the bidirectional reflectance distribution function. The surface reflectance products (i.e., L2A products) of Sentinel-2 multispectral images were acquired from the European Space Agency (ESA) (https://scihub.copernicus.eu/dhus/#/home). In the first experiment, four spectral bands from both Landsat 8 and Sentinel-2 surface reflectance images were utilized: blue (B), green (G), red (R), and near infrared (NIR) bands. The second experiment employed six spectral bands from both MCD43A4 and Landsat-5 surface reflectance images: Blue (B), Green (G), Red (R), NIR, and two short wavelength infrared bands (SWIR-1 and SWIR-2).

To train the proposed STFMCNN, the three input CR difference images were initially resampled to the same size as the two input FR images. Subsequently, they were randomly cropped into patches with 128 × 128 pixels, resulting in five thousand patches for each experiment. The batch size was set to 8 to fit within the GPU memory, and the number of training iterations was set to 50. To optimize STFMCNN during the training phase, the momentum, weight decay, and learning rate were set to 0.9, 10-6, and 0.01, respectively. STFMCNN was implemented using the Keras package and executed on an NVIDIA GeForce GTX 1080Ti GPU with 11 GB of RAM. The parameter α in (5) was set to an empirical value of 1 in [22]. It's important to note that both experiments employed the same STFMCNN structure, with the only difference being the number of input and output image bands. Each experiment was conducted separately to produce a model specifically designed for handling the unique characteristics of its respective experimental data.