HNN-SPOT: Enhancing Spatial Resolution of Coarse Resolution Images Using Recurrent Neural Networks

HNN is a type of recurrent artificial neural network and it is shown that it can always converge to a stable or equilibrium state with the use of symmetric weights of each neuron pair without self-connection [18]. This convergence property can therefore be used in energy minimization problems where the energy is minimized at the equilibrium state [19]. In real world problems, an energy can be defined as the summation of the goal to be achieved and the constraints to be followed [20]: Energy = Goal + Constraints. (1) HNN can then be used as a minimization tool to find the minimum energy for which the goal and the constraints are balanced. In remote sensing, HNN has been used in applications of feature identification/tracking; for example, for ice, clouds, and ocean currents [21], and for super-resolution target identification and mapping [20,22]. In this paper, HNN-SPOT is developed, and it further demonstrates in the process of spatio-temporal data fusion, for enhancing the spatial resolution of coarse resolution images.

In HNN-SPOT, one fine resolution image at an arbitrary date (noted as [Math Processing Error]) and coarse resolution image at the prediction date (noted as [Math Processing Error]) are used to derive the fine resolution image at the prediction date (noted as [Math Processing Error]). Intrinsically, the fusion model strikes a balance between the spatial details in [Math Processing Error] and the spectral response value in [Math Processing Error], and therefore the energy function can be literally expressed as: Energy = Spatial details in fine resolution + Spectral response in coarse resolution. (2)

In order to model the above energy function, the network architecture should firstly be well-defined. Thus, each pixel in the fine resolution space is deemed as a neuron and each neuron is identified by its location at the [Math Processing Error] row and the [Math Processing Error] column, denoted as [Math Processing Error]. Each neuron in the fine resolution space lies in its corresponding coarse pixel at the [Math Processing Error] row and the [Math Processing Error] column in the coarse resolution space and there are [Math Processing Error] neurons in each coarse pixel for the scale factor of [Math Processing Error] between the fine and coarse resolution images (Figure 3). The network is then initialized with the pixel values in [Math Processing Error] as the initial state of each neuron.