MSVR is developed from the original support vector machine (SVM) for realizing multivariate regression (Pérez-Cruz et al., 2002; Tuia et al., 2011). The mathematical expression is given as follows: 4y^=FMSVR(m)=φ(m)TW+B, where FMSVR(⋅) denotes the dataset regression model operator constructed based on MSVR and φ(m) is a non-linear regression function that implicitly maps the input vector m into a high-dimensional feature space. Its inner product defines the kernel function K(m,mi). (Here, we use the Gaussian radial basis function (RBF) kernel with a bandwidth parameter σ: K(m,mi)=φ(m)Tφ(mi)=exp⁡(-0.5∥m-mi∥2/σ2); mi denotes the ith model parameter vector from the surrogate model training dataset.) Assuming Nsamples denotes the number of surrogate model training samples, the regression coefficients W=[w1,…,wNobs]T∈RNobs×Nsamples and B=[b1,…,bNobs]T∈RNobs×1 are determined by minimizing the structural risk, as outlined in Eqs. (5) and (6): 5W,B= argminL(W,B)=12∑j=1Nobs‖wj‖2+C∑j=1NsamplesL(ui), where C is a penalty parameter; and L(u) is a quadratic ε-insensitive loss function, expressed as 6L(u)=0,u<ε(u-ε)2,u≥ε, where ui=‖ei‖=eiTei; eiT=yiT-φT(mi)W-BT. For ε=0, this problem is equivalent to an independent regularized kernel least squares regression for each component. For ε≠0, it becomes feasible to develop individual regression functions for each dimension based on the model outputs and to generate their corresponding support vectors. Solving the optimization problem directly is challenging, and the desired solutions for W and B are determined using an iterative reweighted least squares (IRWLS) procedure, employing the quasi-Newton approach. During the IRWLS process, the term L(u) in Eq. (5) is first transformed into a discrete first-order Taylor expansion, and the corresponding quadratic programming approximation is constructed. Meanwhile, a linear expression is derived based on the principle that the first-order derivatives of the objective function with respect to W and B are zero. Finally, the optimal values of W and B are obtained through a line search. Further details on the IRWLS procedure can be found in Sanchez-Fernandez et al. (2004).

The performance of the MSVR model is influenced by three hyperparameters: C, σ, and ε (Ma et al., 2022). This study optimizes these hyperparameters by minimizing the root mean square error (RMSE) using the four metaheuristic algorithms introduced in this study.

The three DNN models are all feedforward neural networks, which are generally constructed by stacking multiple hidden layers. The structure can be expressed as FDNN(m,θDNN)=fLNN(…fl(…f1(m))). Specifically, FDNN(⋅) and θDNN represent the DNN-based surrogate model operator and the corresponding trainable parameters, respectively. fl(⋅) denotes the non-linear transformation function of the lth layer, and LNN indicates the total number of neural network layers. In DNN model construction, various neural network layers can yield diverse DNN models, resulting in different predictive performances (LeCun et al., 2015). For the DNN models adopted in this study, the involved neural network types are the fully connected layer, the convolutional layer, and the residual block layer.

In fully connected layers, both input and output layers are in vector forms. Assume Xinput∈Rn×1 is the input vector and Xoutput∈Rm×1 is the output vector of the lth fully connected layer fl(⋅). The transformation in this fully connected layer is expressed as 7Xoutput=fl(Xinput,θl)=fσ-l(WDNNXinput+BDNN), where fσ-l(⋅) is a non-linear active function, WDNN∈Rm×n is the weight matrix, and BDNN∈Rm×1 is the bias vector.

In a convolutional layer, both the input and output are in matrix forms. A convolutional layer transfers information through sparse connections by several convolution kernels, essentially small matrices. The mathematical formula of a convolutional layer is as follows (Wang et al., 2019; Jardani et al., 2022): 8hu,vq(xu,v)=fσ-l∑i=1ki′∑j=1kj′wi,jqxu+i,v+j+b, where xu,v is the pixel value at position (u,v) of the input matrix and hu,vq(xu,v) is the output feature calculated by employing the qth (q=1,…,Nout) convolutional kernel filter wq∈Rki′×kj′. In a convolutional layer with Nout filters, the output matrix contains Nout feature layers. The output size (Sout) of each convolutional layer is determined by the input size (Sin) and the hyperparameters (i.e., zero padding p, kernel size k′, and stride s). A pooling layer is often used after a convolutional layer to remove redundant information from the extracted features and to improve the efficiency of model training (Chen et al., 2021).

The residual block is a fundamental component of residual networks (ResNets), designed primarily to mitigate the vanishing and exploding gradient problems commonly encountered during DNN training. A residual block learns a residual mapping, defined as 9R(Xinput,θR)=H(Xinput)-T(Xinput), where θR represents the trainable parameters of a residual block, R(⋅) is the residual function, H(⋅) denotes the target mapping that the residual block aims to approximate, and T(⋅) is chosen as an identity transformation (i.e., T(Xinput)=Xinput) or another suitable transformation depending on the network architecture. The output of the residual block is computed as 10Xoutput=fσ-R(R(Xinput,θR)+T(Xinput)), where fσ-R(⋅) is the activation function of the rectified linear unit (ReLU). Such a design ensures that the output of the residual block at least approximates the input, effectively addressing the vanishing gradient problem. When stacking multiple residual blocks, the relationship between the Lth residual block in a deeper layer and the lth residual block is expressed as follows (He et al., 2016): 11Xoutput(L)=Xinput(l)+∑i=lL-1R(Xoutput(i),θR(i)), where Xinput(i) and θR(i) denote the input data and trainable parameters of the ith residual block, respectively, and Xoutput(L) represents the output from the Lth residual block. According to the chain rule in derivatives, the gradient of the loss function JRes with respect to Xinput(l) can be given by 12∂JRes∂Xinput(l)=∂JRes∂Xoutput(L)×1+∂∂Xinput(l)∑i=lL-1R(Xoutput(i),θR(i)). This formulation highlights two key properties of the residual network. First, the gradient does not vanish during network training processes because the term ∂∂Xinput(l)∑i=lL-1F(Xinput(i),θR(i)) is never equal to -1. Second, the gradient of the deepest residual block ∂JRes/∂Xoutput(L) can directly affect all preceding layers, ensuring the effective transmission of gradients throughout the network (Chang et al., 2022).

Based on the three unique network layer structures described above, the FC-DNN, LeNet, and ResNet models are constructed. The FC-DNN of this study is constructed using fully connected layers, and each hidden layer consists of 512 neurons. The activation function for the output layer is the Sigmoid function, which constrains outputs within the range of 0 to 1. Note that other activation functions whose outputs ranges include [0,1] as a subset, such as the hyperbolic tangent (-1 to 1) and ReLU (0 to +∞), could also be adopted. However, we specifically selected the Sigmoid function to strictly constrain initial model outputs within the target range (0 to 1), thereby reducing the risk of occasional extreme or anomalous predictions, particularly in the early stages of training. For hidden layers, the Swish activation function is adopted due to its smooth form with non-monotonic and continuously differentiable properties, which helps to improve the DNN training procedures (Elfwing et al., 2018). The performance of the FC-DNN is sensitive to the number of hidden layers, whose optimal value is determined based on specific case studies presented in the application section. For the LeNet and ResNet models, when dealing with low-dimensional scenarios, an initial processing module consisting of a fully connected layer followed by a reshaping operation is added to convert the input vector into a fixed-size matrix. In contrast, for high-dimensional parameter scenarios, the discrete grid matrix of the parameter field is directly input into the CNN architecture (see Fig. 1b). Specifically, LeNet consists of two convolutional blocks and two fully connected layers. Each convolutional block consists of a convolutional layer followed by a max-pooling layer. The fully connected layers have 1024 and 512 neurons, respectively. ResNet consists of four stages, and two different Res blocks are adopted. The first stage includes two residual units without down-sampling, while the remaining three stages each have one residual unit with down-sampling and one residual unit without down-sampling. Activation functions in all layers are rectified linear units (ReLUs), except for the output layer, where Sigmoid activation is used. Detailed architecture information for LeNet and ResNet is provided in Figs. S1 and S2 in the Supplement, respectively.

The surrogate models are trained by minimizing the difference between the predicted outputs y^i=FDNN(mi,θDNN) and the corresponding numerical model outputs yi in the training datasets (i=1,…,Nsamples). Following prior studies (Mo et al., 2019, 2020; Chen et al., 2021), the L1 norm-based loss function is adopted and formulated as 13θDNN∗= argmin1Nsamples∑i=1Nsamples|FDNN(mi,θDNN)-yi|+wd2θDNNTθDNN, where wd is the weight decay to avoid overfitting, referred to as the regularization coefficient. It should be noted that the L2 norm can also be employed as a loss function in constructing surrogate model tasks. Due to its squared-error formulation, the L2 norm provides smoother gradients and more stable parameter updates near convergence compared to the L1 norm; however, this formulation also makes it more sensitive to extreme outliers. When the sampled parameters sparsely cover the parameter space, adopting the L1 norm loss function can improve the robustness of surrogate model predictions. This study implemented the DNN models using PyTorch (https://pytorch.org/, last access: 10 September 2024). The neural network weights were initialized using the default initialization method of PyTorch and optimized using the stochastic gradient descent method via the Adam algorithm. Specifically, the hyperparameter of weight decay can be set directly in the Adam optimizer without including it explicitly in the loss function.

When conducting DNN training, the hyperparameter selection primarily influences the update process of trainable parameters. Besides the weight decay mentioned above, the learning rate and the number of epochs are two other crucial hyperparameters directly affecting the training stability and convergence speed. A higher learning rate accelerates initial convergence but may lead to oscillations near the optimal solution, whereas a lower learning rate tends to improve final accuracy but requires more epochs to achieve convergence. In this study, we first set a relatively high number of epochs to ensure that the trainable parameters are adequately updated. Subsequently, appropriate learning rates and weight decay values for different scenarios are determined through a trial-and-error approach.

Let YG(s)∼N(μG(s)C(⋅,⋅)) represent a Gaussian random field, where μG denotes the mean of the random field and C(⋅,⋅) represents the exponential covariance function between two arbitrary spatial points s=(sx,sy) and s′=(sx′,sy′). The covariance function for these two spatial locations is given by 14C(s,s′)=σG2exp⁡-sx-sx′λx2+sy-sy′λy2, where σG2 is the variance and λx and λy are the correlation lengths along the x and y directions, respectively. Since the covariance matrix is symmetric and positive definite, the exponential covariance function in Eq. (14) can be decomposed into an eigenvalue-eigenfunction representation. By solving the second-kind Fredholm integral equation and performing eigenvalue decomposition, the Gaussian random field can be expressed through the Karhunen–Loève expansion (KLE) as follows: 15YG(s)=μG(s)+∑i=1∞ziλiϕi(s), where zi represents a random variable following a Gaussian distribution of zi∼N(0,1), also known as a KL term; and ϕi(s) and λi denote the eigenfunction and eigenvalue, respectively. For discretized numerical models, the index i takes values from 1 to n, representing the number of discrete grid points (e.g., in Eq. 15, ∞ is replaced by n). Dimensionality reduction using KLE is achieved through a truncated expansion (Loève, 1955; Zhang and Lu, 2004; Mariethoz and Caers, 2014).

The octave convolutional adversarial autoencoder (OCAAE) is a generative machine learning approach that combines the variational autoencoder (VAE) with adversarial learning, leveraging octave convolution neural networks (Zhan et al., 2021). It consists of three main components: an encoder, a decoder, and a discriminator. The encoder maps a high-dimensional parameter field mI to a low-dimensional latent vector zi. The distribution of the latent vectors {z1,…,zN}, obtained by mapping the N prior model parameter samples {m1,…,mN}, is denoted as z∼q(z). Specifically, the encoder outputs two low-dimensional vectors: the mean vector μz and the log-variance vector ln⁡(σz2) of the latent vector z. Then, a vector z′ is randomly drawn from a standard normal distribution N(0,I), and the latent vector is produced as z=μz+σz×z′. The decoder reconstructs the high-dimensional parameter field m̃ by taking the latent vector z as input. The discriminator enforces adversarial training, ensuring that the encoded latent vector distribution z∼q(z) approximates a prior Gaussian distribution z∼p(z). It receives input from the latent vectors generated by the encoder z∼q(z) or from the prior distribution z∼p(z), and it discriminates which distribution the input latent vector originates from.

This adversarial framework enhances the generative capability and ensures smooth transitions between different field realizations. In the adversarial autoencoder method, the encoder G(⋅) (which also acts as the generator of the adversarial network), decoder, and discriminator D(⋅) are trained jointly in two phases during each iteration: the reconstruction phase and the regularization phase.

In the reconstruction phase, the encoder and decoder are updated using the following loss function: 16LED=1N∑i=1N‖mi-m̃i‖1-wadv1N∑i=1Nlog⁡{D[G(mi)]}, where wadv is a weight balancing the reconstruction and adversarial losses (set to 0.01 in this study), m̃i is the reconstructed sample of mi, and N is the number of training samples.

In the regularization phase, the discriminator is trained to distinguish real latent vectors from the prior distribution based on the following loss function: 17LD=-1N∑i=1N{log⁡[D(zi)]+log⁡[1-D][G(mi)]}. This loss function helps the discriminator to distinguish between the latent vector zi (from the true distribution p(z)) and the fake latent vector produced by the encoder G(mi).

The constraint loss functions in the adversarial autoencoder framework ensure that the reconstructed high-dimensional parameter field m̃ closely matches the original field m, while also making sure that the distribution of the low-dimensional latent vector z approximates a predefined standard normal distribution p(z). After finishing the training process, it is possible to sample from the low-dimensional space of p(z) and to use the decoder to generate corresponding high-dimensional parameter fields. Then, the high-dimensional parameter field can be reconstructed by indirectly estimating the low-dimensional latent vectors (Makhzani et al., 2015; Mo et al., 2020).

The four metaheuristic algorithms used in this paper essentially update model parameters through distinct heuristic stochastic search strategies. Specifically, particle swarm optimization (PSO) updates the model parameters m based on the personal best position of the particles and the global best position of the swarm (Eberhart and Kennedy, 1995). Genetic algorithm (GA) encodes the initial model parameter samples using binary encoding, then iteratively updates them through crossover (combining portions of encoded solutions to generate new candidate solutions), mutation (randomly altering encoded information to introduce diversity), and selection (choosing candidate solutions based on objective function evaluations) (Holland John, 1975). Differential evolution (DE) initializes a population of real-valued parameter vectors and iteratively updates them through differential mutation (generating trial solutions based on vector differences among population members), crossover (probabilistically combining components from original and mutated vectors), and greedy selection (retaining solutions with better objective function values) (Storn and Price, 1997; Tran et al., 2022). Simulated annealing (SA) starts from a random initial solution and iteratively explores neighboring solutions, accepting them probabilistically based on the Metropolis criterion, while gradually decreasing the temperature parameter until convergence (Metropolis et al., 1953; Kirkpatrick et al., 1983).

A common characteristic of all the methods described above is that each iterative update of model parameters requires multiple evaluations of the objective function, and sufficient iterations are necessary to balance local exploitation and global exploration. Detailed implementation procedures and theoretical foundations of these methods are provided in the supplementary materials. The metaheuristic algorithms used in this study were implemented using the open-source Python package scikit-opt (https://scikit-opt.github.io/, last access: 10 September 2024).

The TNNA algorithm aims to obtain a reverse network FReverse(⋅) that maps the observation vector to model parameters, as shown in Eq. (18). 18m=FReverse(yobs,θReverse), where θReverse is the trainable parameters of FReverse. Since m also serves as the input to the established surrogate model FForward(⋅), by substituting the parameter m in the inversion objective function of Eq. (2) with the expression from Eq. (18), we obtain the objective function constraint for θReverse (i.e., the loss function for training FReverse): 19θReverse∗=argmin∑i=1Nobs1σi[yobs[i]-FForward×(FReverse(yobs,θReverse))[i]]2. After obtaining the optimal trainable parameters θReverse∗ through backpropagation-based stochastic gradient descent within the PyTorch framework, the final inversion results for the model parameters can be computed by m∗=FReverse(yobs,θReverse∗). The required training data here are the normalized observation data. Specifically, the reverse network for this study is designed using an FC-DNN with three hidden layers, each containing 512 neurons.

During the reverse network training processes, each iteration of updating the trainable parameters θReverse involves two main steps: first, the observation vector yobs is input into the reverse network FReverse to obtain the parameter prediction vector m̃. Next, the predicted parameter m̃ is input into the forward network FForward to generate corresponding forward prediction results. Subsequently, the trainable parameters θReverse of the reverse network are updated through standard DNN model training based on the error feedback from the loss function in Eq. (19). This process demonstrates that FReverse and FForward are integrated through a tandem connection, which is why this method is named TNNA. Upon completing the training of FReverse, the final optimal parameters are predicted by inputting observation data into FReverse. Further details on TNNA can be found in Chen et al. (2021).

In the above process, each backpropagation step involves only a single forward calculation of the loss function. After establishing the computational graph, gradients of the trainable parameters θReverse are computed through backpropagation combined with automatic differentiation. These gradients are then used to update the trainable parameters θReverse. Thus, only one forward simulation is executed during each epoch of the reverse network FReverse training procedure. This presents a marked computational advantage of TNNA compared to the four selected metaheuristic algorithms, which require numerous forward simulations for parameter updates at each iteration.

For the low-dimensional parameter scenario, the performance of optimization algorithms is thoroughly evaluated across 100 parameter scenarios using the Monte Carlo strategy. The observation data for these scenarios are derived from the testing dataset after adding multiplicative Gaussian random noise ϵ∼N(1,0.012). The population sizes of GA, DE, and PSO, along with the chain length in SA, are set in four distinct scenarios: 20, 40, 60, and 80. (These population size or chain length values are represented as NPC in subsequent discussions.) These settings determine the number of forward modeling calls required for each iteration, significantly influencing the convergence rate and computational efficiency of optimization procedures. Maximum iterations for these four metaheuristic algorithms are set to 200. The learning rate, epoch number, and weight decay for the TNNA algorithm are set to (6×10-5), 1000, and 1×10-6, respectively.

The performance of the five optimization algorithms is evaluated according to three aspects: average convergence efficiency and accuracy in inversion procedures, predictive accuracy of calibration models for hydraulic heads and solute concentrations, and statistical analysis of the estimated errors for each model parameter. Figure 7 presents the logarithmic average convergence curves (i.e., log⁡10 of the average objective value as a function of the inversion iterations) of four metaheuristic algorithms and the TNNA algorithm throughout 100 parameter scenarios. Specifically, panels (a)–(d) represent the NPC values for metaheuristic algorithms set at 20, 40, 60, and 80, respectively. These figures clearly illustrate the average convergence speed and accuracy of five optimization algorithms. Figure 8 displays the comparison between simulated and observed values across all 100 parameter scenarios for both calibration and spatial predictive evaluation. Panels (a) and (b) illustrate the comparative prediction fit at the 25 observation locations used for model calibration, whereas panels (c) and (d) display the comparative prediction fit at the 24 independent observation locations. In this figure, distinct symbols are used to represent the five optimization algorithms. It should be noted that the NPC values for the four metaheuristic algorithms are uniformly set to 80 during this comparison. Figure 9 illustrates the probability density curves of the estimation errors for nine model parameters across 100 parameter scenarios, with different colors representing the five optimization algorithms.

The results in Fig. 7 demonstrate that the TNNA algorithm achieves the best convergence accuracy, with its convergence logarithmic objective function value (approximately -4.4) being smaller than those of the other four metaheuristic algorithms across these NPC settings. The influence of NPC on the convergence speeds of these four metaheuristic algorithms is not significant, exhibiting a distinct transition from rapid to slower convergence around the 75th iteration. As NPC increased from 20 to 80, each metaheuristic algorithm showed distinct improvements in the accuracy of the final objective function. The DE algorithm showed the least improvement in final convergence accuracy as the NPC value increased from 20 to 80, with the logarithmic value of its objective function dropping from just above -4.0 to slightly below -4.0. The SA algorithm also showed limited improvement, with its logarithmic average convergence value increasing from around -4.1 at NPC=20 to slightly below -4.3 at NPC=80, close to that of the TNNA algorithm. Among the four metaheuristic algorithms, SA exhibited the highest average convergence accuracy. Contrary to the SA and DE algorithms, the PSO and GA algorithms significantly enhanced average convergence accuracy as NPC increased. Specifically, as NPC increased from 20 to 80, the logarithmic convergence values of PSO and GA decreased by more than 0.5. While increasing NPC values may help metaheuristic algorithms to reduce the gap in average convergence accuracy compared to the TNNA algorithm, larger NPC settings also require additional computational burdens. The above results indicate that the TNNA algorithm has a significant efficiency advantage over the four metaheuristic algorithms in parameter optimization. For instance, when conducting the optimization procedure based on scikit-opt, the DE algorithm requires 32 000 forward model realizations (80×2×200) when NPC is set to 80, while the other three metaheuristic algorithms (PSO, GA, and SA) each require 16 000 realizations (80×200). In significant contrast, the TNNA algorithm requires only one forward model realization per iteration, resulting in 200 realizations. These comparisons illustrated that the TNNA method is more effective than the other four metaheuristic algorithms in achieving robust convergence results. It is worth noting that the five optimization algorithms rely on stochastic processes for parameter updates. Therefore, the objective function values are not guaranteed to decrease monotonically with each iteration. According to Fig. 7, the DE algorithm exhibits more noticeable fluctuations compared to other algorithms. Nevertheless, these fluctuations remain within a reasonable range. For example, at NPC=80, the objective function values after 150 iterations range between 9.05×10-5 and 1.32×10-4 (corresponding to the logarithmic values between -4.04 and -3.88 in Fig. 7d). Fluctuations between consecutive iterations typically remain within 1×10-5 (mostly around 3×10-6), which is considered reasonable for optimization algorithms.

The results presented in Fig. 8 indicate that, among the five optimization algorithms, the TNNA algorithm achieves the smallest RMSE and R2 values closest to 1.0 for both hydraulic heads and solute concentration during model calibration and spatial predictive evaluation. Furthermore, the distribution of comparison points demonstrates that the modeling results obtained from both calibration and independent prediction using the TNNA algorithm match the observed values more accurately than those of the other four metaheuristic algorithms, particularly for solute concentrations. Among the four metaheuristic algorithms, SA and DE outperform GA and PSO regarding RMSE and R2 values. During model calibration and predictive evaluation, PSO exhibits the worst predictive accuracy, recording the highest RMSE and R2 values for both hydraulic heads and solute concentrations. It is noteworthy that the RMSE and R2 values for SA during hydraulic head calibration are 0.0085 and 0.9992, respectively, while those for DE during solute concentration calibration are 0.0112 and 0.9969. These values are almost equal to those of the TNNA algorithm. The robustness of an inversion algorithm is determined by its accuracy in both calibration and predictive evaluation for hydraulic heads and solute concentrations. However, DE and SA demonstrate appropriate calibration accuracy for only one of the two simulation components. Overall, the TNNA algorithm provides more robust model calibration and predictive evaluation results than the other four metaheuristic algorithms.

Figure 9 indicates that the estimated error distributions for the nine model parameters derived from the TNNA algorithm are more concentrated than those obtained from the four metaheuristic algorithms. The mean estimated error values for the nine numerical model parameters using the TNNA algorithm are also the lowest. These results highlight the high accuracy and reliability of the TNNA inversion algorithm. Among the four metaheuristic algorithms, DE and SA outperform GA and PSO. This is because the probability density curves of estimation errors for the nine parameters using DE and SA are more concentrated around zero, with mean values lower than those of GA and PSO. The DE algorithm shows a more concentrated distribution of around zero for the overall estimation errors of parameters k1 to k8. In contrast, the SA reveals reduced estimation errors for the C0 parameter in most cases, ranking just behind the TNNA algorithm. GA outperforms PSO in estimation accuracy for seven of the nine model parameters, with PSO matching its probability density curves to that of GA only for parameters k2 and k4. As a whole, the statistical results of the estimated model parameter errors illustrate that the machine-learning-based TNNA algorithm exhibits enhanced inversion performance compared to the four metaheuristic optimization algorithms. However, the findings also reveal that none of the five algorithms consistently offers completely reliable inversion solutions across all scenarios. For example, the TNNA algorithm, despite its generally better performance, demonstrates estimation errors as high as 0.4 for parameters k4 and k6 in some scenarios. Such results are likely because the provided observational data cannot ensure equifinality in some scenarios. In these cases, it is essential to introduce additional regularization constraints to attenuate the equifinality (Wang and Chen, 2013; Arsenault and Brissette, 2014). These findings emphasize the importance of employing the Monte Carlo method in comparative studies of inversion algorithms to ensure comprehensive evaluations and to avoid misleading conclusions.

The above comparison results indicate that the machine-learning-based TNNA algorithm outperforms the other four metaheuristic algorithms in both inversion accuracy and computational efficiency. The primary advantage of the TNNA algorithm over the four metaheuristic algorithms is its well-defined updating direction of model parameters, guided by the loss function, which serves as the objective function for inverse modeling. Research on machine learning applications indicates that DNNs can approximate continuous functions by adjusting weights and biases (LeCun et al., 2015; Goodfellow et al., 2016). The TNNA algorithm leverages this capability by transforming the model parameter inversion issue into the training of a reverse network to achieve reverse mappings. By establishing a loss function based on inversion constraints from the Bayesian theorem, the TNNA algorithm ensures that training the reverse network brings each parameter update closer to the optimal solution during each epoch, thereby improving accuracy and convergence speed. In contrast, the four metaheuristic algorithms require numerous forward simulations for each parameter update. The optimization direction for model parameters is determined by evaluating the objective function. This process is governed by the exploration and exploitation strategies inherent in metaheuristic algorithms. However, these approaches introduce randomness in the direction of model parameter updates, making it challenging to ensure that updates move towards the direction of fastest convergence under specific hyperparameter settings. This also explains why the TNNA algorithm can update model parameters more efficiently and achieve higher convergence accuracy despite requiring only one forward realization in each training epoch.

For estimating the permeability field under five designed observational scenarios, the iteration number for the four metaheuristic algorithms was set at 200, with NPC values of 100, 500, and 1000. The learning rate and weight decay for training reverse networks within the TNNA framework were set to 1×10-3 and 1×10-4, respectively.

Figures 10 and 11 illustrate the log-permeability field estimation results and error distributions for the four metaheuristic algorithms and the TNNA algorithm under the most densely observed scenario (i.e., scenario 5). The corresponding results for scenarios 1–4 are presented in Figs. S7–S14 in the Supplement. Figure 12 compares the RMSE values for the log-permeability fields estimated by the four metaheuristic algorithms and the TNNA algorithm across all five scenarios. These detailed RMSE values can be found in Table 2 (scenario 5) and Table S4 in the Supplement (scenarios 1–4). For scenario 5, the accuracy of permeability estimations by each metaheuristic algorithm improves as the NPC value increases (see Fig. 10 and Table 2). Notably, the GA achieves the best results with an NPC of 1000, recording an RMSE of 0.1057. The DE and SA algorithms yield their most accurate permeability estimations with RMSE values of 0.1597 (NPC=100) and 0.1549 (NPC=1000), respectively. The PSO method is the least effective, achieving an RMSE of 0.3334 at NPC=1000. As shown in Fig. 11 and Table 2, the TNNA algorithm provides inversion results with an RMSE of 0.1063 after training the reverse network for 200 epochs. This suggests that the TNNA algorithm can estimate high-dimensional permeability fields with accuracy comparable to that of the GA method (NPC=1000), with significantly fewer forward model realizations (200 compared to 200 000), reducing the computational burden by 99.9 % and improving inversion efficiency by a factor of 1000. Increasing the training epochs of the reverse network to 1000 further reduces the RMSE of the TNNA method to 0.0595, demonstrating its advantages over the four metaheuristic algorithms in this scenario. Across all scenarios, the accuracy of the estimated permeability fields correlates positively with the density of observation wells, and estimation errors are generally higher in areas not covered by monitoring wells (see Figs. S7–S14). Figure 12 further demonstrates that the RMSE values for permeability estimation using the TNNA algorithm are consistently lower than those of the four metaheuristic algorithms across scenarios 1–4, indicating that the TNNA algorithm exhibits greater robustness compared to the metaheuristic algorithms in all five scenarios.

To evaluate the predictive performance of the numerical model calibrated by various inversion methods, simulations of hydraulic heads and solute concentrations were conducted over 60 d, starting on the second day with recordings every 2 d, using the permeability fields with the lowest RMSE values identified by each inversion method. Observation data from the second day to the 40th day were used for model calibration, while additional data from the 42nd to the 60th day were employed to evaluate the future predictions of the calibrated numerical models. The RMSE values for the calibrated hydraulic heads and time series solute concentrations are presented in Table 3 and Fig. 13. Figure 14 displays the spatial distribution of the calibrated numerical simulation results and errors for hydraulic heads and solute concentration simulation results at three specific times (t=4th, 20th, and 52nd days). Results for the entire 60 d period are presented in Figs. S15–S44 in the Supplement. Note that in Fig. 14, y^H and y^C represent the simulated spatial distributions of hydraulic heads and solute concentrations based on the estimated permeability fields through inverse modeling, while yH and yC represent the spatial distributions simulated using the true permeability field.

According to Fig. 14a, the calibrated simulation errors for hydraulic heads did not exceed 0.02 m for the TNNA method and three of the four considered metaheuristic algorithms, except the PSO method, which exhibited hydraulic head errors larger than 0.06 m in certain areas. Among the four metaheuristic algorithms, the GA method achieved the lowest RMSE in hydraulic head simulations, with a value of 7.4837×10-4. For solute concentrations, the GA algorithm consistently has the highest prediction accuracy among the metaheuristic algorithms, with RMSE values generally around 0.005 (Fig. 13). The TNNA algorithm achieved a similar level of accuracy to GA in the calibrated numerical model predictions. Specifically, during the initial 10 d and from the 41st day to the 60th day, the TNNA algorithm showed slightly higher prediction accuracy than the GA-calibrated model. However, during the intermediate period from the 10th day to the 40th day, the GA-calibrated model had a slight advantage over the TNNA algorithm. The normalized absolute errors in the solute transport simulation results obtained using the TNNA algorithm remained consistently below 0.02 throughout the simulation period (Fig. 14b and c). These results indicate that in high-dimensional settings, the TNNA algorithm provides inversion outcomes that enable the calibrated model to deliver simulation results comparable to those of the best-performing metaheuristic algorithm. Overall, the TNNA method also demonstrates advantages over the four metaheuristic optimization algorithms in the designed high-dimensional scenarios, excelling in both inversion efficiency and accuracy.

In this scenario, the iteration number for the four metaheuristic algorithms was set at 200, with NPC values of 1000. For the TNNA method, the reverse network is trained for 1000 epochs. Thus, each metaheuristic algorithm had 100 times more forward model evaluations than the TNNA algorithm. Figures 14 and 15 show the permeability fields estimated by the five optimization algorithms and their error distributions compared to the true field (i.e., the error fields). Figures 16a and 17a present the comparison between calibrated simulations and hydraulic head observations, as well as solute concentration observations. Figures 16b and 17b compare the solute concentration simulations for the 26th, 28th, and 30th years based on the estimated parameter field and the designed true field.

According to Figs. 15 and 16, the binary channel fields reconstructed by each inversion algorithm are highly consistent with their corresponding true fields, with the estimated errors primarily concentrated at the interfaces between high-permeability channels and low-permeability regions. It is found that increasing the observation noise level from 1 % to 10 % does not lead to a noticeable increase in the number of grid cells exhibiting differences between the estimated parameter fields and the true field. One potential reason for this is that the least squares objective function used in the inversion framework of this study is based on the assumption that the observation noise follows a zero-mean Gaussian distribution. With adequate regularization constraints, such as the dense monitoring network design used in this study, the model responses corresponding to the optimal parameter estimates obtained through global optimization algorithms statistically converge to the mean of the observed data. It can also be evaluated by the calibration simulations. Specifically, the pairwise scatter plots in Figs. 17a and 18a indicate that the calibrated simulation results from different methods are closely distributed around the reference diagonal. This suggests that even with increased observational noise, the inversion-derived calibration results do not exhibit noticeable bias. Furthermore, the predictions based on inversion results remain highly consistent with those of the true permeability field (Figs. 17b and 18b). The RMSEAll and RAll2 values for the predictions beyond the observational period range from 0.018 to 0.044 and from 0.962 to 0.994, respectively. This indicates that even under relatively high Gaussian noise conditions, the non-linear inversion framework used in this study can reliably reconstruct the non-Gaussian permeability field, ensuring high predictive accuracy. Nevertheless, it is important to note that while the inversion accuracy at a 10 % noise level remains comparable to that in the 1 % noise scenario, increasing the observational noise inevitably raises the convergence value of the least squares loss function. This trend is evident from the RMSE values in Figs. 17a and 18a. Moreover, since the observational noise here is assumed to follow a Gaussian distribution, real-world scenarios with more complex noise characteristics may further exacerbate equifinality in the inversion results. In such cases, incorporating additional system information such as regularization constraints is essential to enhance the robustness of the objective function and to mitigate ill-posedness.

Compared to the four metaheuristic algorithms, TNNA demonstrates advantages in computational efficiency and accuracy for non-Gaussian random field inversion. In the low-noise scenario, TNNA achieves an inversion convergence accuracy with an RMSEAll of 0.021 and an RAll2 of 0.996 (Fig. 17a). In contrast, the two best-performing metaheuristic methods, GA and SA, yield RMSEAll values of 0.027 and 0.029, with RAll2 values of 0.994 and 0.993, respectively (Fig. 17a). Moreover, TNNA achieves the highest fitting accuracy for predictive results among the five optimization algorithms, with an RMSE of 0.018 and an R2 of 0.994 (Fig. 17b). Even in high-noise scenarios, TNNA continues to exhibit an advantage over the four metaheuristic algorithms in both inversion convergence accuracy (Fig. 18a) and predictive accuracy (Fig. 18b). Additionally, considering the number of forward simulation calls required by each inversion algorithm, TNNA proves to be a more efficient approach in this case study.