PDE-Constrained Machine Learning with Gaussian Processes towards Digital Twins
Weihao Yan is a PhD student in the Department of Mathematics of Imaging & AI. Promtotors are prof.dr. C. Brune from the Faculty of Electrical Engineering, Mathematics and Computer Science (UT) and Dr. M. Guo from Lund University.
The development of trustworthy digital twins for complex physical systems requires predictive models that are fast, accurate, and capable of quantifying their own uncertainty. This work addresses a fundamental limitation: two widely used machine learning methods tend to exhibit complementary shortcomings. Deep neural networks are excellent at learning from high-dimensional data but generally do not provide reliable confidence estimates. In contrast, Gaussian processes offer a robust framework for uncertainty quantification but face severe computational limits in high dimensions. We develop a unified, PDE-constrained framework that bridges the two approaches and advances scientific discovery.
We first construct a robust model architecture as the basis for the unified framework. Through a careful comparison of existing hybrid models, we demonstrated that deep kernel learning (DKL) offers the most effective and practical approach. This method uses a neural network to learn an adaptive, problem-specific measure of similarity for a GP. This design avoids key limitations of existing methods and establishes a stable foundation for the subsequent developments presented in this thesis.
With this foundation, we first addressed the challenge of solving high-dimensional PDEs. We introduced a novel PDE-constrained DKL (PDE-DKL) model where the neural network learns to represent the complex, high-dimensional problem in a much simpler, low-dimensional space. This strategy effectively mitigates the curse of dimensionality, allowing the model to produce physically consistent solutions and reliable uncertainty estimates even with very limited data.
To ensure our framework would be robust in real-world applications, we then tackled the practical issue of uncertain data. We developed a principled Bayesian method to handle situations where measurement locations are not known with certain precision. By learning the true locations from the data itself, our model becomes more resilient to noise and provides a more reliable assessment of its predictive confidence.
Finally, we applied our framework to solve inverse problems involving the estimation of spatially varying physical parameters in PDEs, given only sparse observational data. Performing a full Bayesian inference on all model and PDE parameters simultaneously is computationally intractable. To solve this, we designed an intelligent two-stage strategy. The first stage is a physics-informed pretraining step that efficiently optimizes the complex neural network part of our model while also finding excellent initial estimates for the unknown PDE parameters. In the second stage, with the neural network's parameters now fixed, we perform a much more focused and computationally tractable Bayesian inference. This allows us to efficiently explore the posterior distribution of only the PDE parameters and the remaining kernel hyperparameters, leading to accurate parameter estimates and their uncertainties.
In summary, this thesis presents a complete and coherent journey, progressing from fundamental architectural principles to advanced applications in forward and inverse modelling. The resulting framework provides a significant and practical step toward scientific machine learning tools. These tools not only predict with high accuracy but also understand their own limitations, offering a reliable and trustworthy foundation for accelerating discovery and innovation across science and engineering.
