A probabilistic framework for prognostics with uncertainty quantification based on physics-guided bayesian neural networks

Abstract

Artificial neural networks are a family of algorithms whose principles are inspired by the behaviour of biological neurons in the human brain. They have been very successful in performing a wide variety of tasks, and are making a considerable impact on our daily lives. Moreover, many industries have been reaping the rewards from the use of these technologies for decades. But this is not the case in many areas of engineering, and more specifically, civil/structural engineering, where their application is mainly confined within the research domain. Even though civil engineering is a sector with tight margins where safety is the number one priority, and therefore, the benefits of a successful implementation could be significant. The reasons behind such little interest are diverse, from scarcity of quality data, to general distrust about their potential and applicability in the sector. Indeed, artificial neural networks are often considered as a black box system, given that they can approximate any function but without providing insights about its structure or form. Besides, they suffer from a series of drawbacks and their predictions are not always correct. Hence, quantifying the uncertainty in the outputs of the neural networks becomes of great importance. The state-of-the-art Bayesian neural networks, such as Variational Inference, Hamiltonian Monte Carlo, or Probabilistic Backpropagation, have largely contributed to cast light on this matter, but their method to quantify the uncertainty may be considered as rigid. This is mainly due to the use of parametric probability models for the likelihood function and/or the weights and bias, but also because of the limitations specific to the backpropagation algorithm. In this thesis, a novel training algorithm for Bayesian neural networks based on approximate Bayesian computation is proposed, hereafter called BNN by ABC-SS. The weights and bias parameters are trained probabilistically without backpropagation or gradient evaluation, thus issues such as local minima are avoided and the stability of the algorithm is improved. Also, predefined parametric probability models are not used for the weights and bias, but they can adopt any form depending on the training data. As a result, BNN by ABC-SS presents great flexibility to learn from the observed data, and more importantly, to quantify the uncertainty inherent in such data. The output of the Bayesian neural network trained with ABC-SS is a non-parametric probability density function that can be understood as the degree of belief of such output in light of the data available. As previously mentioned, lack of data is also an important limitation for artificial neural networks, as their training is entirely dependent on them. Furthermore, extrapolation is outside their capabilities, which means that the predictions made outside the domain of the training data are often random, and should not be trusted in most cases. This problem can be overcome, or at least mitigated, by introducing the knowledge extracted from physics-based models into the neural network architecture. While these hybrid algorithms are becoming more and more popular within the scientific community, they normally insert the physics in the loss function through some known boundary conditions, in the form of partial differential equations. The error is then backpropagated to adjust the weights, forcing them to comply with the given laws of physics. A different approach is followed in this thesis, where the physics are introduced independently in three parts of the neural network, namely the input neurons, the output neurons and the metric function (ρ in ABC-SS), resulting in three different variants which are then trained with ABC-SS. The need for data is reduced and the extrapolation capabilities of the overall model improved notably, especially when the physics are added to the output neurons like an extra bias parameter. Additionally, the use of ABC-SS as learning engine provides stability and a more realistic quantification of the uncertainty, yielding a more reliable algorithm. This is especially interesting in engineering, as it allows us to exploit both the valuable knowledge within the physics-based models and the flexibility of artificial neural networks to capture the nonlinear behaviour often found in real data. The aforementioned principles lead us to the last stage of this doctoral dissertation, when they are applied to prognostics, an engineering discipline that focuses on predicting how the damage and performance of a system will evolve through time. To that end, the capacity of handling sequential data is of great importance, and that is particularly where recurrent neural networks excel at. In the literature, these data-driven algorithms have also been combined with physics-based models and provided promising results, however, they are specially sensitive to gradient related problems, such as vanishing gradients. More complex architectures like Long-Short-Term-Memory have proven to mitigate this issue, but at the expense of increasing the number of parameters and activation functions. In this thesis, a physics-guided recurrent neural network trained with ABC-SS is proposed to make predictions about the future performance of an engineering system based on historical sequential data and physics-based knowledge. The probabilistic nature of the ABC-SS algorithm, along with its flexible quantification of the uncertainty, translates into a reliable algorithm that avoids the issues associated to the evaluation of the gradient and its propagation through time, thus long-term dependencies can be learnt without the need for more complex architectures. Moreover, the combination of physics-based knowledge and Bayesian regularization contributes to an improved extrapolation capacity of the proposed recurrent neural network, which is paramount in multi-step ahead forecasting. Several case studies are presented to evaluate the performance of the proposed algorithms in different engineering problems, from fatigue in composite materials to displacement and accelerations in concrete structures subjected to seismic loads. The key findings from those case studies are the realistic quantification of the uncertainty provided by ABCSS, high accuracy comparable to that of the state-of-the-art neural networks, stability thanks to the absence of gradient evaluation, and the ability to make precise predictions beyond the domain of the training data when combined with physics-based models. Regarding real-world applications, the proposed Bayesian neural networks can be envisaged becoming part of a wider PHM tool, helping to make informed decisions about future maintenance operations based on prognostics about the structural integrity of the system.