Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem
Abstract Physics-informed neural networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs) and are in principle capable of modeling a large variety of differential equations. PINNs are based on simple architectures, and...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-05-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-11058-2 |
| _version_ | 1818467801571000320 |
|---|---|
| author | Ruben Rodriguez-Torrado Pablo Ruiz Luis Cueto-Felgueroso Michael Cerny Green Tyler Friesen Sebastien Matringe Julian Togelius |
| author_facet | Ruben Rodriguez-Torrado Pablo Ruiz Luis Cueto-Felgueroso Michael Cerny Green Tyler Friesen Sebastien Matringe Julian Togelius |
| author_sort | Ruben Rodriguez-Torrado |
| collection | DOAJ |
| description | Abstract Physics-informed neural networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs) and are in principle capable of modeling a large variety of differential equations. PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called physics-informed attention-based neural networks (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside the convex hull of the training set. |
| first_indexed | 2024-04-13T21:04:58Z |
| format | Article |
| id | doaj.art-c2262f050ee747a29b322eb602566cbb |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-13T21:04:58Z |
| publishDate | 2022-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-c2262f050ee747a29b322eb602566cbb2022-12-22T02:30:00ZengNature PortfolioScientific Reports2045-23222022-05-0112111210.1038/s41598-022-11058-2Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problemRuben Rodriguez-Torrado0Pablo Ruiz1Luis Cueto-Felgueroso2Michael Cerny Green3Tyler Friesen4Sebastien Matringe5Julian Togelius6OriGen.AIOriGen.AIUniversidad Politécnica de MadridOriGen.AIOriGen.AIHess CorporationOriGen.AIAbstract Physics-informed neural networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs) and are in principle capable of modeling a large variety of differential equations. PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called physics-informed attention-based neural networks (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside the convex hull of the training set.https://doi.org/10.1038/s41598-022-11058-2 |
| spellingShingle | Ruben Rodriguez-Torrado Pablo Ruiz Luis Cueto-Felgueroso Michael Cerny Green Tyler Friesen Sebastien Matringe Julian Togelius Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem Scientific Reports |
| title | Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem |
| title_full | Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem |
| title_fullStr | Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem |
| title_full_unstemmed | Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem |
| title_short | Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley–Leverett problem |
| title_sort | physics informed attention based neural network for hyperbolic partial differential equations application to the buckley leverett problem |
| url | https://doi.org/10.1038/s41598-022-11058-2 |
| work_keys_str_mv | AT rubenrodrigueztorrado physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem AT pabloruiz physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem AT luiscuetofelgueroso physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem AT michaelcernygreen physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem AT tylerfriesen physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem AT sebastienmatringe physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem AT juliantogelius physicsinformedattentionbasedneuralnetworkforhyperbolicpartialdifferentialequationsapplicationtothebuckleyleverettproblem |