Fourier warm start for physics-informed neural networks
Physics-informed neural networks (PINNs) have shown applicability in a wide range of engineering domains. However, there remain some challenges in their use, namely, PINNs are notoriously difficult to train and prone to failure when dealing with complex tasks with multi-frequency patterns or steep g...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/180175 |
| _version_ | 1826123573469642752 |
|---|---|
| author | Jin, Ge Wong, Jian Cheng Gupta, Abhishek Li, Shipeng Ong, Yew-Soon |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Jin, Ge Wong, Jian Cheng Gupta, Abhishek Li, Shipeng Ong, Yew-Soon |
| author_sort | Jin, Ge |
| collection | NTU |
| description | Physics-informed neural networks (PINNs) have shown applicability in a wide range of engineering domains. However, there remain some challenges in their use, namely, PINNs are notoriously difficult to train and prone to failure when dealing with complex tasks with multi-frequency patterns or steep gradients in the outputs. In this work, we leverage the Neural Tangent Kernel (NTK) theory and introduce the Fourier Warm Start (FWS) algorithm to balance the convergence rate of neural networks at different frequencies, thereby mitigating spectral bias and improving overall model performance. We then propose the Fourier Analysis Boosted Physics-Informed Neural Network (Fab-PINN), a novel integrated architecture based on the FWS algorithm. Finally, we present a series of challenging numerical examples with multi-frequency or sparse observations to validate the effectiveness of the proposed method. Compared to standard PINN, Fab-PINN exhibits a reduction of relative L2 errors in solving the heat transfer equation, the Klein–Gordon equation, and the transient Navier–Stokes equations from 9.9×10−1 to 4.4×10−3, 5.4×10−1 to 2.6×10−3, and 6.5×10−1 to 9.6×10−4, respectively. |
| first_indexed | 2024-10-01T06:06:34Z |
| format | Journal Article |
| id | ntu-10356/180175 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:06:34Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1801752024-09-23T04:22:51Z Fourier warm start for physics-informed neural networks Jin, Ge Wong, Jian Cheng Gupta, Abhishek Li, Shipeng Ong, Yew-Soon School of Computer Science and Engineering Agency for Science, Technology and Research, Singapore Computer and Information Science Fourier warm start Physics-informed neural networks Physics-informed neural networks (PINNs) have shown applicability in a wide range of engineering domains. However, there remain some challenges in their use, namely, PINNs are notoriously difficult to train and prone to failure when dealing with complex tasks with multi-frequency patterns or steep gradients in the outputs. In this work, we leverage the Neural Tangent Kernel (NTK) theory and introduce the Fourier Warm Start (FWS) algorithm to balance the convergence rate of neural networks at different frequencies, thereby mitigating spectral bias and improving overall model performance. We then propose the Fourier Analysis Boosted Physics-Informed Neural Network (Fab-PINN), a novel integrated architecture based on the FWS algorithm. Finally, we present a series of challenging numerical examples with multi-frequency or sparse observations to validate the effectiveness of the proposed method. Compared to standard PINN, Fab-PINN exhibits a reduction of relative L2 errors in solving the heat transfer equation, the Klein–Gordon equation, and the transient Navier–Stokes equations from 9.9×10−1 to 4.4×10−3, 5.4×10−1 to 2.6×10−3, and 6.5×10−1 to 9.6×10−4, respectively. The authors acknowledge the financial support received from the China Scholarship Council (202206030081). The work was supported in part by the Indian Science & Engineering Research Board Ramanujan Fellowship grant RJF/2022/000115. 2024-09-23T04:22:51Z 2024-09-23T04:22:51Z 2024 Journal Article Jin, G., Wong, J. C., Gupta, A., Li, S. & Ong, Y. (2024). Fourier warm start for physics-informed neural networks. Engineering Applications of Artificial Intelligence, 132, 107887-. https://dx.doi.org/10.1016/j.engappai.2024.107887 0952-1976 https://hdl.handle.net/10356/180175 10.1016/j.engappai.2024.107887 2-s2.0-85183455226 132 107887 en Engineering Applications of Artificial Intelligence © 2024 Published by Elsevier Ltd. All rights reserved. |
| spellingShingle | Computer and Information Science Fourier warm start Physics-informed neural networks Jin, Ge Wong, Jian Cheng Gupta, Abhishek Li, Shipeng Ong, Yew-Soon Fourier warm start for physics-informed neural networks |
| title | Fourier warm start for physics-informed neural networks |
| title_full | Fourier warm start for physics-informed neural networks |
| title_fullStr | Fourier warm start for physics-informed neural networks |
| title_full_unstemmed | Fourier warm start for physics-informed neural networks |
| title_short | Fourier warm start for physics-informed neural networks |
| title_sort | fourier warm start for physics informed neural networks |
| topic | Computer and Information Science Fourier warm start Physics-informed neural networks |
| url | https://hdl.handle.net/10356/180175 |
| work_keys_str_mv | AT jinge fourierwarmstartforphysicsinformedneuralnetworks AT wongjiancheng fourierwarmstartforphysicsinformedneuralnetworks AT guptaabhishek fourierwarmstartforphysicsinformedneuralnetworks AT lishipeng fourierwarmstartforphysicsinformedneuralnetworks AT ongyewsoon fourierwarmstartforphysicsinformedneuralnetworks |