Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials
When designing nano-structured metamaterials with an iterative optimization method, a fast deep learning solver is desirable to replace a time-consuming numerical solver, and the related issue of data shift is a subtle yet easily overlooked challenge. In this work, we explore the data shift challeng...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Nanomaterials |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-4991/13/20/2778 |
| _version_ | 1797572726927392768 |
|---|---|
| author | Zhenjia Zeng Lei Wang Yiran Wu Zhipeng Hu Julian Evans Xinhua Zhu Gaoao Ye Sailing He |
| author_facet | Zhenjia Zeng Lei Wang Yiran Wu Zhipeng Hu Julian Evans Xinhua Zhu Gaoao Ye Sailing He |
| author_sort | Zhenjia Zeng |
| collection | DOAJ |
| description | When designing nano-structured metamaterials with an iterative optimization method, a fast deep learning solver is desirable to replace a time-consuming numerical solver, and the related issue of data shift is a subtle yet easily overlooked challenge. In this work, we explore the data shift challenge in an AI-based electromagnetic solver and present innovative solutions. Using a one-dimensional grating coupler as a case study, we demonstrate the presence of data shift through the probability density method and principal component analysis, and show the degradation of neural network performance through experiments dealing with data affected by data shift. We propose three effective strategies to mitigate the effects of data shift: mixed training, adding multi-head attention, and a comprehensive approach that combines both. The experimental results validate the efficacy of these approaches in addressing data shift. Specifically, the combination of mixed training and multi-head attention significantly reduces the mean absolute error, by approximately 36%, when applied to data affected by data shift. Our work provides crucial insights and guidance for AI-based electromagnetic solvers in the optimal design of nano-structured metamaterials. |
| first_indexed | 2024-03-10T20:59:49Z |
| format | Article |
| id | doaj.art-9ebd92668c934e799db412073e66e249 |
| institution | Directory Open Access Journal |
| issn | 2079-4991 |
| language | English |
| last_indexed | 2024-03-10T20:59:49Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Nanomaterials |
| spelling | doaj.art-9ebd92668c934e799db412073e66e2492023-11-19T17:35:59ZengMDPI AGNanomaterials2079-49912023-10-011320277810.3390/nano13202778Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured MetamaterialsZhenjia Zeng0Lei Wang1Yiran Wu2Zhipeng Hu3Julian Evans4Xinhua Zhu5Gaoao Ye6Sailing He7National Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, ChinaNational Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, ChinaNational Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, ChinaNational Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, ChinaNational Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, ChinaShanghai Institute for Advanced Study, Zhejiang University, Shanghai 201203, ChinaTaizhou Research Institute, Zhejiang University, Taizhou 317700, ChinaNational Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, ChinaWhen designing nano-structured metamaterials with an iterative optimization method, a fast deep learning solver is desirable to replace a time-consuming numerical solver, and the related issue of data shift is a subtle yet easily overlooked challenge. In this work, we explore the data shift challenge in an AI-based electromagnetic solver and present innovative solutions. Using a one-dimensional grating coupler as a case study, we demonstrate the presence of data shift through the probability density method and principal component analysis, and show the degradation of neural network performance through experiments dealing with data affected by data shift. We propose three effective strategies to mitigate the effects of data shift: mixed training, adding multi-head attention, and a comprehensive approach that combines both. The experimental results validate the efficacy of these approaches in addressing data shift. Specifically, the combination of mixed training and multi-head attention significantly reduces the mean absolute error, by approximately 36%, when applied to data affected by data shift. Our work provides crucial insights and guidance for AI-based electromagnetic solvers in the optimal design of nano-structured metamaterials.https://www.mdpi.com/2079-4991/13/20/2778data shiftmixed trainingmulti-head attentionAI-based electromagnetic solversnano-structured metamaterials |
| spellingShingle | Zhenjia Zeng Lei Wang Yiran Wu Zhipeng Hu Julian Evans Xinhua Zhu Gaoao Ye Sailing He Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials Nanomaterials data shift mixed training multi-head attention AI-based electromagnetic solvers nano-structured metamaterials |
| title | Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials |
| title_full | Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials |
| title_fullStr | Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials |
| title_full_unstemmed | Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials |
| title_short | Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials |
| title_sort | utilizing mixed training and multi head attention to address data shift in ai based electromagnetic solvers for nano structured metamaterials |
| topic | data shift mixed training multi-head attention AI-based electromagnetic solvers nano-structured metamaterials |
| url | https://www.mdpi.com/2079-4991/13/20/2778 |
| work_keys_str_mv | AT zhenjiazeng utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT leiwang utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT yiranwu utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT zhipenghu utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT julianevans utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT xinhuazhu utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT gaoaoye utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials AT sailinghe utilizingmixedtrainingandmultiheadattentiontoaddressdatashiftinaibasedelectromagneticsolversfornanostructuredmetamaterials |