A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process
In Thin-Film Transistor Liquid-Crystal Display (TFT-LCD) manufacturing, conducting a machine learning based system with multiple data types has become actively desired to solve complicated problems. This paper proposes a multi-modal learning approach: <italic>TabVisionNet</italic>, which...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9733354/ |
| _version_ | 1818066672590782464 |
|---|---|
| author | Yi Liu Hsueh-Ping Lu Ching-Hao Lai |
| author_facet | Yi Liu Hsueh-Ping Lu Ching-Hao Lai |
| author_sort | Yi Liu |
| collection | DOAJ |
| description | In Thin-Film Transistor Liquid-Crystal Display (TFT-LCD) manufacturing, conducting a machine learning based system with multiple data types has become actively desired to solve complicated problems. This paper proposes a multi-modal learning approach: <italic>TabVisionNet</italic>, which is modeled by utilizing the information from both tabular data and image data. A novel attention mechanism called <italic>Sequential Decision Attention</italic> was integrated into the multi-modal modeling framework that improves the comprehension of the information from two modalities. This cross-modal attention mechanism can capture the complex relationship between modalities then gain better generalization and faster convergence in the training process. Conducting an experiment, the performance of our novel approach was significantly better than single-modal and other multi-modal learning approaches in our real case scenario. |
| first_indexed | 2024-12-10T15:11:31Z |
| format | Article |
| id | doaj.art-9655db87f2e14edcbc8f5fbd84c711b6 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T15:11:31Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-9655db87f2e14edcbc8f5fbd84c711b62022-12-22T01:43:56ZengIEEEIEEE Access2169-35362022-01-0110330263303610.1109/ACCESS.2022.31589529733354A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair ProcessYi Liu0https://orcid.org/0000-0002-0533-8233Hsueh-Ping Lu1https://orcid.org/0000-0003-0839-9223Ching-Hao Lai2https://orcid.org/0000-0001-9488-1598Department of Data Science Analysis, Division of Digital Technology, AU Optronics Corporation, Taichung, TaiwanDivision of Digital Technology, AU Optronics Corporation, Hsinchu, TaiwanDepartment of Data Science Analysis, Division of Digital Technology, AU Optronics Corporation, Taichung, TaiwanIn Thin-Film Transistor Liquid-Crystal Display (TFT-LCD) manufacturing, conducting a machine learning based system with multiple data types has become actively desired to solve complicated problems. This paper proposes a multi-modal learning approach: <italic>TabVisionNet</italic>, which is modeled by utilizing the information from both tabular data and image data. A novel attention mechanism called <italic>Sequential Decision Attention</italic> was integrated into the multi-modal modeling framework that improves the comprehension of the information from two modalities. This cross-modal attention mechanism can capture the complex relationship between modalities then gain better generalization and faster convergence in the training process. Conducting an experiment, the performance of our novel approach was significantly better than single-modal and other multi-modal learning approaches in our real case scenario.https://ieeexplore.ieee.org/document/9733354/Smart manufacturingTFT-LCDdeep learningmulti-modality machine learningtabular dataimage data and attention mechanism |
| spellingShingle | Yi Liu Hsueh-Ping Lu Ching-Hao Lai A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process IEEE Access Smart manufacturing TFT-LCD deep learning multi-modality machine learning tabular data image data and attention mechanism |
| title | A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process |
| title_full | A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process |
| title_fullStr | A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process |
| title_full_unstemmed | A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process |
| title_short | A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process |
| title_sort | novel attention based multi modal modeling technique on mixed type data for improving tft lcd repair process |
| topic | Smart manufacturing TFT-LCD deep learning multi-modality machine learning tabular data image data and attention mechanism |
| url | https://ieeexplore.ieee.org/document/9733354/ |
| work_keys_str_mv | AT yiliu anovelattentionbasedmultimodalmodelingtechniqueonmixedtypedataforimprovingtftlcdrepairprocess AT hsuehpinglu anovelattentionbasedmultimodalmodelingtechniqueonmixedtypedataforimprovingtftlcdrepairprocess AT chinghaolai anovelattentionbasedmultimodalmodelingtechniqueonmixedtypedataforimprovingtftlcdrepairprocess AT yiliu novelattentionbasedmultimodalmodelingtechniqueonmixedtypedataforimprovingtftlcdrepairprocess AT hsuehpinglu novelattentionbasedmultimodalmodelingtechniqueonmixedtypedataforimprovingtftlcdrepairprocess AT chinghaolai novelattentionbasedmultimodalmodelingtechniqueonmixedtypedataforimprovingtftlcdrepairprocess |