A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model
Aiming at the demand for extracting the three-dimensional shapes of droplets in microelectronic packaging, life science, and some related fields, as well as the problems of complex calculation and slow running speed of conventional shape from shading (SFS) illumination reflection models, this paper...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2018-09-01
|
| Series: | Micromachines |
| Subjects: | |
| Online Access: | http://www.mdpi.com/2072-666X/9/9/462 |
| _version_ | 1831738078453039104 |
|---|---|
| author | Shizhou Lu Chenliang Ren Jiexin Zhang Qiang Zhai Wei Liu |
| author_facet | Shizhou Lu Chenliang Ren Jiexin Zhang Qiang Zhai Wei Liu |
| author_sort | Shizhou Lu |
| collection | DOAJ |
| description | Aiming at the demand for extracting the three-dimensional shapes of droplets in microelectronic packaging, life science, and some related fields, as well as the problems of complex calculation and slow running speed of conventional shape from shading (SFS) illumination reflection models, this paper proposes a Lambert–Phong hybrid model algorithm to recover the 3D shapes of micro-droplets based on the mask regions with convolutional neural network features (R-CNN) method to extract the highlight region of the droplet surface. This method fully integrates the advantages of the Lambertian model’s fast running speed and the Phong model’s high accuracy for reconstruction of the highlight region. First, the Mask R-CNN network is used to realize the segmentation of the highlight region of the droplet and obtain its coordinate information. Then, different reflection models are constructed for the different reflection regions of the droplet, and the Taylor expansion and Newton iteration method are used for the reflection model to get the final height of all positions. Finally, a three-dimensional reconstruction experimental platform is built to analyze the accuracy and speed of the algorithm on the synthesized hemisphere image and the actual droplet image. The experimental results show that the proposed algorithm based on mask R-CNN had better precision and shorter running time. Hence, this paper provides a new approach for real-time measurement of 3D droplet shape in the dispensing state. |
| first_indexed | 2024-12-21T13:15:25Z |
| format | Article |
| id | doaj.art-96895584ea364ec2856067bb818fb535 |
| institution | Directory Open Access Journal |
| issn | 2072-666X |
| language | English |
| last_indexed | 2024-12-21T13:15:25Z |
| publishDate | 2018-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Micromachines |
| spelling | doaj.art-96895584ea364ec2856067bb818fb5352022-12-21T19:02:45ZengMDPI AGMicromachines2072-666X2018-09-019946210.3390/mi9090462mi9090462A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong ModelShizhou Lu0Chenliang Ren1Jiexin Zhang2Qiang Zhai3Wei Liu4School of Mechanical, Electrical &Information Engineering, Shandong University at Weihai, Weihai 264209, ChinaSchool of Mechanical, Electrical &Information Engineering, Shandong University at Weihai, Weihai 264209, ChinaSchool of Mechatronics Engineering, Shanghai Jiao Tong University, Shanghai 201100, ChinaSchool of Mechanical, Electrical &Information Engineering, Shandong University at Weihai, Weihai 264209, ChinaSchool of Mechanical, Electrical &Information Engineering, Shandong University at Weihai, Weihai 264209, ChinaAiming at the demand for extracting the three-dimensional shapes of droplets in microelectronic packaging, life science, and some related fields, as well as the problems of complex calculation and slow running speed of conventional shape from shading (SFS) illumination reflection models, this paper proposes a Lambert–Phong hybrid model algorithm to recover the 3D shapes of micro-droplets based on the mask regions with convolutional neural network features (R-CNN) method to extract the highlight region of the droplet surface. This method fully integrates the advantages of the Lambertian model’s fast running speed and the Phong model’s high accuracy for reconstruction of the highlight region. First, the Mask R-CNN network is used to realize the segmentation of the highlight region of the droplet and obtain its coordinate information. Then, different reflection models are constructed for the different reflection regions of the droplet, and the Taylor expansion and Newton iteration method are used for the reflection model to get the final height of all positions. Finally, a three-dimensional reconstruction experimental platform is built to analyze the accuracy and speed of the algorithm on the synthesized hemisphere image and the actual droplet image. The experimental results show that the proposed algorithm based on mask R-CNN had better precision and shorter running time. Hence, this paper provides a new approach for real-time measurement of 3D droplet shape in the dispensing state.http://www.mdpi.com/2072-666X/9/9/462shape from shadingMask R-CNNsegment highlight regionLambert–Phong model |
| spellingShingle | Shizhou Lu Chenliang Ren Jiexin Zhang Qiang Zhai Wei Liu A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model Micromachines shape from shading Mask R-CNN segment highlight region Lambert–Phong model |
| title | A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model |
| title_full | A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model |
| title_fullStr | A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model |
| title_full_unstemmed | A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model |
| title_short | A Novel Approach to Droplet’s 3D Shape Recovery Based on Mask R-CNN and Improved Lambert–Phong Model |
| title_sort | novel approach to droplet s 3d shape recovery based on mask r cnn and improved lambert phong model |
| topic | shape from shading Mask R-CNN segment highlight region Lambert–Phong model |
| url | http://www.mdpi.com/2072-666X/9/9/462 |
| work_keys_str_mv | AT shizhoulu anovelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT chenliangren anovelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT jiexinzhang anovelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT qiangzhai anovelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT weiliu anovelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT shizhoulu novelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT chenliangren novelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT jiexinzhang novelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT qiangzhai novelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel AT weiliu novelapproachtodroplets3dshaperecoverybasedonmaskrcnnandimprovedlambertphongmodel |