A collaborative visual sensing system for precise quality inspection at manufacturing lines
Visual sensing has been widely adopted for quality inspection in production processes. This article presents the design and implementation of a smart collaborative camera system, called BubCam, for automated quality inspection of manufactured ink bags in Hewlett-Packard (HP) Inc.'s factories. S...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/182483 |
| _version_ | 1826113871337750528 |
|---|---|
| author | Chen, Jiale Le, Duc Van Tan, Rui Ho, Daren |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Chen, Jiale Le, Duc Van Tan, Rui Ho, Daren |
| author_sort | Chen, Jiale |
| collection | NTU |
| description | Visual sensing has been widely adopted for quality inspection in production processes. This article presents the design and implementation of a smart collaborative camera system, called BubCam, for automated quality inspection of manufactured ink bags in Hewlett-Packard (HP) Inc.'s factories. Specifically, BubCam estimates the volume of air bubbles in an ink bag, which may affect the printing quality. The design of BubCam faces challenges due to the dynamic ambient light reflection, motion blur effect, and data labeling difficulty. As a starting point, we design a single-camera system that leverages various deep learning (DL)-based image segmentation and depth fusion techniques. New data labeling and training approaches are proposed to utilize prior knowledge of the production system for training the segmentation model with a small dataset. Then, we design a multi-camera system that additionally deploys multiple wireless cameras to achieve better accuracy due to multi-view sensing. To save power of the wireless cameras, we formulate a configuration adaptation problem and develop the single-agent and multi-agent deep reinforcement learning (DRL)-based solutions to adjust each wireless camera's operation mode and frame rate in response to the changes of presence of air bubbles and light reflection. The multi-agent DRL approach aims to reduce the retraining costs during the production line reconfiguration process by only retraining the DRL agents for the newly added cameras and the existing cameras with changed positions. Extensive evaluation on a lab testbed and real factory trial shows that BubCam outperforms six baseline solutions including the current manual inspection and existing bubble detection and camera configuration adaptation approaches. In particular, BubCam achieves 1.3x accuracy improvement and 300x latency reduction compared with the manual inspection approach. |
| first_indexed | 2025-03-09T10:58:17Z |
| format | Journal Article |
| id | ntu-10356/182483 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-03-09T10:58:17Z |
| publishDate | 2025 |
| record_format | dspace |
| spelling | ntu-10356/1824832025-02-07T15:38:03Z A collaborative visual sensing system for precise quality inspection at manufacturing lines Chen, Jiale Le, Duc Van Tan, Rui Ho, Daren School of Computer Science and Engineering HP-NTU Digital Manufacturing Corporate Lab Engineering Deep learning Product quality inspection Visual sensing has been widely adopted for quality inspection in production processes. This article presents the design and implementation of a smart collaborative camera system, called BubCam, for automated quality inspection of manufactured ink bags in Hewlett-Packard (HP) Inc.'s factories. Specifically, BubCam estimates the volume of air bubbles in an ink bag, which may affect the printing quality. The design of BubCam faces challenges due to the dynamic ambient light reflection, motion blur effect, and data labeling difficulty. As a starting point, we design a single-camera system that leverages various deep learning (DL)-based image segmentation and depth fusion techniques. New data labeling and training approaches are proposed to utilize prior knowledge of the production system for training the segmentation model with a small dataset. Then, we design a multi-camera system that additionally deploys multiple wireless cameras to achieve better accuracy due to multi-view sensing. To save power of the wireless cameras, we formulate a configuration adaptation problem and develop the single-agent and multi-agent deep reinforcement learning (DRL)-based solutions to adjust each wireless camera's operation mode and frame rate in response to the changes of presence of air bubbles and light reflection. The multi-agent DRL approach aims to reduce the retraining costs during the production line reconfiguration process by only retraining the DRL agents for the newly added cameras and the existing cameras with changed positions. Extensive evaluation on a lab testbed and real factory trial shows that BubCam outperforms six baseline solutions including the current manual inspection and existing bubble detection and camera configuration adaptation approaches. In particular, BubCam achieves 1.3x accuracy improvement and 300x latency reduction compared with the manual inspection approach. Agency for Science, Technology and Research (A*STAR) Published version This study is supported under the RIE2020 Industry Alignment Fund–Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner, HP Inc., through the HP-NTU Digital Manufacturing Corporate Lab. 2025-02-04T05:43:35Z 2025-02-04T05:43:35Z 2024 Journal Article Chen, J., Le, D. V., Tan, R. & Ho, D. (2024). A collaborative visual sensing system for precise quality inspection at manufacturing lines. ACM Transactions On Cyber-Physical Systems, 8(4), 3643136-. https://dx.doi.org/10.1145/3643136 2378-962X https://hdl.handle.net/10356/182483 10.1145/3643136 2-s2.0-85210922272 4 8 3643136 en IAF-ICP ACM Transactions on Cyber-Physical Systems © 2024 the Owner/Author(s). This work is licensed under a Creative Commons Attribution International 4.0 License. application/pdf |
| spellingShingle | Engineering Deep learning Product quality inspection Chen, Jiale Le, Duc Van Tan, Rui Ho, Daren A collaborative visual sensing system for precise quality inspection at manufacturing lines |
| title | A collaborative visual sensing system for precise quality inspection at manufacturing lines |
| title_full | A collaborative visual sensing system for precise quality inspection at manufacturing lines |
| title_fullStr | A collaborative visual sensing system for precise quality inspection at manufacturing lines |
| title_full_unstemmed | A collaborative visual sensing system for precise quality inspection at manufacturing lines |
| title_short | A collaborative visual sensing system for precise quality inspection at manufacturing lines |
| title_sort | collaborative visual sensing system for precise quality inspection at manufacturing lines |
| topic | Engineering Deep learning Product quality inspection |
| url | https://hdl.handle.net/10356/182483 |
| work_keys_str_mv | AT chenjiale acollaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT leducvan acollaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT tanrui acollaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT hodaren acollaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT chenjiale collaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT leducvan collaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT tanrui collaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines AT hodaren collaborativevisualsensingsystemforprecisequalityinspectionatmanufacturinglines |