SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network
Siamese networks have been extensively studied in recent years. Most of the previous research focuses on improving accuracy, while merely a few recognize the necessity of reducing parameter redundancy and computation load. Even less work has been done to optimize the runtime memory cost when designi...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/4/1585 |
| _version_ | 1797476688406249472 |
|---|---|
| author | Li Cheng Xuemin Zheng Mingxin Zhao Runjiang Dou Shuangming Yu Nanjian Wu Liyuan Liu |
| author_facet | Li Cheng Xuemin Zheng Mingxin Zhao Runjiang Dou Shuangming Yu Nanjian Wu Liyuan Liu |
| author_sort | Li Cheng |
| collection | DOAJ |
| description | Siamese networks have been extensively studied in recent years. Most of the previous research focuses on improving accuracy, while merely a few recognize the necessity of reducing parameter redundancy and computation load. Even less work has been done to optimize the runtime memory cost when designing networks, making the Siamese-network-based tracker difficult to deploy on edge devices. In this paper, we present SiamMixer, a lightweight and hardware-friendly visual object-tracking network. It uses patch-by-patch inference to reduce memory use in shallow layers, where each small image region is processed individually. It merges and globally encodes feature maps in deep layers to enhance accuracy. Benefiting from these techniques, SiamMixer demonstrates a comparable accuracy to other large trackers with only 286 kB parameters and 196 kB extra memory use for feature maps. Additionally, we verify the impact of various activation functions and replace all activation functions with ReLU in SiamMixer. This reduces the cost when deploying on mobile devices. |
| first_indexed | 2024-03-09T21:05:12Z |
| format | Article |
| id | doaj.art-6d49481484d04214a3ac3cd4d9998aa7 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T21:05:12Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-6d49481484d04214a3ac3cd4d9998aa72023-11-23T22:01:54ZengMDPI AGSensors1424-82202022-02-01224158510.3390/s22041585SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking NetworkLi Cheng0Xuemin Zheng1Mingxin Zhao2Runjiang Dou3Shuangming Yu4Nanjian Wu5Liyuan Liu6State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaState Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaState Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaState Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaState Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaState Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaState Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, ChinaSiamese networks have been extensively studied in recent years. Most of the previous research focuses on improving accuracy, while merely a few recognize the necessity of reducing parameter redundancy and computation load. Even less work has been done to optimize the runtime memory cost when designing networks, making the Siamese-network-based tracker difficult to deploy on edge devices. In this paper, we present SiamMixer, a lightweight and hardware-friendly visual object-tracking network. It uses patch-by-patch inference to reduce memory use in shallow layers, where each small image region is processed individually. It merges and globally encodes feature maps in deep layers to enhance accuracy. Benefiting from these techniques, SiamMixer demonstrates a comparable accuracy to other large trackers with only 286 kB parameters and 196 kB extra memory use for feature maps. Additionally, we verify the impact of various activation functions and replace all activation functions with ReLU in SiamMixer. This reduces the cost when deploying on mobile devices.https://www.mdpi.com/1424-8220/22/4/1585visual object-trackingdeep featuressiamese networklightweight neural networkedge computing devices |
| spellingShingle | Li Cheng Xuemin Zheng Mingxin Zhao Runjiang Dou Shuangming Yu Nanjian Wu Liyuan Liu SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network Sensors visual object-tracking deep features siamese network lightweight neural network edge computing devices |
| title | SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network |
| title_full | SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network |
| title_fullStr | SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network |
| title_full_unstemmed | SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network |
| title_short | SiamMixer: A Lightweight and Hardware-Friendly Visual Object-Tracking Network |
| title_sort | siammixer a lightweight and hardware friendly visual object tracking network |
| topic | visual object-tracking deep features siamese network lightweight neural network edge computing devices |
| url | https://www.mdpi.com/1424-8220/22/4/1585 |
| work_keys_str_mv | AT licheng siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork AT xueminzheng siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork AT mingxinzhao siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork AT runjiangdou siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork AT shuangmingyu siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork AT nanjianwu siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork AT liyuanliu siammixeralightweightandhardwarefriendlyvisualobjecttrackingnetwork |