Content‐augmented feature pyramid network with light linear spatial transformers for object detection
Abstract As one of the prevalent components, feature pyramid network (FPN) is widely used in current object detection models for improving multi‐scale object detection performance. However, its feature fusion mode is still in a misaligned and local manner, thus limiting the representation power. To...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-11-01
|
| Series: | IET Image Processing |
| Online Access: | https://doi.org/10.1049/ipr2.12575 |
| _version_ | 1811205686187524096 |
|---|---|
| author | Yongxiang Gu Xiaolin Qin Yuncong Peng Lu Li |
| author_facet | Yongxiang Gu Xiaolin Qin Yuncong Peng Lu Li |
| author_sort | Yongxiang Gu |
| collection | DOAJ |
| description | Abstract As one of the prevalent components, feature pyramid network (FPN) is widely used in current object detection models for improving multi‐scale object detection performance. However, its feature fusion mode is still in a misaligned and local manner, thus limiting the representation power. To address the inherited defects of FPN, a novel architecture termed content‐augmented feature pyramid network (CA‐FPN) is proposed in this paper. Firstly, a global content extraction module (GCEM) is proposed to extract multi‐scale context information. Secondly, lightweight linear spatial Transformer connections are added in the top‐down pathway to augment each feature map with multi‐scale features, where a linearized approximate self‐attention function is designed for reducing model complexity. By means of the self‐attention mechanism in Transformer, it is no longer needed to align feature maps during feature fusion, thus solving the misaligned defect. By setting the query scope to the entire feature map, the local defect can also be solved. Extensive experiments on COCO and PASCAL VOC datasets demonstrated that the CA‐FPN outperforms other FPN‐based detectors without bells and whistles and is robust in different settings. |
| first_indexed | 2024-04-12T03:35:13Z |
| format | Article |
| id | doaj.art-1fcf535dfb374ef0b950848ddbbebe58 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-04-12T03:35:13Z |
| publishDate | 2022-11-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-1fcf535dfb374ef0b950848ddbbebe582022-12-22T03:49:26ZengWileyIET Image Processing1751-96591751-96672022-11-0116133567357810.1049/ipr2.12575Content‐augmented feature pyramid network with light linear spatial transformers for object detectionYongxiang Gu0Xiaolin Qin1Yuncong Peng2Lu Li3Chengdu Institute of Computer Applications Chinese Academy of Sciences Chengdu ChinaChengdu Institute of Computer Applications Chinese Academy of Sciences Chengdu ChinaChengdu Institute of Computer Applications Chinese Academy of Sciences Chengdu ChinaZenseact Gothenburg SwedenAbstract As one of the prevalent components, feature pyramid network (FPN) is widely used in current object detection models for improving multi‐scale object detection performance. However, its feature fusion mode is still in a misaligned and local manner, thus limiting the representation power. To address the inherited defects of FPN, a novel architecture termed content‐augmented feature pyramid network (CA‐FPN) is proposed in this paper. Firstly, a global content extraction module (GCEM) is proposed to extract multi‐scale context information. Secondly, lightweight linear spatial Transformer connections are added in the top‐down pathway to augment each feature map with multi‐scale features, where a linearized approximate self‐attention function is designed for reducing model complexity. By means of the self‐attention mechanism in Transformer, it is no longer needed to align feature maps during feature fusion, thus solving the misaligned defect. By setting the query scope to the entire feature map, the local defect can also be solved. Extensive experiments on COCO and PASCAL VOC datasets demonstrated that the CA‐FPN outperforms other FPN‐based detectors without bells and whistles and is robust in different settings.https://doi.org/10.1049/ipr2.12575 |
| spellingShingle | Yongxiang Gu Xiaolin Qin Yuncong Peng Lu Li Content‐augmented feature pyramid network with light linear spatial transformers for object detection IET Image Processing |
| title | Content‐augmented feature pyramid network with light linear spatial transformers for object detection |
| title_full | Content‐augmented feature pyramid network with light linear spatial transformers for object detection |
| title_fullStr | Content‐augmented feature pyramid network with light linear spatial transformers for object detection |
| title_full_unstemmed | Content‐augmented feature pyramid network with light linear spatial transformers for object detection |
| title_short | Content‐augmented feature pyramid network with light linear spatial transformers for object detection |
| title_sort | content augmented feature pyramid network with light linear spatial transformers for object detection |
| url | https://doi.org/10.1049/ipr2.12575 |
| work_keys_str_mv | AT yongxianggu contentaugmentedfeaturepyramidnetworkwithlightlinearspatialtransformersforobjectdetection AT xiaolinqin contentaugmentedfeaturepyramidnetworkwithlightlinearspatialtransformersforobjectdetection AT yuncongpeng contentaugmentedfeaturepyramidnetworkwithlightlinearspatialtransformersforobjectdetection AT luli contentaugmentedfeaturepyramidnetworkwithlightlinearspatialtransformersforobjectdetection |