Image Captioning with Word Gate and Adaptive Self-Critical Learning
Although the policy-gradient methods for reinforcement learning have shown significant improvement in image captioning, how to achieve high performance during the reinforcement optimizing process is still not a simple task. There are at least two difficulties: (1) The large size of vocabulary leads...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2018-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | http://www.mdpi.com/2076-3417/8/6/909 |
| _version_ | 1828171159295754240 |
|---|---|
| author | Xinxin Zhu Lixiang Li Jing Liu Longteng Guo Zhiwei Fang Haipeng Peng Xinxin Niu |
| author_facet | Xinxin Zhu Lixiang Li Jing Liu Longteng Guo Zhiwei Fang Haipeng Peng Xinxin Niu |
| author_sort | Xinxin Zhu |
| collection | DOAJ |
| description | Although the policy-gradient methods for reinforcement learning have shown significant improvement in image captioning, how to achieve high performance during the reinforcement optimizing process is still not a simple task. There are at least two difficulties: (1) The large size of vocabulary leads to a large action space, which makes it difficult for the model to accurately predict the current word. (2) The large variance of gradient estimation in reinforcement learning usually causes severe instabilities in the training process. In this paper, we propose two innovations to boost the performance of self-critical sequence training (SCST). First, we modify the standard long short-term memory (LSTM)based decoder by introducing a gate function to reduce the search scope of the vocabulary for any given image, which is termed the word gate decoder. Second, instead of only considering current maximum actions greedily, we propose a stabilized gradient estimation method whose gradient variance is controlled by the difference between the sampling reward from the current model and the expectation of the historical reward. We conducted extensive experiments, and results showed that our method could accelerate the training process and increase the prediction accuracy. Our method was validated on MS COCO datasets and yielded state-of-the-art performance. |
| first_indexed | 2024-04-12T03:23:07Z |
| format | Article |
| id | doaj.art-a0aef56050d0464ca041692cb29ee181 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-04-12T03:23:07Z |
| publishDate | 2018-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-a0aef56050d0464ca041692cb29ee1812022-12-22T03:49:50ZengMDPI AGApplied Sciences2076-34172018-06-018690910.3390/app8060909app8060909Image Captioning with Word Gate and Adaptive Self-Critical LearningXinxin Zhu0Lixiang Li1Jing Liu2Longteng Guo3Zhiwei Fang4Haipeng Peng5Xinxin Niu6Information Security Center, State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaInformation Security Center, State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaNational Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaNational Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaNational Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaInformation Security Center, State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaInformation Security Center, State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaAlthough the policy-gradient methods for reinforcement learning have shown significant improvement in image captioning, how to achieve high performance during the reinforcement optimizing process is still not a simple task. There are at least two difficulties: (1) The large size of vocabulary leads to a large action space, which makes it difficult for the model to accurately predict the current word. (2) The large variance of gradient estimation in reinforcement learning usually causes severe instabilities in the training process. In this paper, we propose two innovations to boost the performance of self-critical sequence training (SCST). First, we modify the standard long short-term memory (LSTM)based decoder by introducing a gate function to reduce the search scope of the vocabulary for any given image, which is termed the word gate decoder. Second, instead of only considering current maximum actions greedily, we propose a stabilized gradient estimation method whose gradient variance is controlled by the difference between the sampling reward from the current model and the expectation of the historical reward. We conducted extensive experiments, and results showed that our method could accelerate the training process and increase the prediction accuracy. Our method was validated on MS COCO datasets and yielded state-of-the-art performance.http://www.mdpi.com/2076-3417/8/6/909image captionimage understandingdeep learningcomputer vision |
| spellingShingle | Xinxin Zhu Lixiang Li Jing Liu Longteng Guo Zhiwei Fang Haipeng Peng Xinxin Niu Image Captioning with Word Gate and Adaptive Self-Critical Learning Applied Sciences image caption image understanding deep learning computer vision |
| title | Image Captioning with Word Gate and Adaptive Self-Critical Learning |
| title_full | Image Captioning with Word Gate and Adaptive Self-Critical Learning |
| title_fullStr | Image Captioning with Word Gate and Adaptive Self-Critical Learning |
| title_full_unstemmed | Image Captioning with Word Gate and Adaptive Self-Critical Learning |
| title_short | Image Captioning with Word Gate and Adaptive Self-Critical Learning |
| title_sort | image captioning with word gate and adaptive self critical learning |
| topic | image caption image understanding deep learning computer vision |
| url | http://www.mdpi.com/2076-3417/8/6/909 |
| work_keys_str_mv | AT xinxinzhu imagecaptioningwithwordgateandadaptiveselfcriticallearning AT lixiangli imagecaptioningwithwordgateandadaptiveselfcriticallearning AT jingliu imagecaptioningwithwordgateandadaptiveselfcriticallearning AT longtengguo imagecaptioningwithwordgateandadaptiveselfcriticallearning AT zhiweifang imagecaptioningwithwordgateandadaptiveselfcriticallearning AT haipengpeng imagecaptioningwithwordgateandadaptiveselfcriticallearning AT xinxinniu imagecaptioningwithwordgateandadaptiveselfcriticallearning |