Enhanced Image Captioning with Color Recognition Using Deep Learning Methods
Automatically describing the content of an image is an interesting and challenging task in artificial intelligence. In this paper, an enhanced image captioning model—including object detection, color analysis, and image captioning—is proposed to automatically generate the textual descriptions of ima...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/1/209 |
| _version_ | 1827668799131746304 |
|---|---|
| author | Yeong-Hwa Chang Yen-Jen Chen Ren-Hung Huang Yi-Ting Yu |
| author_facet | Yeong-Hwa Chang Yen-Jen Chen Ren-Hung Huang Yi-Ting Yu |
| author_sort | Yeong-Hwa Chang |
| collection | DOAJ |
| description | Automatically describing the content of an image is an interesting and challenging task in artificial intelligence. In this paper, an enhanced image captioning model—including object detection, color analysis, and image captioning—is proposed to automatically generate the textual descriptions of images. In an encoder–decoder model for image captioning, VGG16 is used as an encoder and an LSTM (long short-term memory) network with attention is used as a decoder. In addition, Mask R-CNN with OpenCV is used for object detection and color analysis. The integration of the image caption and color recognition is then performed to provide better descriptive details of images. Moreover, the generated textual sentence is converted into speech. The validation results illustrate that the proposed method can provide more accurate description of images. |
| first_indexed | 2024-03-10T03:50:39Z |
| format | Article |
| id | doaj.art-cbfd48818a8041e5a0a3e7821232d486 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T03:50:39Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-cbfd48818a8041e5a0a3e7821232d4862023-11-23T11:09:25ZengMDPI AGApplied Sciences2076-34172021-12-0112120910.3390/app12010209Enhanced Image Captioning with Color Recognition Using Deep Learning MethodsYeong-Hwa Chang0Yen-Jen Chen1Ren-Hung Huang2Yi-Ting Yu3Department of Electrical Engineering, Chang Gung University, Taoyuan City 333, TaiwanDepartment of Electrical Engineering, Chang Gung University, Taoyuan City 333, TaiwanDepartment of Electrical Engineering, Chang Gung University, Taoyuan City 333, TaiwanDepartment of Electrical Engineering, Chang Gung University, Taoyuan City 333, TaiwanAutomatically describing the content of an image is an interesting and challenging task in artificial intelligence. In this paper, an enhanced image captioning model—including object detection, color analysis, and image captioning—is proposed to automatically generate the textual descriptions of images. In an encoder–decoder model for image captioning, VGG16 is used as an encoder and an LSTM (long short-term memory) network with attention is used as a decoder. In addition, Mask R-CNN with OpenCV is used for object detection and color analysis. The integration of the image caption and color recognition is then performed to provide better descriptive details of images. Moreover, the generated textual sentence is converted into speech. The validation results illustrate that the proposed method can provide more accurate description of images.https://www.mdpi.com/2076-3417/12/1/209image captioncolor recognitionLSTMobject detection |
| spellingShingle | Yeong-Hwa Chang Yen-Jen Chen Ren-Hung Huang Yi-Ting Yu Enhanced Image Captioning with Color Recognition Using Deep Learning Methods Applied Sciences image caption color recognition LSTM object detection |
| title | Enhanced Image Captioning with Color Recognition Using Deep Learning Methods |
| title_full | Enhanced Image Captioning with Color Recognition Using Deep Learning Methods |
| title_fullStr | Enhanced Image Captioning with Color Recognition Using Deep Learning Methods |
| title_full_unstemmed | Enhanced Image Captioning with Color Recognition Using Deep Learning Methods |
| title_short | Enhanced Image Captioning with Color Recognition Using Deep Learning Methods |
| title_sort | enhanced image captioning with color recognition using deep learning methods |
| topic | image caption color recognition LSTM object detection |
| url | https://www.mdpi.com/2076-3417/12/1/209 |
| work_keys_str_mv | AT yeonghwachang enhancedimagecaptioningwithcolorrecognitionusingdeeplearningmethods AT yenjenchen enhancedimagecaptioningwithcolorrecognitionusingdeeplearningmethods AT renhunghuang enhancedimagecaptioningwithcolorrecognitionusingdeeplearningmethods AT yitingyu enhancedimagecaptioningwithcolorrecognitionusingdeeplearningmethods |