Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing
Recently, deep learning models have become more prominent due to their tremendous performance for real-time tasks like face recognition, object detection, natural language processing (NLP), instance segmentation, image classification, gesture recognition, and video classification. Image captioning i...
| Hlavní autoři: | , , , , , |
|---|---|
| Médium: | Článek |
| Jazyk: | English |
| Vydáno: |
IEEE
2023-01-01
|
| Edice: | IEEE Access |
| Témata: | |
| On-line přístup: | https://ieeexplore.ieee.org/document/10355962/ |
| _version_ | 1827398971380727808 |
|---|---|
| author | Rana Othman Alnashwan Samia Allaoua Chelloug Nabil Sharaf Almalki Imene Issaoui Abdelwahed Motwakel Ahmed Sayed |
| author_facet | Rana Othman Alnashwan Samia Allaoua Chelloug Nabil Sharaf Almalki Imene Issaoui Abdelwahed Motwakel Ahmed Sayed |
| author_sort | Rana Othman Alnashwan |
| collection | DOAJ |
| description | Recently, deep learning models have become more prominent due to their tremendous performance for real-time tasks like face recognition, object detection, natural language processing (NLP), instance segmentation, image classification, gesture recognition, and video classification. Image captioning is one of the critical tasks in NLP and computer vision (CV). It completes conversion from image to text; specifically, the model produces description text automatically based on the input images. In this aspect, this article develops a Lighting Search Algorithm (LSA) with a Hybrid Convolutional Neural Network Image Captioning System (LSAHCNN-ICS) for NLP. This introduced LSAHCNN-ICS system develops an end-to-end model which employs convolutional neural network (CNN) based ShuffleNet as an encoder and HCNN as a decoder. At the encoding part, the ShuffleNet model derives feature descriptors of the image. Besides, in the decoding part, the description of text can be generated using the proposed hybrid convolutional neural network (HCNN) model. To achieve improved captioning results, the LSA is applied as a hyperparameter tuning strategy, representing the innovation of the study. The simulation analysis of the presented LSAHCNN-ICS technique is performed on a benchmark database, and the obtained results demonstrated the enhanced outcomes of the LSAHCNN-ICS algorithm over other recent methods with maximum Consensus-based Image Description Evaluation (CIDEr Code) of 43.60, 59.54, and 135.14 on Flickr8k, Flickr30k, and MSCOCO datasets correspondingly. |
| first_indexed | 2024-03-08T19:37:34Z |
| format | Article |
| id | doaj.art-a9f94894aeae4f27bf785f94f07d0806 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T19:37:34Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a9f94894aeae4f27bf785f94f07d08062023-12-26T00:04:39ZengIEEEIEEE Access2169-35362023-01-011114264314265110.1109/ACCESS.2023.334270310355962Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language ProcessingRana Othman Alnashwan0Samia Allaoua Chelloug1https://orcid.org/0000-0002-9711-0235Nabil Sharaf Almalki2Imene Issaoui3Abdelwahed Motwakel4https://orcid.org/0000-0003-4084-5457Ahmed Sayed5Department of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box, 84428, Riyadh, Saudi ArabiaDepartment of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box, 84428, Riyadh, Saudi ArabiaDepartment of Special Education, College of Education, King Saud University, Riyadh, Saudi ArabiaUnit of Scientific Research, Applied College, Qassim University, Buraydah, Saudi ArabiaDepartment of Information Systems, College of Business Administration in Hawtat Bani Tamim, Prince Sattam bin Abdulaziz University, Al-Kharj, Saudi ArabiaResearch Center, Future University in Egypt, New Cairo, EgyptRecently, deep learning models have become more prominent due to their tremendous performance for real-time tasks like face recognition, object detection, natural language processing (NLP), instance segmentation, image classification, gesture recognition, and video classification. Image captioning is one of the critical tasks in NLP and computer vision (CV). It completes conversion from image to text; specifically, the model produces description text automatically based on the input images. In this aspect, this article develops a Lighting Search Algorithm (LSA) with a Hybrid Convolutional Neural Network Image Captioning System (LSAHCNN-ICS) for NLP. This introduced LSAHCNN-ICS system develops an end-to-end model which employs convolutional neural network (CNN) based ShuffleNet as an encoder and HCNN as a decoder. At the encoding part, the ShuffleNet model derives feature descriptors of the image. Besides, in the decoding part, the description of text can be generated using the proposed hybrid convolutional neural network (HCNN) model. To achieve improved captioning results, the LSA is applied as a hyperparameter tuning strategy, representing the innovation of the study. The simulation analysis of the presented LSAHCNN-ICS technique is performed on a benchmark database, and the obtained results demonstrated the enhanced outcomes of the LSAHCNN-ICS algorithm over other recent methods with maximum Consensus-based Image Description Evaluation (CIDEr Code) of 43.60, 59.54, and 135.14 on Flickr8k, Flickr30k, and MSCOCO datasets correspondingly.https://ieeexplore.ieee.org/document/10355962/Deep convolutional neural networknatural language processingimage captioningmachine learninghyperparameter tuning |
| spellingShingle | Rana Othman Alnashwan Samia Allaoua Chelloug Nabil Sharaf Almalki Imene Issaoui Abdelwahed Motwakel Ahmed Sayed Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing IEEE Access Deep convolutional neural network natural language processing image captioning machine learning hyperparameter tuning |
| title | Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing |
| title_full | Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing |
| title_fullStr | Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing |
| title_full_unstemmed | Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing |
| title_short | Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing |
| title_sort | lighting search algorithm with convolutional neural network based image captioning system for natural language processing |
| topic | Deep convolutional neural network natural language processing image captioning machine learning hyperparameter tuning |
| url | https://ieeexplore.ieee.org/document/10355962/ |
| work_keys_str_mv | AT ranaothmanalnashwan lightingsearchalgorithmwithconvolutionalneuralnetworkbasedimagecaptioningsystemfornaturallanguageprocessing AT samiaallaouachelloug lightingsearchalgorithmwithconvolutionalneuralnetworkbasedimagecaptioningsystemfornaturallanguageprocessing AT nabilsharafalmalki lightingsearchalgorithmwithconvolutionalneuralnetworkbasedimagecaptioningsystemfornaturallanguageprocessing AT imeneissaoui lightingsearchalgorithmwithconvolutionalneuralnetworkbasedimagecaptioningsystemfornaturallanguageprocessing AT abdelwahedmotwakel lightingsearchalgorithmwithconvolutionalneuralnetworkbasedimagecaptioningsystemfornaturallanguageprocessing AT ahmedsayed lightingsearchalgorithmwithconvolutionalneuralnetworkbasedimagecaptioningsystemfornaturallanguageprocessing |