Semantic Scene Graph Generation Using RDF Model and Deep Learning

Over the last several years, in parallel with the general global advancement in mobile technology and a rise in social media network content consumption, multimedia content production and reproduction has increased exponentially. Therefore, enabled by the rapid recent advancements in deep learning t...

Full description

Bibliographic Details
Main Authors: Seongyong Kim, Tae Hyeon Jeon, Ilsun Rhiu, Jinhyun Ahn, Dong-Hyuk Im
Format: Article
Language:English
Published: MDPI AG 2021-01-01
Series:Applied Sciences
Subjects:
Online Access:https://www.mdpi.com/2076-3417/11/2/826
Description
Summary:Over the last several years, in parallel with the general global advancement in mobile technology and a rise in social media network content consumption, multimedia content production and reproduction has increased exponentially. Therefore, enabled by the rapid recent advancements in deep learning technology, research on scene graph generation is being actively conducted to more efficiently search for and classify images desired by users within a large amount of content. This approach lets users accurately find images they are searching for by expressing meaningful information on image content as nodes and edges of a graph. In this study, we propose a scene graph generation method based on using the Resource Description Framework (RDF) model to clarify semantic relations. Furthermore, we also use convolutional neural network (CNN) and recurrent neural network (RNN) deep learning models to generate a scene graph expressed in a controlled vocabulary of the RDF model to understand the relations between image object tags. Finally, we experimentally demonstrate through testing that our proposed technique can express semantic content more effectively than existing approaches.
ISSN:2076-3417