Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis

With the advancement of virtual reality and 3D game technology, the demand for high-quality 3D indoor scene generation has surged. Addressing this need, this paper presents a method leveraging a VAE-GAN-based framework to conquer two primary challenges in 3D scene representation and deep generative...

Full description

Bibliographic Details
Main Authors: Shuai Li, Hongjun Li
Format: Article
Language:English
Published: Hindawi Limited 2023-01-01
Series:International Journal of Computer Games Technology
Online Access:http://dx.doi.org/10.1155/2023/3368647
_version_ 1797672253569105920
author Shuai Li
Hongjun Li
author_facet Shuai Li
Hongjun Li
author_sort Shuai Li
collection DOAJ
description With the advancement of virtual reality and 3D game technology, the demand for high-quality 3D indoor scene generation has surged. Addressing this need, this paper presents a method leveraging a VAE-GAN-based framework to conquer two primary challenges in 3D scene representation and deep generative networks. First, we introduce a matrix representation to encode fine-grained object attributes, alongside a complete graph to implicitly capture object spatial relations—effectively encapsulating both local and global scene structures. Second, we devise a unique generative framework based on VAE-GAN and the Bayesian optimization. This framework learns a Gaussian distribution of encoded object attributes through a VAE-GAN network, allowing for sampling and decoding of the distribution to generate new object attributes. Subsequently, a U-Net is employed to learn spatial relations between objects. Lastly, the Bayesian optimization module amalgamates the generated object attributes, spatial relations, and priors learned from data, conducting global optimization to generate a logical scene layout. Experimental results on a large-scale 3D indoor scene dataset substantiate that our method effectively learns inter-object relations and generates diverse and plausible indoor scenes. Comparative experiments and user studies further validate that our method surpasses the current state-of-the-art techniques in indoor scene generation and is comparable to real training scenes.
first_indexed 2024-03-11T21:27:22Z
format Article
id doaj.art-ea2be7bb23be417a96c90f1587c58bc8
institution Directory Open Access Journal
issn 1687-7055
language English
last_indexed 2024-03-11T21:27:22Z
publishDate 2023-01-01
publisher Hindawi Limited
record_format Article
series International Journal of Computer Games Technology
spelling doaj.art-ea2be7bb23be417a96c90f1587c58bc82023-09-28T00:00:07ZengHindawi LimitedInternational Journal of Computer Games Technology1687-70552023-01-01202310.1155/2023/3368647Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene SynthesisShuai Li0Hongjun Li1College of ScienceCollege of ScienceWith the advancement of virtual reality and 3D game technology, the demand for high-quality 3D indoor scene generation has surged. Addressing this need, this paper presents a method leveraging a VAE-GAN-based framework to conquer two primary challenges in 3D scene representation and deep generative networks. First, we introduce a matrix representation to encode fine-grained object attributes, alongside a complete graph to implicitly capture object spatial relations—effectively encapsulating both local and global scene structures. Second, we devise a unique generative framework based on VAE-GAN and the Bayesian optimization. This framework learns a Gaussian distribution of encoded object attributes through a VAE-GAN network, allowing for sampling and decoding of the distribution to generate new object attributes. Subsequently, a U-Net is employed to learn spatial relations between objects. Lastly, the Bayesian optimization module amalgamates the generated object attributes, spatial relations, and priors learned from data, conducting global optimization to generate a logical scene layout. Experimental results on a large-scale 3D indoor scene dataset substantiate that our method effectively learns inter-object relations and generates diverse and plausible indoor scenes. Comparative experiments and user studies further validate that our method surpasses the current state-of-the-art techniques in indoor scene generation and is comparable to real training scenes.http://dx.doi.org/10.1155/2023/3368647
spellingShingle Shuai Li
Hongjun Li
Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis
International Journal of Computer Games Technology
title Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis
title_full Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis
title_fullStr Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis
title_full_unstemmed Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis
title_short Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis
title_sort deep generative modeling based on vae gan for 3d indoor scene synthesis
url http://dx.doi.org/10.1155/2023/3368647
work_keys_str_mv AT shuaili deepgenerativemodelingbasedonvaeganfor3dindoorscenesynthesis
AT hongjunli deepgenerativemodelingbasedonvaeganfor3dindoorscenesynthesis