SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self
Scene flow tracks the 3D motion of each point in adjacent point clouds. It provides fundamental 3D motion perception for autonomous driving and server robot. Although red green blue depth (RGBD) camera or light detection and ranging (LiDAR) capture discrete 3D points in space, the objects and motion...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-04-01
|
| Series: | Advanced Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/aisy.202100197 |
| _version_ | 1817991788618579968 |
|---|---|
| author | Guangming Wang Chaokang Jiang Zehang Shen Yanzi Miao Hesheng Wang |
| author_facet | Guangming Wang Chaokang Jiang Zehang Shen Yanzi Miao Hesheng Wang |
| author_sort | Guangming Wang |
| collection | DOAJ |
| description | Scene flow tracks the 3D motion of each point in adjacent point clouds. It provides fundamental 3D motion perception for autonomous driving and server robot. Although red green blue depth (RGBD) camera or light detection and ranging (LiDAR) capture discrete 3D points in space, the objects and motions usually are continuous in the macroworld. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the generative adversarial networks (GAN) is utilized to self‐learn 3D scene flow without ground truth. The fake point cloud is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset show that our method realizes promising results. Just as human, the proposed method can identify the similar local structures of two adjacent frames even without knowing the ground truth scene flow. Then, the local correspondence can be correctly estimated, and further the scene flow is correctly estimated. An interactive preprint version of the article can be found here: https://www.authorea.com/doi/full/10.22541/au.163335790.03073492. |
| first_indexed | 2024-04-14T01:16:59Z |
| format | Article |
| id | doaj.art-9a25b0ea27d844329b479efbcf130241 |
| institution | Directory Open Access Journal |
| issn | 2640-4567 |
| language | English |
| last_indexed | 2024-04-14T01:16:59Z |
| publishDate | 2022-04-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Intelligent Systems |
| spelling | doaj.art-9a25b0ea27d844329b479efbcf1302412022-12-22T02:20:48ZengWileyAdvanced Intelligent Systems2640-45672022-04-0144n/an/a10.1002/aisy.202100197SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene SelfGuangming Wang0Chaokang Jiang1Zehang Shen2Yanzi Miao3Hesheng Wang4Department of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Engineering Research Center of Intelligent Control and Management Shanghai Jiao Tong University Shanghai 200240 ChinaEngineering Research Center of Intelligent Control for Underground Space, Ministry of Education, School of Information and Control Engineering, Advanced Robotics Research Center China University of Mining and Technology Xuzhou 221116 ChinaDepartment of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Engineering Research Center of Intelligent Control and Management Shanghai Jiao Tong University Shanghai 200240 ChinaEngineering Research Center of Intelligent Control for Underground Space, Ministry of Education, School of Information and Control Engineering, Advanced Robotics Research Center China University of Mining and Technology Xuzhou 221116 ChinaDepartment of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Engineering Research Center of Intelligent Control and Management Shanghai Jiao Tong University Shanghai 200240 ChinaScene flow tracks the 3D motion of each point in adjacent point clouds. It provides fundamental 3D motion perception for autonomous driving and server robot. Although red green blue depth (RGBD) camera or light detection and ranging (LiDAR) capture discrete 3D points in space, the objects and motions usually are continuous in the macroworld. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the generative adversarial networks (GAN) is utilized to self‐learn 3D scene flow without ground truth. The fake point cloud is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset show that our method realizes promising results. Just as human, the proposed method can identify the similar local structures of two adjacent frames even without knowing the ground truth scene flow. Then, the local correspondence can be correctly estimated, and further the scene flow is correctly estimated. An interactive preprint version of the article can be found here: https://www.authorea.com/doi/full/10.22541/au.163335790.03073492.https://doi.org/10.1002/aisy.2021001973D point cloudsgenerative adversarial networkscene flow estimationsoft correspondenceunsupervised learning |
| spellingShingle | Guangming Wang Chaokang Jiang Zehang Shen Yanzi Miao Hesheng Wang SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self Advanced Intelligent Systems 3D point clouds generative adversarial network scene flow estimation soft correspondence unsupervised learning |
| title | SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self |
| title_full | SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self |
| title_fullStr | SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self |
| title_full_unstemmed | SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self |
| title_short | SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self |
| title_sort | sfgan unsupervised generative adversarial learning of 3d scene flow from the 3d scene self |
| topic | 3D point clouds generative adversarial network scene flow estimation soft correspondence unsupervised learning |
| url | https://doi.org/10.1002/aisy.202100197 |
| work_keys_str_mv | AT guangmingwang sfganunsupervisedgenerativeadversariallearningof3dsceneflowfromthe3dsceneself AT chaokangjiang sfganunsupervisedgenerativeadversariallearningof3dsceneflowfromthe3dsceneself AT zehangshen sfganunsupervisedgenerativeadversariallearningof3dsceneflowfromthe3dsceneself AT yanzimiao sfganunsupervisedgenerativeadversariallearningof3dsceneflowfromthe3dsceneself AT heshengwang sfganunsupervisedgenerativeadversariallearningof3dsceneflowfromthe3dsceneself |