SemanticDepth: Fusing Semantic Segmentation and Monocular Depth Estimation for Enabling Autonomous Driving in Roads without Lane Lines

Typically, lane departure warning systems rely on lane lines being present on the road.<br />However, in many scenarios, e.g., secondary roads or some streets in cities, lane lines are either<br />not present or not sufficiently well signaled. In this work, we present a vision-based meth...

Full description

Bibliographic Details
Main Authors: Pablo R. Palafox, Johannes Betz, Felix Nobis, Konstantin Riedl, Markus Lienkamp
Format: Article
Language:English
Published: MDPI AG 2019-07-01
Series:Sensors
Subjects:
Online Access:https://www.mdpi.com/1424-8220/19/14/3224
Description
Summary:Typically, lane departure warning systems rely on lane lines being present on the road.<br />However, in many scenarios, e.g., secondary roads or some streets in cities, lane lines are either<br />not present or not sufficiently well signaled. In this work, we present a vision-based method to<br />locate a vehicle within the road when no lane lines are present using only RGB images as input.<br />To this end, we propose to fuse together the outputs of a semantic segmentation and a monocular<br />depth estimation architecture to reconstruct locally a semantic 3D point cloud of the viewed scene.<br />We only retain points belonging to the road and, additionally, to any kind of fences or walls that<br />might be present right at the sides of the road. We then compute the width of the road at a certain<br />point on the planned trajectory and, additionally, what we denote as the fence-to-fence distance.<br />Our system is suited to any kind of motoring scenario and is especially useful when lane lines are<br />not present on the road or do not signal the path correctly. The additional fence-to-fence distance<br />computation is complementary to the road’s width estimation. We quantitatively test our method<br />on a set of images featuring streets of the city of Munich that contain a road-fence structure, so as<br />to compare our two proposed variants, namely the road’s width and the fence-to-fence distance<br />computation. In addition, we also validate our system qualitatively on the Stuttgart sequence of the<br />publicly available Cityscapes dataset, where no fences or walls are present at the sides of the road,<br />thus demonstrating that our system can be deployed in a standard city-like environment. For the<br />benefit of the community, we make our software open source.
ISSN:1424-8220