| Summary: | When deploying a semantic segmentation model into the real world, it will inevitably be confronted
with semantic classes unseen during training. Thus, to safely deploy such systems, it is crucial to
accurately evaluate and improve their anomaly segmentation capabilities. However, acquiring and
labelling semantic segmentation data is expensive and unanticipated conditions are long-tail and
potentially hazardous. Indeed, existing anomaly segmentation datasets capture a limited number of
anomalies, lack realism or have strong domain shifts. In this paper, we propose the Placing Objects
in Context (POC) pipeline to realistically add any object into any image via diffusion models. POC
can be used to easily extend any dataset with an arbitrary number of objects. In our experiments,
we present different anomaly segmentation datasets based on POC-generated data and show that
POC can improve the performance of recent state-of-the-art anomaly fine-tuning methods in several
standardized benchmarks. POC is also effective to learn new classes. For example, we use it to edit
Cityscapes samples by adding a subset of Pascal classes and show that models trained on such data
achieve comparable performance to the Pascal-trained baseline. This corroborates the low sim-to-real
gap of models trained on POC-generated images.
|
|---|