Panoptic Anomalies In Context (PANIC)

 

The Dataset

PANIC (panoptic anomalies in context) is an open-world panoptic segmentation test set in the autonomous
driving context. The dataset is composed of 800 images and contains 58 unknown categories and more than 4000 object instances. We consider Cityscapes as the standard training set with known classes. The 58 categories are split into two groups: validation classes and test classes. Validation categories are either less interesting classes, or classes that appear in Cityscapes in the void area (i.e., unlabeled). Such classes are, for example, garbage bins, parkimeters, monuments, etc. Test classes are, instead, more challenging objects such as electric scooters, recumbent bicycles, forklifts, etc. Thus, we split the dataset into two parts: the validation set, containing only validation classes, and the test set, containing both validation and test classes. Anomalies can appear anywhere in the image. They can have any size and are not limited to active traffic participants. We provide pixel-wise annotations of semantic classes and individual object instances.

 

Description

We recorded all images of PANIC in Bonn, Germany, with our sensor platform (see IPBCar). The 19 Cityscapes evaluation classes on which most segmentation models for autonomous driving are trained, serve as the basis to determine anomalies. Everything that belongs to those 19 classes is labeled as “not anomaly”. For all the rest, we provide a pixel-wise semantic and instance annotation. To comply with privacy restrictions, we anonymize all faces,
license plates, and windows of private buildings when necessary. The images we provide have those areas in black.

We propose four benchmarks accompanied by public competitions (see below).

 

Contacts

Matteo Sodano: matteo.sodano@nulligg.uni-bonn.de
Federico Magistri: federico.magistri@nulligg.uni-bonn.de

 

How to Use

To download the dataset, please visit this link.

To participate in our competition, please visit:

  1. link for competition on anomaly segmentation;
  2. link for competition on open-world semantic segmentation;
  3. link for competition on open-set panoptic segmentation;
  4. link for competition on open-world panoptic segmentation.

Paper

If you use our dataset, please cite us:

@article{sodano2024arxiv,
  author = {M. Sodano and F. Magistri and J. Behley and C. Stachniss},
  title = {{Open-World Panoptic Segmentation}},
  journal = arxiv,
  year = 2024,
  volume = {arXiv:2412.12740},
  url = {http://arxiv.org/pdf/2412.12740},
}