Description

In this challenge, you are tasked to provide a complete 3D mesh given a partial RGB-D observation of sweet peppers. You are provided with RGB-D frames with corresponding instance masks and poses where sweet peppers are only partially visible. We ask the participants to predict a 3D mesh representing the complete fruit. Obtaining such information is a fundamental building block for agricultural autonomous systems across different downstream tasks, such as harvesting and yield estimation.

Dataset Structure

To facilitate developments based on our data, we provide a visual representation of the data structure.

Codalab Submission

If you want to use our dataset, visit the codalab page: https://codalab.lisn.upsaclay.fr/competitions/18987

We expect the following data structure for the submissions:

We ran a challenge as part of the CVPPA workshop at ECCV 2024, see https://cvppa2024.github.io/. Here we report the results of the challenge at the time of the workshop:

Download

You can download the entire dataset at https://www.ipb.uni-bonn.de/html/projects/shape_completion/shape_completion_challenge.zip

Resources

We provide a small API to access the data at https://github.com/PRBonn/shape_completion_toolkit. Additionally, we provide a tech report available on ArXiv http://arxiv.org/abs/2407.13304

How to Cite

If you find this dataset useful, consider citing the following papers

@inproceedings{magistri2024icra,
author = {F. Magistri and R. Marcuzzi and E.A. Marks and M. Sodano and J. Behley and C. Stachniss},
title = {{Efficient and Accurate Transformer-Based 3D Shape Completion and Reconstruction of Fruits for Agricultural Robots}},
booktitle = icra,
year = 2024,
videourl = {https://youtu.be/U1xxnUGrVL4},
codeurl = {https://github.com/PRBonn/TCoRe},
}

@inproceedings{pan2023iros,
author = {Y. Pan and F. Magistri and T. L\"abe and E. Marks and C. Smitt and C.S. McCool and J. Behley and C. Stachniss},
title = {{Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots}},
booktitle = iros,
year = 2023,
codeurl = {https://github.com/PRBonn/HortiMapping},
videourl = {https://youtu.be/fSyHBhskjqA}
}

@article{magistri2022ral-iros,
author = {Federico Magistri and Elias Marks and Sumanth Nagulavancha and Ignacio Vizzo and Thomas L{\"a}be and Jens Behley and Michael Halstead and Chris McCool and Cyrill Stachniss},
title = {Contrastive 3D Shape Completion and Reconstruction for Agricultural Robots using RGB-D Frames},
journal = ral,
url = {https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/magistri2022ral-iros.pdf},
year = {2022},
volume={7},
number={4},
pages={10120-10127},
videourl = {https://www.youtube.com/watch?v=2ErUf9q7YOI},
}

Contact

Contact: Federico Magistri if you have any questions.

Acknowledgments

This work has been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy, EXC-2070 – 390732324 – PhenoRob.