Overview

The Pano3D 360^o depth estimation benchmark provides a standard Matterport3D train and test split, as well as a secondary GibsonV2 partioning for testing and training as well. The latter is used for zero-shot cross dataset transfer performance assessment and decomposes it into 3 different splits, each one focusing on a specific generalization axis.

Data Description

Modalities

The dataset offers 3 different modalities as indicated below:

Image Type	Data Format	Invalid Value
Color	.png	true black, i.e. (0, 0, 0)
Depth	single channel, floating point .exr	(inf)
Normal	3-channel (x, y, z), floating point .exr	(0.0f, 0.0f, 0.0f) & (nan, nan, nan)

Each modality has a corresponding data format and invalid values (due to imperfect scanning, holes manifest during rendering).

Splits

M3D Filmic
GibsonV2 Tiny	GibsonV2 Medium
GibsonV2 Full	GibsonV2 Fullplus
GibsonV2 Tiny Filmic	GibsonV2 Fullplus Filmic

When training on Matterport3D the above splits offer:

• Medium: Same context (residential buildings) but different depth distribution.
  
• Full: An expanded (~3x compared to Matterport3D) training set.
  
• Filmic: Same as the original splits but with a shifted camera (color) domain. This corresponds to the covariate shift of our decomposed distribution shifts benchmark.
  

• Tiny: Same context (residential buildings) but different depth distribution. This corresponds to the prior shift of our decomposed distribution shifts benchmark.
  

• Fullplus: Different context (non exclusively residential buildings) but of the same depth distribution. This corresponds to the concept shift of our decomposed distribution shifts benchmark.
  

• Tiny Filmic: Same as the Tiny split but with a shifted camera (color) domain. This corresponds to the covariate & prior shift of our decomposed distribution shifts benchmark.
  
• Fullplus Filmic: Same as the Fullplus split but with a shifted camera (color) domain. This corresponds to the covariate & concept shift of our decomposed distribution shifts benchmark.
  

Download Data

To download the Pano3D dataset a two-step process is employed as the rendered dataset is a derivative of third party 3D datasets. To get access to download the dataset two parallel steps are required:

1. First, to agree with the terms and conditions for each of the 3D datasets that were used to create (i.e. render) it, and more specifically, Matterport3D and GibsonV2. This is accomplished by filling in this request form.
2. Second, a request for access to the respective Zenodo repositories where the data are hosted is required, one for each split/part. Due to Zenodo's quota limitations, the dataset is split into eight (8) repositories grouped in resolutions and splits/parts as follows:
   1. `1024 x 512` Resolution
      • Matterport3D Train & Test (/w Filmic)
      • GibsonV2 Full (w/o normals)
      • GibsonV2 Tiny, Medium & Fullplus (w/o normals)
      • GibsonV2 Tiny & Fullplus Filmic
   2. `512 x 256` Resolution
      • Matterport3D Train & Test (/w Filmic)
      • GibsonV2 Full
      • GibsonV2 Tiny, Medium & Fullplus (/w Filmic)

Therefore, a separate request for access needs to be made to each repository in order to download the corresponding data.

Note that only completing one step of the two (i.e. only filling out the form, or only requesting access from the Zenodo repositories will not be enough to get access to the data. We will do our best to contact you in such cases and notify you to complete all steps as needed, but our mails may be lost (e.g. spam filters/folders). The only exception to this, is if you have already filled in the form and need access to another Zenodo repository (for example you need extra viewpoint renders which are hosted on different Zenodo repositories), then you only need to fill in the Zenodo request but please, make sure to mention that the form has already been filled in so that we can verify it.

Each volume is delivered as multi-part archive and broken down in several .7z files (2GB or 4GB each) for more convinient downloading on low bandwidth connections. You need all the .7z archives of each volume in order to extract the containing files, recommendably with 7zip's GUI.

Data splits

We follow the same data-split logic as defined in Matterport3D. The GV2 subgroups are used as distinct splits with only the full variant used for training.

Acknowledgements

This dataset has been generated within the European Union’s Horizon 2020 innovation programme ATLANTIS under grant agreement No 951900.