Overview

The UAVA,UAV-Assistant, dataset is specifically designed for fostering applications which consider UAVs and humans as cooperative agents. We employ a real-world 3D scanned dataset (Matterport3D), physically-based rendering, a gamified simulator for realistic drone navigation trajectory collection, to generate realistic multimodal data both from the user’s exocentric view of the drone, as well as the drone’s egocentric view

Motivation

With the advent of low-cost commercial mini-UAVs, new applications and ways of interactions have emerged. However, most of the existing UAV related datasets do not target such applications, prohibiting the development of data-driven methods. We introduce UAVA a dataset designed for facilitating the data-driven development of such methods. The dataset was created by leveraging an existing photorealistic dataset of indoor scenes, and by following a carefully designed gamification approach.

Trajectories

For the collection of realistic and unbiased trajectories, we developed a game on Unity3D using AirSim. Collectible cube “coins” were placed at each of the known panorama positions (anchors) and players were forced to navigate within the whole scene to collect the coins. We record the drone’s world pose among the scene at each time step t.

In the following link we provide the sampled trajectories for each scene as well as the recorded drone pose.

Modalities

We use Blender to synthesize our dataset. For the egocentric “UAV” view we generate color images, depth, and surface maps, in addition to the optical flow for two consecutive frames t, t + 1 sampled from the dense play-through trajectories. For the exocentric “user” view apart from the afforementioned modalities we also utilize a composition pipeline to add a photorealistically shared drone into the scene, and also generate its corresponding silhouette image.

Samples

Egocentric View
Exocentric View

Usage

Download

We follow a two-step procedure to download the UAVA dataset.

1. Access to UAVA dataset requires to agree with the terms and conditions for each of the 3D datasets that were used to create (i.e. render) it. Therefore, in order to grant you access to this dataset, we need to you to first fill this request form.
2. Then you need to perform request for access to Zenodo, the data hosting portal for the UAVA dataset. Due to data-size limitations and future dataset enhancements (e.g addition of new drone mdoels, etc.), the dataset is split per drone model, camera view, and modalities into seperate volumes. The first volumes contains the M2ED drone model, and data from different views. In more detail the first link contains colour, depth , and silhouette of the drone from an exocentric view M2ED Exocentic View CDS The second volume contains colour and depth data from the egocentric (i.e. UAV view) M2ED Egocentic View CD. Therefore, a separate request for access needs to be made for each viewpoint to the corresponding Zenodo repository.

Note that only completing one step of the two (i.e. only filling out the form, or only requesting access from the Zenodo repositories will not be enough to get access to the data. We will do our best to contact you in such cases and notify you to complete all steps as needed, but our mails may be lost (e.g. spam filters/folders). The only exception to this, is if you have already filled in the form and need access to another Zenodo repository (for example you need extra viewpoint renders which are hosted on different Zenodo repositories), then you only need to fill in the Zenodo request but please, make sure to mention that the form has already been filled in so that we can verify it.

Each volume is broken down in several .zip files (2GB each) for more convinient downloading on low bandwidth connections. You need all the .zip archives of each volume in order to extract the containing files.

Data splits

We follow the same data-split logic as defined in Matterport3D.

Data organisation

Acknowledgements

This dataset has been generated within the European Union’s Horizon 2020 innovation programme FASTER under grant agreement No 833507.