Skip to Main content Skip to Navigation
Conference papers

SAILenv: Learning in Virtual Visual Environments Made Simple

Enrico Meloni 1 Luca Pasqualini 1 Matteo Tiezzi 1 Marco Gori 1, 2 Stefano Melacci 1
2 MAASAI - Modèles et algorithmes pour l’intelligence artificielle
CRISAM - Inria Sophia Antipolis - Méditerranée , UNS - Université Nice Sophia Antipolis (... - 2019), JAD - Laboratoire Jean Alexandre Dieudonné, Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
Abstract : Recently, researchers in Machine Learning algorithms, Computer Vision scientists, engineers and others, showed a growing interest in 3D simulators as a mean to artificially create experimental settings that are very close to those in the real world. However, most of the existing platforms to interface algorithms with 3D environments are often designed to setup navigation-related experiments, to study physical interactions, or to handle ad-hoc cases that are not thought to be customized, sometimes lacking a strong photorealistic appearance and an easy-to-use software interface. In this paper, we present a novel platform, SAILenv, that is specifically designed to be simple and customizable, and that allows researchers to experiment visual recognition in virtual 3D scenes. A few lines of code are needed to interface every algorithm with the virtual world, and non-3D-graphics experts can easily customize the 3D environment itself, exploiting a collection of photorealistic objects. Our framework yields pixel-level semantic and instance labeling, depth, and, to the best of our knowledge, it is the only one that provides motion-related information directly inherited from the 3D engine. The client-server communication operates at a low level, avoiding the overhead of HTTP-based data exchanges. We perform experiments using a state-of-the-art object detector trained on real-world images, showing that it is able to recognize the photorealistic 3D objects of our environment. The computational burden of the optical flow compares favourably with the estimation performed using modern GPU-based convolutional networks or more classic implementations. We believe that the scientific community will benefit from the easiness and high-quality of our framework to evaluate newly proposed algorithms in their own customized realistic conditions.
Document type :
Conference papers
Complete list of metadata
Contributor : Marco Gori <>
Submitted on : Tuesday, October 13, 2020 - 2:39:23 PM
Last modification on : Friday, June 4, 2021 - 9:44:02 AM

Links full text


  • HAL Id : hal-02965715, version 1
  • ARXIV : 2007.08224


Enrico Meloni, Luca Pasqualini, Matteo Tiezzi, Marco Gori, Stefano Melacci. SAILenv: Learning in Virtual Visual Environments Made Simple. 25th International Conference on Pattern Recognition, Jan 2021, Milan, Italy. ⟨hal-02965715⟩



Record views