4D reconstruction & Tele-Immersion

Video gallery

Digitization has already disrupted and changed the media sector creating a complex ecosystem. Tele immersion alters the way people communicate and interact, lifts the barrier of physical presence and creates new pathways in industries such as education, healthcare, entertainment and broadcasting. Virtual Reality and 360 video are reshaping the media landscape, creating a fertile research and business environment.

Real-time photo-realistic 4D reconstruction of users and efficient compression of 4D human representations are capable of supporting varying degrees of relevant applications. Our research team has extensive experience and significant scientific expertise on real-time human 4D reconstruction and motion capture using multiple RGB-D sensors, as well as on development of immersive Virtual Reality social interaction experiences (virtual user 4D representation and self-representation) and development of AI and machine learning algorithms to process the 3D data.

The main objective of 4D reconstruction is capturing videos of a scene from multiple viewpoints and automatically building dynamic 3D models of articulated or deformable objects acting in the scene. Typical applications range from game, film industries and interactive media (virtual and augmented reality) to motion measurement and visualisation for scientific, medical or other purposes. 

Tele Immersion (TI) refers to an emerging Future Internet (FI) technology that can support realistic interpersonal communications allowing remote users to share activities and interact within shared simulated environments. With this technology the restrictions imposed by geographical location are lifted at that fine line separating the real world from the virtual, offering an all-around immersive experience, where the actual 3D appearance of all peers will be embedded.

Media sector is undergoing a metamorphosis as a result of these emerging technologies and the continuous evolution of mobile devices. The media and entertainment industry is currently seeing content consumers also acting as content creators. New forms of entertainment which are more interactive and immersive are becoming popular. The 3D effect behind the tele-immersion makes it feel like the real thing.

The ultimate goal of 4D reconstruction & Tele-Immersion is to provide the “next generation” interface for collaborators, worldwide, to work together in a virtual environment that is seamlessly enhanced by computation and large databases. When participants are tele-immersed, are able to see and interact with each other and objects in a shared virtual environment. Tele-immersion could be of immense use in medical industry and it also finds its application in the field of education.

Social VR

Imagine a party of friends emplaced into the virtual environment simulating the Athenian Acropolis, taking an exciting tour along with their guide into this cultural re-enactment, or a tele-learning environment where an expert is tele-immersed in a simulated factory environment to teach a party of workers how to operate some newly acquired machine types. Given the freedom offered by virtual environments, the interaction possibilities are endless.

VCL’s vision around TI-related technologies started since 2011 and this experience has been cultivated into an original expertise, with rich scientific knowledge and technological know-how. 


VCL’s TI incarnation, targeting a mixed reality game experience.

SpaceWars is a Tele-Immersive game where two players are placed into the same virtual arena on top of futuristic hovercrafts, where they engage each other in a Capture-the-Flag type of game. SpaceWars utilizes the latest version of the TI platform developed at VCL and was developed in order to stress-test the technology with respect to the real-time interactions between remote users within the challenging responsiveness setting of multiplayer game.

The current version of VCL’s TI platform consists only of 4 Microsoft Kinect sensors surrounding the user. The platform is portable and can be easily deployed and operated as a local TI capturing station as it is also low-cost in terms of equipment by using consumer grade vision sensors. Additionally, it utilizes an easy-to-use calibration scheme, using a custom built structure requiring commercially available materials.

 TI based ski competition

Our tele-immersion prototype allowed one indoor skier (on a ski simulator) in Greece, another indoor in Germany and an outdoor skier in Schladming, Austria to ski together and share in real-time their experience. The user in Greece, was captured and reconstructed in 3D, so that the user in Germany was able to see the performance of his 3D “replicant” (either in a Cave or in augmented reality glasses).

Amazing things are expected to happen in the fields of TI and 4D reconstruction. Our team is ready to witness and lead the development of those technologies and contribute towards creating the next big things in this area.


© Copyright 2022 VCL - All Rights Reserved

The website was made with Mobirise themes