The future of augmented reality (AR) and virtual reality (VR) technology in vehicles took a leap forward as Meta, the owner of Facebook and Instagram, and German car company BMW announced successful integration of stable VR and mixed reality (MR) content for passengers in a fast-moving vehicle. This joint research, initiated in 2021, aims to explore how AR and VR technology can enhance passenger experiences in the age of smart vehicles.
The technology comes with future applications beyond the realm of traditional automobiles, with potential to transform travel in trains, planes, and other modes by offering hands-free communication, entertainment, and utility. This would replace the current reliance on screens and instruments in vehicles.
Claus Dorrer, Head of BMW Group Technology Office USA, expressed optimism about the research but acknowledged that it is too early to predict when or how this technology will become commercially available. However, he discussed a variety of potential use cases for these devices, from helping drivers find their cars in a crowded parking lot to alerting them of potential hazards on the road and providing information on vehicle conditions.
A core challenge faced by the researchers revolves around the headset sensors’ interpretation of movement inside a vehicle. The sensors within VR headsets use cameras and inertial motion sensors (IMUs) to estimate user’s location and motion. However, inside a moving car, these sensors experience conflicts where the motion sensors detect movement, while the cameras perceive the car’s interior as stationary.
To overcome this issue, researchers combined the IMU data from BMW’s sensor array with the tracking system of Project Aria research glasses. This allowed the system to calculate the glasses’ position relative to the car while also transferring the tracking system to a Meta Quest Pro headset.
The team successfully used a digital twin of the car to anchor virtual objects to the moving vehicle, thus enabling a range of compelling VR and mixed reality experiences for passengers. The next step would be to incorporate the car’s location relative to the world, enabling world-locked rendering and identifying external landmarks and points of interest.
Meta’s ongoing collaboration with BMW signifies a promising development in AR and VR technology for vehicles. It is expected to be critical in shaping the functionality of AR glasses and personalized AI assistants, and could potentially revolutionize transportation experiences. However, the technology’s commercial availability and definitive timeline for implementation remain uncertain.
Source: Cryptonews
