Exploring the Future of Mixed-Reality Navigation with Android XR
At this year’s Google I/O, Google unveiled an exciting update for spatial experiences that is set to revolutionize navigation in mixed-reality environments. The Geospatial API is now available in a preview format within ARCore for Jetpack XR, bringing the power of Google’s Visual Positioning System (VPS) to Android XR. This advancement allows digital content to be anchored to the physical world with remarkable sub-meter accuracy and orientation precision in designated areas.
Introducing the XR Geospatial Tour
To demonstrate the potential of the Geospatial API, Google’s team developed a prototype called the XR Geospatial Tour. This innovative application enables users to explore new cities equipped with wired XR glasses, such as the upcoming XREAL Project Aura. Instead of relying on traditional 2D maps, users can navigate through interactive 3D models and receive insightful information about historical landmarks directly in their field of vision.
Combining Technologies for Immersive Experiences
The XR Geospatial Tour is the result of an impressive integration of various technologies. By leveraging the Geospatial APIs, Gemini API, Google Maps Grounding, and Jetpack XR SDK, the team created a hands-free and immersive walking tour experience. This fusion of technologies allows for a seamless transition from digital to physical navigation.
Precision Navigation with Geospatial API
The key to enhancing navigation in XR lies in the combination of GPS and VPS. The Geospatial API provides a GeospatialPose that includes precise latitude, longitude, and heading, surpassing traditional GPS accuracy. To ensure the effectiveness of the experience, the system continuously monitors horizontal accuracy and orientation yaw accuracy. In case the user is indoors or in an unrecognized area, prompts are provided to guide them to an outdoor public space.
AI-Powered Tour Guidance
Once the user’s location is established, the Gemini API utilizes Firebase AI Logic to generate a structured JSON response that includes nearby walking tours. The Gemini model acts as a virtual tour guide, offering rich descriptions of locations. However, to avoid inaccuracies in coordinates, Google Maps Grounding is employed to ensure the AI-generated information is reliable.
Dynamic Voiceovers and User Interaction
To enhance the user experience further, dynamic voiceovers are integrated using the gemini-2.5-flash-tts model. This allows the tour guide to provide audio feedback rather than just text, making the experience more engaging. The Jetpack XR SDK facilitates the rendering of this data in the user’s view, creating an intuitive blend of 2D and spatial computing.
Creating Interactive Points of Interest
The development team also utilized Jetpack Compose for XR to create spatial components that represent points of interest along the tour. One such component is the InfoSphere, a 3D orb that floats in space and can be interacted with to reveal additional information. The integration of AnimatedSpatialVisibility with SceneCoreEntity 3D elements allows for a seamless interaction experience.
Empowering Developers with New Tools
The creation of the XR Geospatial Tour app highlights that the barriers to crafting world-scale spatial experiences are diminishing for Android developers. With the Geospatial API now in preview on Android XR, developers can create applications that understand their surroundings with ease. By incorporating Compose for XR’s APIs alongside the high-precision location data from VPS and the generative capabilities of Gemini, the possibilities for innovative applications are vast.
Join the Android XR Developer Catalyst Program
To encourage developers to explore the potential of Android XR, Google has opened applications for the Android XR Developer Catalyst Program. This initiative includes access to the XREAL Project Aura devkit and display glasses devkit, providing developers with the tools needed to create cutting-edge mixed-reality applications.
As we look ahead, the integration of advanced technologies like the Geospatial API and Gemini will undoubtedly shape the future of navigation and immersive experiences in mixed-reality environments.
Source for the original facts: Original source.
