Mixed Reality – STEM Innovation Lab

Tips and Tricks: Exploring Augmented Reality for the Countdown to Mars!

The Mars Perseverance Rover on Mars, drilling into a rock. — In this illustration, NASA’s Mars 2020 rover uses its drill to core a rock sample on Mars. Image Credit: NASA/JPL-Caltech

In the coming weeks, NASA will launch Mars Perseverance, a new rover that will seek signs of ancient life and collect rock and soil samples for possible return to Earth. Perseverance, slated to launch between July 30 and August 15, will arrive on the Red Planet on February 18, 2021 at the Jezero Crater. The new Mars Helicopter will also hitch a ride to perform a technology demonstration to test the first powered flight on Mars.

In the NASA STEAM Innovation Lab, researchers are always looking for new ways to explore NASA’s latest missions. For example, the Augmented Reality and Visualizing Solar Missions Exploration Idea Profile shows how members of the NASA STEAM Innovation Lab are researching different augmented reality technologies to display and learn about NASA solar missions. But, did you know that with this profile, you can develop with more than just missions that study our Sun?

With this research, you can also visualize different spacecraft missions through the creation of a virtual world, such as displaying a segment of Mars surface and introducing the Mars Perseverance rover’s path along the surface. To learn more about this process, visit the Exploration Idea Profile to get started!

Lani Sasser, NASA STEAM Innovation Lab Specialist and creator of this Exploration Idea Profile, says you can also experience AR and Mars with just your smartphone!

“Spacecraft AR, developed by Jet Propulsion Laboratory, is a free app downloadable on any device where you can explore Earth, Mars, planetary, launch, and antenna augmented reality experiences,” Lani says. “Have you ever wanted to see a to-scale model of the Curiosity rover in your home? Using this app, you can place a full-scale or smaller version of the Curiosity rover on any surface and examine the different features of the spacecraft and even see how certain instruments move.”

If you’re interested in trying the app out, follow this link that shows the different JPL apps, including Spacecraft AR: https://www.jpl.nasa.gov/apps/.

Mixed Reality: New setup for the Lab

Virtual Reality environments can be a challenge to fully perceive by watching through a TV screen. However, Mixed Reality, where a camera plus software can remove a green screen background to place a user directly into the environment for all to see is hugely more collaborative.

Prior to the STEM Innovation team traveling down to Kennedy Space Center (KSC) last week, the VR development team were lucky enough to receive and quickly test a Mixed Reality setup.

Mixed Reality, depending on what articles you read, seems to have slightly different meanings. Here we talk about Mixed Reality as a method for people to more clearly see what a VR user is seeing – by overlaying the person directly inside their VR environment and displaying it onto a TV screen.

Once we got the camera and software (we tested a product called LIV), we went about learning how to set it up for our Lab and our Space science related products. Our VR summer intern spent a few days trying, and found it challenging to setup the ‘triangulation part’ of the software. This part of the setup effectively generates a depth perception of the VR environment i.e. to estimate when a person is in front/behind a virtual object.

We did manage to get Mixed Reality working briefly, just prior to shipping our equipment to Kennedy, but we did not have the time to perform robust tests to ensure it would smoothly operate when the team went on the road. This meant the VR team were able to use the Parker launch at KSC was a great testbed for us to learn what is needed to demonstrate Mixed Reality while on tour during future conventions and conferences.

Have you tried a different system to operate Mixed reality in your Lab, or on the road? If so, let us know how you got along and what you liked about it?