US Navy Scientists Teach Zero-Gravity Robot to Fly in Space Without Human Interference
The US Naval Research Laboratory (NRL) has announced the successful test of reinforcement-learning (RL)-based autonomous robotic flight in space, using an ‘Astrobee’ zero-gravity robot stationed aboard the International Space Station.
According to a statement provided to The Debrief, the first-of-its-kind autonomous spaceflight took place aboard the ISS on May 27th, with the Astrobee robot successfully undocking, maneuvering, and then re-docking with its station over a five-minute period without any need for human assistance.
The scientific team behind the project, known as APIARY (the Autonomous Planning In-space Assembly Reinforcement-learning free-flYer), believes using RL to teach robots to carry out complex tasks without direct human control could offer unparalleled capabilities to scientists, such as assembling large space telescopes or future solar power beaming stations.
“This research is significant because it marks, to our knowledge, the first autonomous robotic control in space using reinforcement learning algorithms,” explained NRL Computer Research Scientist Kenneth Stewart, Ph.D. “We believe this breakthrough will build confidence in these algorithms for space applications and generate further interest in expanding this research.”
The NRL team said the successful test also opens the possibility of teaching robots to operate autonomously in other environments, thereby providing military personnel with a critical tactical advantage.
“Reinforcement learning provides flexibility and potential to control robots across domains, from space to the ground, and from ships to underwater,” explained NRL Senior Scientist for Robotics and Autonomous Systems, Glen Henshaw, Ph.D.
[embedded content]
Simulated Zero Gravity Robots
In most space robotic applications, a controller uses teleoperation—the remote control of a mechanical device—to command and control the robot’s movements. For example, rovers on Mars receive driving instructions from Earth-based operators, which they then execute based on those specific instructions.
Although engineers are increasingly incorporating artificial intelligence into robotic systems on Earth, the team behind the new achievement notes that the complexity of operating in space has thus far limited the adoption of similar AI systems.
“Space robotics are currently in the early stages in terms of how complex autonomy is in space,” Stewart said. “It’s a risk-averse environment where teleoperation by humans is still the norm for critical tasks.”
The team also highlighted the difficulty in testing autonomous systems in space due to the high cost, noting that “one can’t realistically send a robot up to space solely for training.”
As a result, researchers in this field must rely on Earth-based simulations before deploying a real-world system in space. Unfortunately, the gap between simulation and reality can often hamper these efforts.
Hoping to bridge what they termed the “sim-to-real” gap, the NRL team utilized reinforcement-learning algorithms and powerful simulation tools to create a highly accurate simulated zero-gravity robot platform. Unlike most operator-directed robotics, RL provides the robot with a general task and a promised reward for completing it, but doesn’t tell the robot how to go about it. Instead, an RL-trained robot uses a trial-and-error process to test and eliminate different approaches before finding the correct one.
“We specialize in reinforcement learning, a cutting-edge approach to robotic control,” NRL’s Computer Research Scientist, Roxana Leontie, Ph.D., said.
For their zero-gravity robot application, the team used the Proximal Policy Optimization algorithm, a method of deep reinforcement learning. Stewart explained how, in this approach, an ‘actor network’ trains the robot to perform actions like maneuvering, while a separate ‘critic network’ evaluates its performance. Together, the two networks “efficiently train the robot to move effectively in a 3D, zero-gravity environment,” the researcher explained.
On April 30, NASA astronaut Anne McClain unpacked the first Astrobee robot—named Bumble—in the Kibo module of the International Space Station and worked with Astrobee’s team at NASA’s Ames Research Center in California’s Silicon Valley to complete an initial series of tests to verify that the robot’s subsystems were working properly (Credit: NASA).
To create a simulated version of the environment aboard the ISS to train their simulated Astrobee, the NRL team used NVIDIA’s Omniverse, a highly accurate physics simulator that can simulate the space station’s zero-gravity environment. The team also used curriculum learning, which starts training the robot in simplified environments before gradually increasing the complexity of the assigned task.
For example, the team initially tasked the simulated Astrobee zero-gravity robot with moving to a single, fixed position in space. Then the team increased the levels of randomization over time to prepare the robot to adapt to greater variation without the need for real-world testing. Stewart said this progressive training approach “substantially helped in bridging the ‘sim-to-real’ gap.”
The Test in Space
To verify if their simulated robot training would translate to the zero-gravity robots aboard the ISS, the team took advantage of a five-minute window of operations where one of the Astrobees was available. Because the small robot, which navigates the ISS with enclosed, ducted fans, is also equipped with multiple cameras, it was considered ideal to test the RL training.
“In addition to acting as a platform for space robotics experimentation, these volleyball-sized robots can help provide NASA Mission Control with flexible camera views in areas lacking fixed cameras,” Henshaw explained. “This allows ground teams to remotely inspect equipment or monitor operations without requiring astronaut intervention, freeing up valuable crew time.”
In a video released by NRL, an Astrobee loaded with the RL-trained algorithm can be seen leaving its dock, completing a maneuver, and then returning to its dock, all without operator intervention. Due to a camera glitch, the team was unable to witness the redocking of the Astrobee. However, when the video feed resumed, they saw that the zero-gravity robot had successfully completed its five-minute mission and returned to its docking station.
[embedded content]
“Our experiment marked a momentous milestone: the first successful application of reinforcement learning to a free-flying robot in space,” Leontie said. “This is particularly critical in the highly risk-averse space environment, where the immense cost of orbital assets often hinders the adoption of cutting-edge technologies.”
Applications of RL-based Robot Autonomy for Space Science
Although the test was relatively short compared to other robotic missions, NRL Space Roboticist Samantha Chapin, Ph.D., said the success of APIARY proved the viability of using RL for robot control. The researcher also described the achievement as “transformative” because it validated the team’s ability to implement highly complex autonomous robotic behaviors, “paving the way for a new era of advanced robotic operations and services in orbit.”
Leonite agreed, noting that by completing this demonstration, the team had taken a “crucial step” toward increasing mission planner confidence for incorporating autonomous robots into future space missions.
“This achievement is vital for accelerating the integration of RL into future space applications, ultimately enabling more complex and adaptable robotic missions,” the researcher said.
When discussing potential applications of their RL robot training approach beyond controlling an Astrobee, Chapin said future projects that involve deep space exploration and large-scale construction “urgently need higher levels of robotic autonomy.”
“The goal for free-flying robots in in-space assembly and servicing is to enable rapid, multi-client operations, like refueling or correcting deployment failures,” the researcher explained. “While current efforts, such as the Robotic Servicing of Geosynchronous Satellites [RSGS] project, largely rely on scripted maneuvers with limited autonomy for rendezvous and proximity operations due to their high-speed, contact-intensive nature, our research pushes for fuller autonomous capabilities.”
Conquering Diverse Domains for the Modern Warfighter
Although the successful test of the APIARY system occurred in space, the NRL team said the technology will allow the team to “rapidly adapt” robotic platforms to new tasks and environments. According to Henshaw, the team is already developing tools to rapidly model “terrestrial, maritime, and undersea environments.”
“The APIARY team’s demonstration that reinforcement learning enables autonomous systems to operate effectively in orbit proves the technology’s viability and unlocks its potential across diverse domains,” the researcher explained.
One example Henshaw offered involved building a computer model of an environment and retraining a robot to operate in that environment “in under an hour,” with just a few scans of the location. He added that this ability “will allow warfighters in the field to define new tasks and environments and then have the robot train itself to solve those problems.”
“Our vision is to equip warfighters with the power to adapt robots to any environment and any task, on demand,” Henshaw explained. “Reinforcement learning provides flexibility and potential to control robots across domains, from space to the ground, and from ships to underwater.”
Christopher Plain is a Science Fiction and Fantasy novelist and Head Science Writer at The Debrief. Follow and connect with him on X, learn about his books at plainfiction.com, or email him directly at christopher@thedebrief.org.
US Navy Scientists Teach Zero-Gravity Robot to Fly in Space Without Human Interference Read More »
