It is almost a short time since NASA’s newest astronaut landed on the Red Planet. Perseverance is the first human mission on the Red Planet of its kind. Its job is to collect and store geological specimens and eventually return them to Earth. Endurance spends days exploring the Jezero Crater, an ancient delta on Mars, and the specimens it collects may provide the first evidence of extraterrestrial life. But first he must find them. This requires amazing computers, at least by Martian standards!
Endurance is significantly more self-propelled than all previous NASA astronauts, and is designed in such a way that Philip Twu, NASA’s Jet Propulsion Laboratory (JPL) robotic systems engineer, calls it a “self-propelled machine on Mars,” just like Cars on the ground.
The astronaut moves using an array of sensors that send data to the machine’s vision algorithm; But unlike land-based cars, which are equipped with the best and most expensive computers possible, the main endurance computer is almost as fast as the best personal computers (the ones we had in 1997, of course). The only reason endurance coal brains can handle automatic driving is because NASA has installed a second computer that acts like a robotic driver.
In previous rovers, routing software had to share limited processing resources with other systems. So to get from one point to another, an astronaut had to take a picture to identify his surroundings, move forward a little, then stand for a few minutes to calculate the next move. But since endurance can integrate many routing processes with a special computer, there is no need for a stop-motion approach to exploring Mars.
Instead, the main computer calculates how the astronaut will reach the desired point, and the sighted computer makes sure that the car does not hit any rocks on the way. “We’re getting closer and closer to the point where the car can drive and think at the same time,” says Tu.
Self-control is critical to the endurance probe mission. The distance between Earth and Mars is so great that light-speed radio signals take 22 minutes to make a one-way trip. This long delay makes it impossible for the astronaut to control in real time, and it is practically impossible to wait almost an hour for the signal to go back and forth.
The astronaut has a busy schedule: he has to drop a small helicopter to test flight, then collect dozens of samples and find a place on the planet to store them. A future mission returns this reserve to Earth to be studied for signs of life. If the astronaut wants to do all of this in one year for his main mission, he must be able to make many routing decisions independently.
Ground-based vehicles typically use lasers to determine the location and distance of objects; But these laser systems are gigantic, expensive, and vulnerable. Instead, Endurance uses a stereo view and a video rangefinder to find its place on the Red Planet. Stereo view combines two images from a camera on the left and a camera on the right to produce a 3D image of the surroundings. The image rangefinder, on the other hand, compares images at different times to see how far the astronaut has moved.
“We were not optimistic about the reliability of laser systems for space missions,” said Larry Matthies, senior scientist and head of computer vision at NASA’s Jet Propulsion Laboratory. “A few decades ago at JPL, when radars were not yet mature, we used stereo vision for 3D detection, and this system worked well for us.”
Metis has been involved in building video navigation systems for all the astronauts who have ever been to Mars. With the exception of Sojourner, NASA’s first astronaut, all mobile spacecraft used a combination of stereo vision and image rangefinder. But what sets endurance apart is that the astronaut uses dedicated hardware and a set of new machine vision algorithms.
The new digital glasses enable its endurance to navigate many times faster than its predecessors, resulting in more time to focus on its core scientific goals. Of course, it still takes a full day for this astronaut to cover the distance that a lazy person travels in an hour.
But endurance against previous NASA astronauts is a race car. “The maximum distance a astronaut has ever traveled in a day is 219 meters,” says Tu. “We can move about 200 meters a day, so on average, endurance either reaches or breaks the current record of astronauts.”
It is not the fault of endurance that he thinks slowly; Blame the radiation. Mars does not have a dense magnetic field or atmosphere to protect it from the sun’s charged particles, and these charged particles are destructive to computers. They can cause the transistors to turn on and off unintentionally, and if these errors accumulate, the computer will crash. This probably means losing valuable data or even failing the entire mission, so NASA engineers do everything they can to prevent these crashes from happening in the first place.
There are many techniques that make a computer safe from radiation. For example, additional transistors can be used that are more difficult to turn on and off, which greatly reduces the likelihood of this happening by a wandering ion.
Minal Sawant, space systems architect at Xilinx, a California technology company that designed and built the endurance machine vision chip, says the chip design is radiation-resistant. According to resistance tests conducted by the company, the chip should not have more than two errors – in which an ion causes a bit of data stored in memory to be converted from one to zero or vice versa – per year.
But in general, protecting a processor from radiation means sacrificing its performance. Part of this is due to the design of the processor, and part is due to the fact that testing the components of the processor is time consuming. NASA engineers do not want to use outdated technologies; Rather, they want to use technologies that they know will work. This model of Xylenex chip, which uses endurance, has been sent into space on several missions in the past, and its performance data is available for almost a decade.
“The American space industry has traditionally been very risk-averse, and there is a rationale behind it,” says Savant. A small mistake can derail the entire mission, so they prefer to use components that were already in space instead of new technologies. “Reliability is the key!”
The Xylenex machine vision computer implements new vision algorithms developed by Tu, Metis, and colleagues at NASA. Unlike self-propelled cars on the ground, the endurance does not use a set of powerful computers in the trunk to process the image. Energy and processing power on the surface of the Red Planet are precious resources. That is, the algorithms that endurance uses for routing should be as light and efficient as possible, without compromising accuracy.
“Algorithms can always go wrong, even if the hardware is flawless,” says Metis. In computer vision, there is junk data that can cause the algorithm to err. “So we have to eliminate that possibility.”
Outdated data can include situations where the astronaut cannot see an object or confuse it with something else. One solution is for the routing system data to be provided by other sensors, so the astronaut will no longer rely on sight to move. For example, gyroscopes and accelerometers help the astronaut to understand the slope and ridges of the surface.
Another solution is to expose the algorithm to as many scenarios as possible before sending it on a mission so that it will not be surprised when it reaches Mars. At NASA’s Jet Propulsion Laboratory in Pasadena, there is a large open ground that looks like the surface of Mars with boulders and red soil.
This is called the Martian Yard, and in recent years has played the role of a training ground for algorithms that guide endurance. Tu and his colleagues regularly took a copy of the astronaut to the Martian orbit, deliberately creating scenarios that they thought might confuse the astronaut. For example, if the astronaut reached a dead end, could he go back and try a new route?
“The more complex the system, the more decisions it can make,” says Tu. Making sure you cover all possible scenarios for the astronaut is a very challenging task. “But by doing a lot of real tests like this, we find algorithm flaws.”
The only problem is that there are so many ways to spread the slate in a large yard. Most tests of endurance routing algorithms were performed by virtual simulations. Here, the Mars rover team put all conceivable scenarios against the Mars rover software to see what it would do in those conditions.
There are still many modes for rocks, but there are no limits to the levels and scenarios that can be modeled. Tu says these extensive tests of the algorithm, combined with data received from endurance sensors, allow it to pass more difficult levels than any previous astronaut could.
But even the most perfect simulations fade against the real thing. If all goes well, perseverance can pave the way for evidence of extraterrestrial life in the distant past.