How Can AI Be Used for Space Applications?

[sanman]

I came across this interesting article about how AI is being used to conjure up completely new proteins very quickly:

https://www.nature.com/articles/d41586-022-02947-7


It occurred to me that this could enable all sorts of designer bugs for ISRU purposes, and maybe even terraforming.

Perhaps we could have bugs designed to survive the Martian day/night cycle, which would come alive during the day, to perform useful conversion of natural resources, like through Sabatier or whatever.

Could we even use AI to design complex ecosystems of organisms that would cope with the existing Mars conditions while working to transform the environment into one that's more human-friendly?

[MickQ]

Sounds a bit “Sax Russell “ to me.

A great idea though.  A lot more sophisticated than throwing rocks and/or nukes around, and more controllable IMHO.

[sanman]

Another interesting use of AI is in image enhancement:

https://ai.googleblog.com/2021/07/high-fidelity-image-generation-using.html

The article shows various examples of low-resolution images being enhanced into higher-resolution images. This would obviously be useful for satellite imaging.

What are the limits of this approach? What are the risks of introducing errors through this process.

[Phil Stooke]

Garbage in, garbage out.  The low resolution images are being made to look like high resolution images.  Actual new information is not being created.  I admit that the better-looking image might be easier to interpret and can be a help, but new information is not being created.  Scientific uses would be a lot more limited than uses in other fields (like movie CGI).  The people who want to sell these kinds of things will claim they can do things they cannot really do.

[laszlo]

Also, keep in mind that none of this is artificial intelligence, any more than solving integrals, playing chess, natural language translation or image identification was. The tendency is for AI investigators is to identify something that they find difficult to do as a benchmark of intelligence, develop an "AI" system that accomplishes the task and then realize that the product is performing with no intelligence at all. So they go back to their terminals, move the goalposts and try again. In the meantime, entrepreneurs productize these idiot savant creations and the popular press announces the imminent Rise of the Machines, again. This has been going on since the 1950's, at least.

The current iteration of AI is a deep learning system that instead of being explicitly programmed to do a specific task learns how to do it by being fed enormous amount of training data, guessing at what the right answer is and having the guess evaluated. The quality of the current guess is used to adjust how future guesses are made until they are consistently good enough. Then whenever new data is presented the deep learning system will make a guess based on its experience with the training data sets. This is the principle behind the current generation of "AI" systems, minus the mathematics and implementation details.

While this sounds similar to the way an intelligent being learns, there's no intelligence involved at all, at least not in operation (the design and implementation is a different story). These systems are basically self-training classifiers. They divide training data into sets and they determine which set the operational data belongs to. They are pattern matchers based on statistical models derived from the training data. And as Phil Stooke points out above, GIGO.

If you really want to determine how deep learning ("AI")  can be used for space applications, look for problems that require the classification of data into sets. You also need massive amounts of training data to be available, which is why I am skeptical that we're ready to design Mars ISRU bugs yet. We don't have enough information about the Martian environment and interactions with living creatures yet to train the deep learning systems.

[sanman]

Well, there also appear to be more recently explored approaches like Generative Adversarial Networks that are using Darwinistic type algorithms, and just rapidly iterating through them to create new content. So I assume that's how it gets new information to come up with possible proteins. Real biological Darwinism hasn't been able to get a foothold on Mars, but maybe some kind of artificial/digital forced Darwinism can get us past the initial bootstrapping phase. Then after that, we can rely on real biological Darwinism to take over on Mars.

As for classifiers, for a lot of routine Earthly applications, that's all you need. If your AI is looking through satellite imagery to spot a tank on the ground, then as long as it's already trained by looking at enough tanks, then it should be able to meet the needs. If it's looking for debris of a crashed space probe on the surface of the Moon, then it would have had to train by looking at enough debris fields. If you're trying to upscale a person's face from low-res to high-res, then having trained on a bunch of faces won't be enough to know if a mole should be visible in the high-res image that wasn't apparent in the low-res one. But for generalizations, it should be okay to interpolate/extrapolate. Rather than creating new information, you're "transferring" (inferring) it from the training data.

[john smith 19]

Garbage in, garbage out.  The low resolution images are being made to look like high resolution images.  Actual new information is not being created.  I admit that the better-looking image might be easier to interpret and can be a help, but new information is not being created.  Scientific uses would be a lot more limited than uses in other fields (like movie CGI).  The people who want to sell these kinds of things will claim they can do things they cannot really do.

Now if they'd said "We are using these low resolution images as part of a collection of data sources that help us decide what something is (or is not)" that would be closer to a task humans do, and have done that does call for intelligence. Examples include submarine tracking and (in principle) weather forecasting (the way humans did it, not using GCMs)

Technology looking for a problem? Surely if the problem was that difficult to solve someone would have thought of applying AI long ago.

There is an interesting subset of design called "Inverse" or "optimal" design which seeks to take parameters constituting an "optimal" solution (for some suitable defintion of the word) and work backward to an actual design (or a set of designs that fulfil those criteria).

I don't think any of the proponents of such concepts have ever labelled them as AI, and given the deterministic nature of the process I don't think it is. OTOH it could save a hell of a lot of time and money.

[leovinus]

Also, keep in mind that none of this is artificial intelligence, any more than solving integrals, playing chess, natural language translation or image identification was.

Which is why the practitioners, scientists and engineers, in the field refer to "machine learning" (ML) or ML/AI.

The term "Artificial Intelligence" works well for the public but I always cringe when I hear it. One reason we do not like the AI name is that we grew up with it in university in the 80s and 90s. AI was going to explain "why" things do work, and generalize from one given information to new insights. Prolog was one of the main languages (and I still do not like it )

Japan had a major investment to build a 5th generation AI machine, made major investments, and it all did not work as the tools like Prolog were not up to the job. For example The fifth generation : artificial intelligence and Japan's computer challenge to the world. As far as I concerned, Japan's effort from that time was to be applauded but again the tools were not there.

Same thing with "simple" backprop in the 80s. We could not compute quick enough with enough data so it was a toy.

Then we used hidden Markov models (HMM) for decades (80s to 10s) on NLP, translation, speech, handwriting

Then we had computing power and suddenly the old tool of multi layer perception (MLP) from the 80s worked so much better than HMMs which was a surprise. 

And now we are back on "explainable AI" or XAI in the context of deep learning.

Science and engineering moves in waves and circles, iterative improvement.

Have a look at something like this ESA-Ariel Data Challenge NeurIPS 2022: Introduction to exo-atmospheric studies and presentation of the Ariel Big Challenge (ABC) Database and Reinforcement Learning for robotics et al

[Twark_Main]

Also, keep in mind that none of this is artificial intelligence, any more than solving integrals, playing chess, natural language translation or image identification was. The tendency is for AI investigators is to identify something that they find difficult to do as a benchmark of intelligence, develop an "AI" system that accomplishes the task and then realize that the product is performing with no intelligence at all. So they go back to their terminals, move the goalposts and try again. In the meantime, entrepreneurs productize these idiot savant creations and the popular press announces the imminent Rise of the Machines, again. This has been going on since the 1950's, at least.

The joke-that's-not-a-joke within the field is that "AI is the set of all software problems we can't solve yet."

Once we solve a given AI problem (eg defeating a chess grandmaster) it immediately gets its own specialized name ("two-player game playing") and thus it no longer falls under "AI." A nice catch-22!

AI is, fundamentally, a marketing term. It has no consistent technical definition. I once heard a tongue-in-cheek suggestion of "any program which contains at least one branching instruction," and based on real-world corporate usage that definition seems about right actually... 

[deltaV]

AI is, fundamentally, a marketing term. It has no consistent technical definition.

I agree.

I once heard a tongue-in-cheek suggestion of "any program which contains at least one branching instruction," and based on real-world corporate usage that definition seems about right actually... 

I bet the max(0, x) in deep neural network ReLU units is typically computed using instructions for max rather than branches. If that's true then deep neural networks don't actually include any branches in the core code. So modern AI code may actually execute fewer branches per second than many non-AI tasks such as sorting.

[JohnFornaro]

Garbage in, garbage out.  The low resolution images are being made to look like high resolution images.  Actual new information is not being created.  I admit that the better-looking image might be easier to interpret and can be a help, but new information is not being created.  Scientific uses would be a lot more limited than uses in other fields (like movie CGI).  The people who want to sell these kinds of things will claim they can do things they cannot really do.

I would add that in some cases, "new" info is artificially added to the image.  Remember that face on Mars?  Some of those images were doctored enhanced to make the face more realistic, simian in some cases.  They didn't call that AI then, but now they do.

[sanman]

Here's an overview from DARPA's John Launchbury on the "3 Waves of AI":




So we're currently in the middle of realizing the fruits of the 2nd wave, but the 3rd wave is yet to come.

[Greg Hullender]

I spent my whole professional career working on "AI" (actually natural-language and machine-learning software), so I think I can add a few useful remarks.

For this discussion, I think the following definition of "AI" will serve everyone well: AI is software that attacks problems that cannot be solved by brute force but which humans can do "intuitively" without being able to explain how they do it.

So in the protein-folding example, humans use a lot of intuition to find solutions even though the search space is way too large to fully explore. An AI solution looks at the results the humans come up with and deduces some rules. These are almost certainly not rules the humans actually use, but as long as they do the job, who cares?

It occurred to me that this could enable all sorts of designer bugs for ISRU purposes, and maybe even terraforming.
.
.
.
Could we even use AI to design complex ecosystems of organisms that would cope with the existing Mars conditions while working to transform the environment into one that's more human-friendly?

As far as I know, a human team would be unable to create "designer bugs," so AI can't do it either. There is no magic here.

Also, keep in mind that none of this is artificial intelligence, any more than solving integrals, playing chess, natural language translation or image identification was. The tendency is for AI investigators is to identify something that they find difficult to do as a benchmark of intelligence, develop an "AI" system that accomplishes the task and then realize that the product is performing with no intelligence at all. So they go back to their terminals, move the goalposts and try again. In the meantime, entrepreneurs productize these idiot savant creations and the popular press announces the imminent Rise of the Machines, again. This has been going on since the 1950's, at least.

There are people who do this, but I don't consider them serious researchers. "AI" actually has two wings to it, which I like to describe as corresponding to Astronomy vs. Astrology. The folks who freely use the term "AI" are the astrologers. The ones who prefer to say "machine learning" and who don't make extravagant claims are the astronomers. (Obviously I have a bias here.) :-)

The current iteration of AI is a deep learning system that instead of being explicitly programmed to do a specific task learns how to do it by being fed enormous amount of training data, guessing at what the right answer is and having the guess evaluated. The quality of the current guess is used to adjust how future guesses are made until they are consistently good enough. Then whenever new data is presented the deep learning system will make a guess based on its experience with the training data sets. This is the principle behind the current generation of "AI" systems, minus the mathematics and implementation details.

This is a pretty good description of how machine learning works. No one pretends it has anything to do with actual intelligence, nor imagines that it ever will. Not all machine learning is classifiers, though. The protein-folding software isn't, for example.

Well, there also appear to be more recently explored approaches like Generative Adversarial Networks that are using Darwinistic type algorithms, and just rapidly iterating through them to create new content.

Sadly, this is from the astrology side of AI. Things like "evolutionary algorithms" and "swarms" all sound cool, but they all underperform simple hill-climbing-with-restart algorithms--when they work at all. A co-worker at Microsoft spent eighteen months trying to get something useful out of this stuff, and that was his conclusion at the end of it. We also attended a few conferences and read a number of papers, and the results weren't encouraging. Things can always change, of course, but there's a certain kind of magical thinking that suffuses that camp which contrasts sharply with the coldly mathematical air you find at machine-learning conferences, and that doesn't make me optimistic.

As for classifiers, for a lot of routine Earthly applications, that's all you need. If your AI is looking through satellite imagery to spot a tank on the ground, then as long as it's already trained by looking at enough tanks, then it should be able to meet the needs. If it's looking for debris of a crashed space probe on the surface of the Moon, then it would have had to train by looking at enough debris fields.

The problem with training a system to spot a crashed space probe is that you don't have very many examples of real space probes to learn from. And AI is not very good at identifying the unexpected. Classifiers tend to be like my late great aunt, who always had an opinion on everything, whether she knew anything about it or not. It's quite difficult to design a useful system that will say, "I don't know what this is." Either they say it too often or not often enough. There are exceptions, e.g. when a problem can be characterized statistically, but I struggle to see how to characterize the problem of finding a crashed probe on Mars that way.

If you're trying to upscale a person's face from low-res to high-res, then having trained on a bunch of faces won't be enough to know if a mole should be visible in the high-res image that wasn't apparent in the low-res one. But for generalizations, it should be okay to interpolate/extrapolate. Rather than creating new information, you're "transferring" (inferring) it from the training data.

I've played with this software some, and I've been pretty unimpressed. I'm not sure exactly what they're doing, but on the dozen or so pictures I tested it with, it wasn't clear that it made any of them better. At this point, I'd say, don't expect too much from this. Again, I could be wrong, but I'm not optimistic at this point.

The joke-that's-not-a-joke within the field is that "AI is the set of all software problems we can't solve yet."

Once we solve a given AI problem (eg defeating a chess grandmaster) it immediately gets its own specialized name ("two-player game playing") and thus it no longer falls under "AI." A nice catch-22!

I've been making this joke myself since the late 1970s. :-) It never gets old. Another one is, "AI is any cool algorithm that almost works." (If it actually does work, it's not AI anymore.)

Anyway, I'd say that machine learning does have a lot to contribute to space applications, mostly by automating routine human tasks so they can be done by a probe in real time rather than requiring a long wait due to the speed-of-light lag. There are probably other areas--look for anything that requires lots of tedious human labor.

[jimvela]

Autonomous onboard navigation...

[Greg Hullender]

Another point worth mentioning: Machine learning is usually something that happens in the factory, not in the field. That is, the classifier (or other "AI" technology) in a space probe would most likely be trained on the ground here on Earth. Based on the data collected during the mission, NASA might retrain the system and upload the result. Retraining takes huge computer resources and can run for days, weeks--even months. In this process, the system does not learn to do anything new; it just learns how to do better at the tasks it was originally designed for. (Of course nothing stops the engineers from also adding new capabilities at the same time.)

This retraining might happen once every year or two. But the system itself would not learn anything in between times.

There are systems that do attempt to learn in real time, but I have never seen one that worked very well. You've got a system that was trained (for example) on millions of samples of data and you're trying to update it with a tiny fraction of that amount of new data. Usually, you either get no measurable effect or else you "poison" the system by placing too much emphasis on the new data.

And even those systems don't try to learn new things; they just learn to do their old jobs better.

[leovinus]

AI is, fundamentally, a marketing term. It has no consistent technical definition.

I agree.

I once heard a tongue-in-cheek suggestion of "any program which contains at least one branching instruction," and based on real-world corporate usage that definition seems about right actually... 

I bet the max(0, x) in deep neural network ReLU units is typically computed using instructions for max rather than branches. If that's true then deep neural networks don't actually include any branches in the core code. So modern AI code may actually execute fewer branches per second than many non-AI tasks such as sorting.

With respect to "instructions for max", that just means that the "if" conditions are coded in microcode on the CPU one level lower. In other words, the "if" condition is still there.

With respect to "So modern AI code may actually execute fewer branches per second than many non-AI tasks such as sorting.", sure, but essentially that is just good software design. For example, a matrix-matrix calculation should be coded not with three nested loops, but with a GEMM() library call optimized for whatever platform you work which therefore minimizes overhead, branches etc.

[leovinus]

Autonomous onboard navigation...

Well, let's start with making better software for avionics and anything. While I might be missing something, there was tweet yesterday by Chris G. saying something like "If the SLS countdown is halted shorter than T-33 seconds then the consequence is a SCRUB as the SLS avionics cannot handle that.". My memory might be wrong and there are various experts here who will know the answer but I seem to remember that the Shuttle allowed for halt below T-33 seconds? If my memory is correct then it would mean that newer SLS avionics have less functionality in a Shuttle derived system which would be disturbing. Adding an "Autonomous" level is additional level of complexity beyond that. Hence, first get the basics right and then we can talk about AI additions

[JayWee]

It occurred to me that this could enable all sorts of designer bugs for ISRU purposes, and maybe even terraforming.
.
.
.
Could we even use AI to design complex ecosystems of organisms that would cope with the existing Mars conditions while working to transform the environment into one that's more human-friendly?

As far as I know, a human team would be unable to create "designer bugs," so AI can't do it either. There is no magic here.

Isn't this exactly the kind of stuff a genetic evolutionary approach could tackle? To try to synthesize a viable biochemical paths working in the Martian environment?
Btw, there've been things like genetically evolved antennas done by NASA:
https://en.wikipedia.org/wiki/Evolved_antenna
(As a side note - as you've been in this field for a long time - do genetic evolution algos belong usually under the AI moniker?)

Autonomous onboard navigation...

...
Adding an "Autonomous" level is additional level of complexity beyond that. Hence, first get the basics right and then we can talk about AI additions

Well, the Mars rovers do have some autonomy. They do absolutely need it - you can't teleoperate them directly from Earth.  I absolutely expect Tesla's FSD-derived autonomy on Mars/Moon one day.

[Phil Stooke]

"If it's looking for debris of a crashed space probe on the surface of the Moon, then it would have had to train by looking at enough debris fields. "

Not necessarily.  I many cases you compare your new image with a pre-impact image.  Example: the Chang'e 1 orbiter impact... not present in an Apollo 16 panoramic camera image but present in LRO images.  Almost any future impact site will be findable that way.  Older impacts like (for instance) the Apollo 16 LM ascent stage are much trickier.  But if you think AI can do it better than a human, think again.

[Greg Hullender]

Isn't this exactly the kind of stuff a genetic evolutionary approach could tackle? To try to synthesize a viable biochemical paths working in the Martian environment?
Btw, there've been things like genetically evolved antennas done by NASA:
https://en.wikipedia.org/wiki/Evolved_antenna
(As a side note - as you've been in this field for a long time - do genetic evolution algos belong usually under the AI moniker?)

Yes, evolutionary algorithms are definitely part of "AI". And, when you can do a generation every six seconds or so (give or take), you at least have a chance of getting a useful result from them. The question, though, is whether there's actually a simpler, cheaper way to get that same result (or better). I'm not familiar with the evolved antenna, but I suspect that some other, less fancy search algorithm would work at least as well with less complexity. In other words, evolutionary algorithms always appear to be a solution in search of a problem, and the places that actually use them seem to have viewed using that algorithm as a goal "Yes! We used evolutionary algorithms!" rather than trying to find the best solution to the underlying problem.

I should confess here that part of my negativity comes from a particular group I had the misfortune of interacting with when I worked at Amazon. They kept coming up with "solutions" for hard problems in different areas, but their software almost never worked very well--it was generally way too slow and nowhere near as accurate as they claimed it to be. However, they had friends in high places, and they consistently tried to use politics to force people to adopt their software when they couldn't win on the merits. Obviously, it's not fair to blame everyone working on EA for what one group of people did.

However, the gripes against evolutionary algorithms are common in the field. Go to just about any ML conference and ask some of the experts their opinions on them. I'm far from the only person who's down on them.

As for applying them to biological evolution on Mars, the fact that the algorithms are inspired by evolution doesn't mean they're really all that well suited to solving problems involving the real thing. In this case, the challenge is that a) I don't think we can engineer new organisms all that accurately in the first place and b) I suspect we can't really model the Martian environment all that well either. We might get there someday, of course, but even then I suspect there will be better approaches.