Humanoid Robots for Moon and Mars

[Cheapchips]

If it was possible to get results with simple verbal or text instructions, would the robots be more useful?

World model research is essentially trying to achieve instruction to action for unseen scenarios.  The current state of the art has limited time horizons for actions and the success rate needs to improve.  Rapid improvement is what we see across all these 'AI' fields.  It's reasonable to assume that this will be no different.

1X posted an article on their efforts with this. 

https://www.1x.tech/discover/world-model-self-learning

(I'm not sure how universal the peace V sign they demo is, but that's definately the wrong way round for a UK audience!)

[InterestedEngineer]

I think I understand your confusions now. You think AI is needed for existing machines doing predictable work. Totally unnecessary. No need for the expensive AI "training" on a task that can be done by a fairly small programing team at a tiny fraction of the cost.

Your other confusion seems to be that the robot, having human form, will be able to do human activities. You might eventually teach it to move the way you want. That is useless without the ability to also think the way you want. That problem is much easier to understand for anyone that has attempted to train untalented people.

and

Can't you program modern robots like this?

'A humanoid robot picking up pipe and connecting two tanks together'
Or
'A humanoid robot on Mars connecting an electric truck to a charging station'

Or are we still at programing X,Y,Z coordinate displacements and only using predefined objects?
 

I think the above answers your question, and it is "yes" for large equipment.   

Autonomous, that will not be.  Assistive to remote control a few seconds away, probably.

Assistive to remote control light minutes away - probably not.

[Coastal Ron]

...
The bottom line though is that it does not appear that we are even close to being there, if the demos at this months CES show are examples of where the industry are at.

The Tesla FSD I had for a month during Christmas was a great improvement.  It did get me to the target with a simple command line, and did all the rest on its own, including at least one mistake a day.  I am curious as to how the upcoming 'Grok' version will be usable, once it is fit to drive the car.  If ever 

Elon Musk has been promising that Full Self Driving (FSD) was almost here since 2018, and he is still promising that it is almost here.

That is not a failure to understand where they truly are at, that is someone that doesn't want to admit that they over promised and don't really understand when they will finally get to what they did promise.

It's great at not hitting pedestrians though.

One of the reasons I bought my Tesla Model Y. And so far I haven't hit anyone... 

I can definitively see it driving safely on Mars and avoiding obstacles, and parking at exact positions.

Remember that when you are driving a Tesla car, that you are in a very structured environment regarding the driving environment (i.e. roads, signage and markings, how level everything is, a general lack of debris in the roadway, etc.). That won't be the situation on Mars, where literally every time you shift your viewpoint, the challenges for driving change. I don't think Tesla FSD is built for that type of environment.

I do feel that the field is improving rapidly...

What is rapidly improving, and how are you measuring that?

For instance, as I stated above, Musk has been promising FSD since 2018, and it still isn't here in the form that he promised. Is that what you think is "rapidly" improving?

...but have no idea how it can be translated to humanoid robots.

All knowledge is good, and I'm sure there will be things that are learned from car navigation that can help humanoid robots, but cars and humanoid robots operate so differently that I'm not sure how much overlap there can be.

If it was possible to get results with simple verbal or text instructions, would the robots be more useful?

This is a great question, and I keep thinking about all the times I've had trouble communicating simple things to other people. Either it was an issue on my end or theirs, but there was a problem.

I think what will absolutely be required is making sure that humanoid robots ask questions, and don't just assume they know the answer. This will need to be done for a while until we build up the knowledge and confidence that we know how to communicate between humans and humanoid robots with a high degree of accuracy and understanding.

Safety will be a big aspect of that too, since you don't want to ask a humanoid robot to do something which results in a misunderstanding that causes a safety issue. How does the humanoid robot understand if being asked to do something will cause a safety issue?

[lamontagne]

...
The bottom line though is that it does not appear that we are even close to being there, if the demos at this months CES show are examples of where the industry are at.

The Tesla FSD I had for a month during Christmas was a great improvement.  It did get me to the target with a simple command line, and did all the rest on its own, including at least one mistake a day.  I am curious as to how the upcoming 'Grok' version will be usable, once it is fit to drive the car.  If ever 

Elon Musk has been promising that Full Self Driving (FSD) was almost here since 2018, and he is still promising that it is almost here.

That is not a failure to understand where they truly are at, that is someone that doesn't want to admit that they over promised and don't really understand when they will finally get to what they did promise.

It's great at not hitting pedestrians though.

One of the reasons I bought my Tesla Model Y. And so far I haven't hit anyone... 

I can definitively see it driving safely on Mars and avoiding obstacles, and parking at exact positions.

Remember that when you are driving a Tesla car, that you are in a very structured environment regarding the driving environment (i.e. roads, signage and markings, how level everything is, a general lack of debris in the roadway, etc.). That won't be the situation on Mars, where literally every time you shift your viewpoint, the challenges for driving change. I don't think Tesla FSD is built for that type of environment.

I do feel that the field is improving rapidly...

What is rapidly improving, and how are you measuring that?

For instance, as I stated above, Musk has been promising FSD since 2018, and it still isn't here in the form that he promised. Is that what you think is "rapidly" improving?

...but have no idea how it can be translated to humanoid robots.

All knowledge is good, and I'm sure there will be things that are learned from car navigation that can help humanoid robots, but cars and humanoid robots operate so differently that I'm not sure how much overlap there can be.

If it was possible to get results with simple verbal or text instructions, would the robots be more useful?

This is a great question, and I keep thinking about all the times I've had trouble communicating simple things to other people. Either it was an issue on my end or theirs, but there was a problem.

I think what will absolutely be required is making sure that humanoid robots ask questions, and don't just assume they know the answer. This will need to be done for a while until we build up the knowledge and confidence that we know how to communicate between humans and humanoid robots with a high degree of accuracy and understanding.

Safety will be a big aspect of that too, since you don't want to ask a humanoid robot to do something which results in a misunderstanding that causes a safety issue. How does the humanoid robot understand if being asked to do something will cause a safety issue?

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level, and then it will not change for centuries, except for what is removed by humans and their tools.  So it should be very easy to drive on Mars, once the first vehicle has passed through.

I found that the newesr version of FSD was way better at catching up after I made a decision, i.e. communicated with the robot.  In older versions, the instant I intervened it gave up.  Now it mostly accepts my suggestions, for example in a parking lot where I know an exit it hasn't seen and I contradict it's turning indicator, it 'obeys' and then carries on.  I found that an impressive progression.  Won't be able to do that on Mars though, although the robot is infinitely patient, so it can easily wait half an hour for an instruction, which is something most humans cannot do.

Regarding the form factor, I think the important thing is to choose one, and then carry on.  Humanoid is fine in many cases, very energy efficient.  Limited joint movement is possibly a plus, as it makes adding flexible dust tight covers possible with no mobile seal. 

As I found out this week with my Tesla though, don't leave it unpluged all night at -30°C, it really doesn't respond well.  -68°C must be something else again...

[redneck]

FSD on cars is dealing with fairly high speeds and other traffic. Equipment on another planet could be quite effective at 5 mph. 5 mph is roughly mower, forklift, and skid steer speeds that can be automated with fairly modest programing. A scout rover at 5 mph and 10 hour days could cover 1,500 miles per month.

 IE suggested assisted remote control with operators within light seconds. The "assisted" could mean that equipment runs for minutes to hours at a time with no remote instruction other than monitoring. The moon would be an ideal place for the first off Earth testing. On Earth there are already machines that work with minimal supervision. I would consider the possibility that the advanced artificial brains Might eventually oversee many specialized machines from a secure location in lieu of human operators.

[Paul451]

Remember that when you are driving a Tesla car, that you are in a very structured environment regarding the driving environment (i.e. roads, signage and markings, how level everything is, a general lack of debris in the roadway, etc.). That won't be the situation on Mars, where literally every time you shift your viewpoint, the challenges for driving change. I don't think Tesla FSD is built for that type of environment.

It's much more than that. For the purpose of this discussion, a Tesla is a "wheeled robot" or "specialised robot", not a "generalist robot" that people imagine humanoid robots to be.

As such, the control inputs are incredibly simple: Left/right, go, stop, and optional fwd/rev toggle. It doesn't change its centre of balance when it moves, it doesn't change its effective size as it moves. It's inherently stable, to the point that you can completely remove control inputs and it just happily stays put (or continues to travel onwards). There's no similarity with the complexity of making a humanoid robot perform even a basic task.

The hard part, and the part they've struggled with for nearly a decade, is understanding the environment around the vehicle. Which suggests that, despite of the enormous volume of training data, the current AI development models are really poor at solving the last 5%. They might have hit a wall, such that each new problem has to be effectively hand-solved.

My problem with that, is that using a humanoid robot makes every task "the last 5%". There's no equivalent (useful/sellable) stepping stones like "electronic cruise-control" to "adaptive/radar cruise-control" to "lane following" to "auto-park"... etc etc...

[Paul451]

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level, and then it will not change for centuries, except for what is removed by humans and their tools.

Meaning pretty much everywhere around the base, mines, etc.

Re: Chat-style commands

I should point out that the difficulties inherent in humanoid robots that I and others are talking about, aren't about giving the robot instructions. But in developing the control systems (the AI) to get it to do those things. Adding ChatGPT/Grok/etc to the robot isn't changing that.

[edit: I changed that from quote. I didn't mean to imply that Lamontage specifically said that, it was a place-holder summary for a bunch of similar comments.]


Off-topic bit...

As I found out this week with my Tesla though, don't leave it unpluged all night at -30°C, it really doesn't respond well.  -68°C must be something else again...

Do they not have a battery heater for cold weather?

[lamontagne]

Remember that when you are driving a Tesla car, that you are in a very structured environment regarding the driving environment (i.e. roads, signage and markings, how level everything is, a general lack of debris in the roadway, etc.). That won't be the situation on Mars, where literally every time you shift your viewpoint, the challenges for driving change. I don't think Tesla FSD is built for that type of environment.

It's much more than that. For the purpose of this discussion, a Tesla is a "wheeled robot" or "specialised robot", not a "generalist robot" that people imagine humanoid robots to be.

As such, the control inputs are incredibly simple: Left/right, go, stop, and optional fwd/rev toggle. It doesn't change its centre of balance when it moves, it doesn't change its effective size as it moves. It's inherently stable, to the point that you can completely remove control inputs and it just happily stays put (or continues to travel onwards). There's no similarity with the complexity of making a humanoid robot perform even a basic task.

The hard part, and the part they've struggled with for nearly a decade, is understanding the environment around the vehicle. Which suggests that, despite of the enormous volume of training data, the current AI development models are really poor at solving the last 5%. They might have hit a wall, such that each new problem has to be hand-solved.

Regarding the environment, Mars seems like a much simpler problem than Earth.  I agree it's hard to get the humanoid robot to work, but it seems to be on the way of being solved.  In a sense, I think it really started beeing solved with the Segway. Now, once it is completely solved, is it a practical solution?  Can programing a working robot be as simple as a AI querry? 
BTW, are two hands better in a significant way than a single arm?

[Paul451]

Regarding the environment, Mars seems like a much simpler problem than Earth.

Not just simpler, but something that will be solved for mining and construction, which are adopting increasing automation.

I agree it's hard to get the humanoid robot to work, but it seems to be on the way of being solved.

That's what I strongly disagree with. There's no sign of generalised solutions. Only grinding out each new task with as much difficulty as each previous one, and with the same "almost-but-not-quite" results.

[Don't get me wrong. As research projects, I'm ridiculously impressed with what they've achieved. It's extraordinary. But for a practical general purpose robot, I don't see it being possible with the current AI development models. There's something missing. So humanoid AI is always going to be harder to develop for a set of tasks than more specialised form-factor robots. Ironically, "specialised" robots end up being more adaptable and generalised than the supposedly "general purpose" humanoid form-factor. The form-factor is what makes tasks harder, not easier.]

Now, once it is completely solved, is it [...]

Which is why I don't think the above is a useful topic to discuss. We're too far away from it mattering.

[lamontagne]

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level, and then it will not change for centuries, except for what is removed by humans and their tools.

Meaning pretty much everywhere around the base, mines, etc.

Re: Chat-style commands

I should point out that the difficulties inherent in humanoid robots that I and others are talking about, aren't about giving the robot instructions. But in developing the control systems (the AI) to get it to do those things. Adding ChatGPT/Grok/etc to the robot isn't changing that.


Off-topic bit...

As I found out this week with my Tesla though, don't leave it unpluged all night at -30°C, it really doesn't respond well.  -68°C must be something else again...

Do they not have a battery heater for cold weather?

Yes, but since the battery was also down to 20%, I think it was a case of one parameter vs another.  Keep warm, or risk going to zero, which I believe is very bad.

What I was trying to show with my visual example above is that AI software seems to already have a pretty good idea about a number of things.  But if it draws a tank when I ask for a tank, does it know a tank when it 'sees' one and can it plug a pipe into it? 

[Twark_Main]

I'm not saying you couldn't do with this with an excavator.  One could produce 10,000 excavators that are autonomously learning from remote operators, I'm just saying the cost to do so is prohibitive.  Where would you find 10,000 qualified remote operators fully aware that this will replace their 6 figure jobs?  To train Optimus you just have to be a curious human unaware of their possible future self-demise.  Plenty of those to go around.

"Surely 100% of atomized race-to-the-bottom labor will strike against automation" is a badly ahistorical rack to hang your hat on.

[Twark_Main]

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level, and then it will not change for centuries, except for what is removed by humans and their tools.

Meaning pretty much everywhere around the base, mines, etc.

If only we had thought to put the robots in the same place on Mars as the humans, then the robots could simply map any changes continuously (likely because a robot is already there helping, but otherwise with drone mapping flights).  Sadly we landed the robots on the other side of the planet for no reason, so there's no solution.   

Edit: or some kind of suit we could force humans to wear any time they're outside (ideally on pain of death) that we could mount a compact LiDAR scanner on. So again, no possible solution.

[Coastal Ron]

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level...

Well, first of all Musk is famous for NOT liking lidar, so we'll have to see if he wants to take any of it to Mars. 

Secondly, you are trying to solve a problem with brute force, requiring highly detailed maps that have to be referenced for the humanoid robot to move anywhere. I don't think that is a workable solution.

Humans normally keep track of their local environment, and then remember details about their pathways, but we don't store every detail, plus we are looking for updates along the way.

...and then it will not change for centuries, except for what is removed by humans and their tools.

Or what is ADDED by humans or other autonomous systems. Because unless you are the only autonomous entity in an area, you can't assume everyone else is static and not moving around or moving items around. The mapping only deals with terrain, not with changes brought by colonization.

Regarding the form factor, I think the important thing is to choose one, and then carry on.  Humanoid is fine in many cases, very energy efficient.

Today I would say that humanoid robots are less efficient than wheeled alternatives. Because not only do you need to worry about ambulation, but balance too. All of that requires far more motors, electronics/computers, and energy for moving versus wheeled systems.

Limited joint movement is possibly a plus, as it makes adding flexible dust tight covers possible with no mobile seal.

For Mars, I think dust is manageable. For the Moon, that is such a harsh environment, especially when you are working in a shadow. Not only the dust issue, which is far worse on the Moon (no air to wear down sharp edges), but the temperature extremes.

[Twark_Main]

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level...

Well, first of all Musk is famous for NOT liking lidar, so we'll have to see if he wants to take any of it to Mars. 

He also famously pushed for LiDAR for the Dragon docking system (DragonEye).

https://twitter.com/elonmusk/status/1329847917834870784

https://twitter.com/elonmusk/status/1960385338699874758

[lamontagne]

It should be fairly simple to lidar/map the landing area and fully characterise the martian surface down to a cm or even mm level...

Well, first of all Musk is famous for NOT liking lidar, so we'll have to see if he wants to take any of it to Mars. 

Secondly, you are trying to solve a problem with brute force, requiring highly detailed maps that have to be referenced for the humanoid robot to move anywhere. I don't think that is a workable solution.

Humans normally keep track of their local environment, and then remember details about their pathways, but we don't store every detail, plus we are looking for updates along the way.

...and then it will not change for centuries, except for what is removed by humans and their tools.

Or what is ADDED by humans or other autonomous systems. Because unless you are the only autonomous entity in an area, you can't assume everyone else is static and not moving around or moving items around. The mapping only deals with terrain, not with changes brought by colonization.

Regarding the form factor, I think the important thing is to choose one, and then carry on.  Humanoid is fine in many cases, very energy efficient.

Today I would say that humanoid robots are less efficient than wheeled alternatives. Because not only do you need to worry about ambulation, but balance too. All of that requires far more motors, electronics/computers, and energy for moving versus wheeled systems.

Limited joint movement is possibly a plus, as it makes adding flexible dust tight covers possible with no mobile seal.

For Mars, I think dust is manageable. For the Moon, that is such a harsh environment, especially when you are working in a shadow. Not only the dust issue, which is far worse on the Moon (no air to wear down sharp edges), but the temperature extremes.

Tesla did not like Lidar as an active control system used in addition to cameras.  This is completely different from using Lidar to set up a Map, similar to Google maps but without the availability of GPS, and way more precise anyway.
Once the Map is set up, then you add movable elements such as vehicles and cargo pallets as they are moved around, and update the map as trenches are added as they are dug using another lidar survey.
Then purely visual data is used to guide the robots/vehicles in the map and the real world at the same time.
The Lidar survey is very useful to set up your trench diging equipment and can be used to create construction plans and civil works quantities, as is commonly done today in civil engineering. 
So the map will be continuously updated, which is a common practive in many cities, and the basis of the Google maps (and indeed, all other) systems.  It's called a digital twin, among other names.

Control of heat on the Moon during the day is an old and well understood science.  During the night, I expect the robots will mosly be in shelters, or wear a simple infra red reflective jacket and pants to control thermal radiation and conserve heat.

AFAIK, abrasives are only dangerous to robots in joints and for optics? Covered joints are imprevious to dust, which is why Optimus seems like a better fit than Atlas.  Optics can be covered with a layer of synthetic saphire.

You don't worry about balance, the software and controls takes care of the balance.  Acceleromers and position actuators do need require much energy.  The Segway, despite its many flaws, did prove that over a decade ago.

[Cheapchips]

Unitree did this 'extreme cold challenge' with one of their G1 robots. Might still need more than a down coat on Mars!

130,000 autonomous steps with precision in snow is impressive, even they leaned into BDS positioning.

[Tywin]

More video of Atlas:

[catdlr]

Atlas Airborne | Boston Dynamics & ‪@rai-inst‬

Feb 7, 2026
Now that the Atlas enterprise platform is getting to work, the research version gets one last run in the sun. Our engineers made one final push to test the limits of full-body control and mobility, with help from the RAI Institute.

https://youtube.com/watch?v=UNorxwlZlFk

[clongton]

Nice. It's necessary for the robots to be at least as flexible as a gymnast if they are going to be truly useful. But those are still just "tricks".

The acid test is developing sufficient self awareness in its software, with latent capability, that one of them can hear a verbal command, understand it, respond in kind and then execute the command flawlessly. Not just programmed to do one or two things extremely well. Yes, it will need to be taught how to do something; just like we all are taught to do things. And it will need to be intelligent enough to innovate when executing a command, recognizing that the environment is different from its training and be able to adjust to the existing environment and still execute the command. The question is how far are we away from that?

Example:
Training: Teach it how to pick up several fragile items, put them in a container that it is carrying, and then go to where they are supposed to be stored and transfer them - intact - to the storage container.
Real world: "Atlas, go to the chicken coop and collect the eggs, and them bring them into the kitchen and put them in the egg carton".

THAT is a useful robot.

[JulesVerneATV]

Cyborgs in Animals, bio-robots, Remote control animals
India seems to be avoiding the use of animals in its space program and with 'Animal Rights' becoming more of an issue in the West there are 'Ethics' concerns

a remote controlled series of moths or rats or beetle that would fly through a shaft or go through a pipe and check out a Mars colony or farm that was in danger or had 'problems'

unfortunately according to some news reports this seems to be for the horrific application of a meat grinder war

Russia developing 'cyborg' creatures that could be used to spread deadly disease, expert warns
https://www.ladbible.com/news/technology/russia-pigeons-spying-brainwashed-disease-615265-20260205