Humanoid Robots for Moon and Mars

[MickQ]

Try having the robot consistently hit a baseball pitched by a human.

[Twark_Main]

Tesla aren't the first, since we've seen Unitree, Boston etc do similarly complex full body motion. What is significant is the 'how it's done' of all these recent demos.

You can't just map full body mocap of any action and then have a robot perform it.  Even with inverse kinematics, physics takes over and you end up with heap of robot on the floor.

With reinforcement learning the mocap or animation data is fed into a simulation where the simulated robot tries to perform that action hundred of millions of times.  Natural selection from failure eventually creates a policy which can execute the moves successfully across a range of scenarios, including the natural variability in a robots own actuation. It's not an animation playback.

Tesla and others are currently keen to point out that you RL train the action, drop the policy into the robot and it just works.

The great white hope is that this same methodology works for complex full body manipulation. The real killer demo is going to be a robot picking something up in one hand and opening a door with the other.

I'd imagine a major killer demo could be in robot athletic sports.

Set up a Robot Sports League that makes use of standardized robots which they supply, and then have competing sports teams drop their respective policies into them, to compete on that basis. And since AI's can support multiple policies simultaneously coexisting together, then the Robot Sports League could also supply the policies for adhering to the rules of a given particular sport.

Teams could then compete for recognition, prize money, sponsorships, etc among their various incentives, just as happens with classical professional sports.

https://en.wikipedia.org/wiki/RoboCup

https://en.wikipedia.org/wiki/Soccer_robot

[BN]

Try having the robot consistently hit a baseball pitched by a human.

robots have been able to do this for 20 years

[Twark_Main]

Try having the robot consistently hit a baseball pitched by a human.

robots have been able to do this for 20 years

Hard to believe it's been so long.  This popular demo video was released 15 years ago now. 




Timeline of the lab's work:

[MickQ]

I’d like to see an Optimus with the bat standing up to a top league pitcher.

[sanman]

I’d like to see an Optimus with the bat standing up to a top league pitcher.

And/or an Optimus pitching to top batters?

And ultimately even competing against itself by playing all positions?

[Twark_Main]

https://en.wikipedia.org/wiki/Baseball_robot

Try having the robot consistently hit a baseball pitched by a human.

I’d like to see an Optimus with the bat standing up to a top league pitcher.

Love how in the span of merely ~40 hours the goalpost moved from "A human" to "the best human on Earth for the skill."   

This shows how we should not rely on how "impressed" people are as a judge of progress in AI and humanoid robots.

[sanman]

https://en.wikipedia.org/wiki/Baseball_robot

Love how in the span of merely ~40 hours the goalpost moved from "A human" to "the best human on Earth for the skill."   

This shows how we should not rely on how "impressed" people are as a judge of progress in AI and humanoid robots.

Would it really take that long? You've just been shown Tesla's Optimus dancing.
How long might it take for Optimus to out-perform human dance performers?
Less than a decade? Or more?

[Coastal Ron]

https://en.wikipedia.org/wiki/Baseball_robot

Love how in the span of merely ~40 hours the goalpost moved from "A human" to "the best human on Earth for the skill."   

This shows how we should not rely on how "impressed" people are as a judge of progress in AI and humanoid robots.

Would it really take that long? You've just been shown Tesla's Optimus dancing.
How long might it take for Optimus to out-perform human dance performers?
Less than a decade? Or more?

Depends on the energy density of the batteries Optimus has to carry. Humans are incredibly energy efficient, not so for humanoid robots - at least not yet.

But dancing or swinging a bat at a ball are NOT the skills needed for operations on our Moon or Mars. They may be indicators of overall abilities, but the abilities needed for autonomous roaming in a hostile environment is something no humanoid robot has demonstrated yet here on Earth.

We've talked about the temperatures that a humanoid robot would need to survive, what about the untethered exploration radius needed? Or the payload that they need to carry, such as tools or sensitive sensors? What else would be needed?

[Cheapchips]

https://en.wikipedia.org/wiki/Baseball_robot

Love how in the span of merely ~40 hours the goalpost moved from "A human" to "the best human on Earth for the skill."   

This shows how we should not rely on how "impressed" people are as a judge of progress in AI and humanoid robots.

Would it really take that long? You've just been shown Tesla's Optimus dancing.
How long might it take for Optimus to out-perform human dance performers?
Less than a decade? Or more?

Depends on the energy density of the batteries Optimus has to carry. Humans are incredibly energy efficient, not so for humanoid robots - at least not yet.

But dancing or swinging a bat at a ball are NOT the skills needed for operations on our Moon or Mars. They may be indicators of overall abilities, but the abilities needed for autonomous roaming in a hostile environment is something no humanoid robot has demonstrated yet here on Earth.

We've talked about the temperatures that a humanoid robot would need to survive, what about the untethered exploration radius needed? Or the payload that they need to carry, such as tools or sensitive sensors? What else would be needed?

They don't need to stray far from whatever long range transport took them to the worksite or point of scientific interest.  That carries the tools, instruments and charging station.

Mapping and following a route through an environment is something robots do already on Earth and beyond.  It can have the same mix of autonomy and human planning.

The only novel bit is doing it on two legs.  Bipedal locomotion is solved on mild terrain and the dance demos show the huge potential of RL to drive full body movement.  Traversal of rough terrain that's out of reach for rovers could be possible soon, given how things are headed.  I'd never have thought that a few years ago.

Some of the current research on RL driven robots is hardware independent too.  You might not even need to retrain policies for a moon/mars hardened version of your commodity humanoid. 

Using tools and instruments that require the application of physical force, like a drill, are the biggest gap that software needed to bridge.

[sanman]

Depends on the energy density of the batteries Optimus has to carry. Humans are incredibly energy efficient, not so for humanoid robots - at least not yet.

But dancing or swinging a bat at a ball are NOT the skills needed for operations on our Moon or Mars. They may be indicators of overall abilities, but the abilities needed for autonomous roaming in a hostile environment is something no humanoid robot has demonstrated yet here on Earth.

We've talked about the temperatures that a humanoid robot would need to survive, what about the untethered exploration radius needed? Or the payload that they need to carry, such as tools or sensitive sensors? What else would be needed?

Dancing, acrobatics, sports, athletics, etc are over-the-top levels of performance which can be used to push the performance envelope on these robots and stress-test them, so that they can exceed the performance required for their primary intended activities on the Moon and Mars.

We've talked about setting up recharging stations on the lunar surface, including sand batteries which can store solar-thermal energy collected during the daytime.
Your untethered operational range would be through proximity to these charging stations, with their layout arranged accordingly.

So imagine a bunch of Optimus robots performing various duties on the Moon, including perhaps building and maintaining landing pads, keeping them debris-free, etc.

Then once in awahle when they're not on duty, they can stage an exhibition baseball game on the Moon. This could then evolve into a Lunar Baseball League, whereby different teams compete against each other across various matches, by uploading their particular custom-trained  AI into the same set of robots.
Lunar travel/transport may be expensive, but the cost of uploading is much more manageable.

[Coastal Ron]

Depends on the energy density of the batteries Optimus has to carry. Humans are incredibly energy efficient, not so for humanoid robots - at least not yet.

But dancing or swinging a bat at a ball are NOT the skills needed for operations on our Moon or Mars. They may be indicators of overall abilities, but the abilities needed for autonomous roaming in a hostile environment is something no humanoid robot has demonstrated yet here on Earth.

We've talked about the temperatures that a humanoid robot would need to survive, what about the untethered exploration radius needed? Or the payload that they need to carry, such as tools or sensitive sensors? What else would be needed?

Dancing, acrobatics, sports, athletics, etc are over-the-top levels of performance which can be used to push the performance envelope on these robots and stress-test them, so that they can exceed the performance required for their primary intended activities on the Moon and Mars.

Mars and our Moon have far less gravity than Earth, so anything built for Earth will be over-designed for Mars or our Moon.

So imagine a bunch of Optimus robots performing various duties on the Moon, including perhaps building and maintaining landing pads, keeping them debris-free, etc.

I'm not sure a full-time use case has been identified for humanoid robots for Mars or our Moon...

[sanman]

Mars and our Moon have far less gravity than Earth, so anything built for Earth will be over-designed for Mars or our Moon.

Is overbuilt so bad, if it means greater durability/longevity for that hardware on the Moon/Mars?
Durability/longevity means less frequent replacement, less frequent transport of replacement units or their components between Earth and Moon/Mars.

I'm not sure a full-time use case has been identified for humanoid robots for Mars or our Moon...

Well, as discussed before in this thread, it was pointed out that these types of robots could be versatile multi-purpose jack-of-all-trades. We'd only need to design and bring in the more specialized-looking robots for the more frequent/heavy-duty tasks tasks where higher efficiency & throughput are required. There can be greater ease of training for the humanoid bots, since we can use human activity examples for the training data.

[Coastal Ron]

Mars and our Moon have far less gravity than Earth, so anything built for Earth will be over-designed for Mars or our Moon.

Is overbuilt so bad, if it means greater durability/longevity for that hardware on the Moon/Mars?

What overbuilt means is that money is being wasted, and resources are being wasted. Weight is an issue too, since if your Earth-version Optimus weighs 2X of the Mars version, then you can only carry 1/2 as many to Mars (or something else is traded off the ship).

Durability/longevity means less frequent replacement, less frequent transport of replacement units or their components between Earth and Moon/Mars.

Not talking about durability, talking about weight and power requirements.

I'm not sure a full-time use case has been identified for humanoid robots for Mars or our Moon...

Well, as discussed before in this thread, it was pointed out that these types of robots could be versatile multi-purpose jack-of-all-trades.

An answer like that means that, no, there isn't a specific use case identified, but it is HOPED that one will be identified at some point. And if you can't demonstrate the ability here on Earth, then you likely won't be able to perform it on Mars.

[edzieba]

Mars and our Moon have far less gravity than Earth, so anything built for Earth will be over-designed for Mars or our Moon.

That depends on whether design constrains are primarily from stationary G-loads, or whether they are from inertia loads in motion, which will remain the same regardless of local gravity. 100kg is 100kg regardless of local gravity (i.e. mass is not weight).

[BN]

Mars and our Moon have far less gravity than Earth, so anything built for Earth will be over-designed for Mars or our Moon.

That depends on whether design constrains are primarily from stationary G-loads, or whether they are from inertia loads in motion, which will remain the same regardless of local gravity. 100kg is 100kg regardless of local gravity (i.e. mass is not weight).

this is true, however these robots should be training millions of hours in simulation at mars/moon gravity. furthermore, every novel action taken, such as specific interactions with delivered hardware, should be run many times in such a local-g simulator on earth first. and I mean virtual dojo, not a tether, etc.

[sanman]

An answer like that means that, no, there isn't a specific use case identified, but it is HOPED that one will be identified at some point. And if you can't demonstrate the ability here on Earth, then you likely won't be able to perform it on Mars.

But Ron, sometimes you want your robot's main feature to be versaility, because you can't just cover the entire span of your operational needs with only robots built for specialized use cases.
As someone pointed out upthread, it's when some particular task/case becomes sufficiently important/frequent/prevalent that this can spawn a need for a more specialized robot rather than getting by with the more generic versatile ones.

[clongton]

I'm waiting for the day when these humanoid robots will be capable of understanding and executing spoken commands.

[Twark_Main]

https://en.wikipedia.org/wiki/Baseball_robot

Love how in the span of merely ~40 hours the goalpost moved from "A human" to "the best human on Earth for the skill."   

This shows how we should not rely on how "impressed" people are as a judge of progress in AI and humanoid robots.

Would it really take that long? You've just been shown Tesla's Optimus dancing.
How long might it take for Optimus to out-perform human dance performers?
Less than a decade? Or more?

Did you mean to reply to someone else?

When you say "that long," what do you refer to?

[Paul451]

Traversal of rough terrain that's out of reach for rovers could be possible soon, given how things are headed.

Are there any examples of bipedal robots can access terrain that wheeled vehicles can't? Even early demo level, even tethered?

Ie, not staged indoor demos with boxes, seesaws, etc, but natural outdoor terrain (ideally with sand/rock) that a wheeled vehicle (let's say a quad-bike or side-by-side for comparison) couldn't cross, but a human can? Doesn't need to be elegant, hell it can fall over, but just something that shows it's there.

There seems to be an attitude on this thread that it's so obvious that bipedal robots can handle rougher terrain than wheeled bots, that it doesn't even need to be (and shouldn't be) questioned, but I can't see that it's justified.

They might get there, I'm not saying it's impossible, but I don't think it's so obvious that people should be dismissing any suggestion otherwise.


Likewise:

But Ron, sometimes you want your robot's main feature to be versaility, because you can't just cover the entire span of your operational needs with only robots built for specialized use cases.
As someone pointed out upthread, it's when some particular task/case becomes sufficiently important/frequent/prevalent that this can spawn a need for a more specialized robot rather than getting by with the more generic versatile ones.

Note the deeply baked-in assumption that a wheeled robot is not / cannot be versatile?