Author Topic: Future Control/Display Interfaces  (Read 2455 times)

Offline sanman

  • Senior Member
  • *****
  • Posts: 4310
  • Liked: 633
  • Likes Given: 8
Future Control/Display Interfaces
« on: 08/19/2018 04:48 AM »
Could there be benefit in one day upgrading control and viewing interfaces in the spacecraft cockpit?

What about the idea of an interactive holographic interface?

[EDIT: The following is not a holographic image - it's a 3D volumetric image]







https://www.seeker.com/gadgets/new-technique-generates-free-floating-3d-images-just-dont-call-it-a-hologram

https://www.nature.com/articles/s41598-018-20454-6



Instead of being seated in front of a panel with screens, the interfaces could be projected in the space in front of the pilot via laser holography, and could be made interactive via a reconfigurable grid of light beams.

This could enable cabin space to be freed up, by not having a large physical dashboard panel in front of the crew/occupants.

The display/control interface could be flexibly and dynamically reconfigurable to different dimensions, based on the particular need/application.

The display/control interface hardware could be made more miniaturized, compact and rugged against shock/vibration/stresses associated with liftoff & re-entry or even depressurization, due to its virtually projected nature, as opposed to being a large physical display.

There could be more backup miniaturized hardware available for redundancy, in case the main interface broke down - as opposed to carrying extra physical monitor screens for redundancy.

Virtual projected screens can't be damaged the way physical monitor screens can by bumping into them, etc - nor can they bang into you and cause you injury.


What is the most effective and ergonomic control/display interface technology candidate available on the horizon or near/medium term, to upgrade to?
« Last Edit: 08/19/2018 08:20 AM by sanman »

Offline tyrred

  • Full Member
  • *
  • Posts: 183
  • Seattle
  • Liked: 100
  • Likes Given: 431
Re: Future Control/Display Interfaces
« Reply #1 on: 08/19/2018 05:33 AM »
Very interesting video.  Not a hologram.  I wonder, what is the particle being trapped and dragged?

Online nacnud

  • Extreme Veteran
  • Senior Member
  • *****
  • Posts: 2290
  • Liked: 446
  • Likes Given: 211
Re: Future Control/Display Interfaces
« Reply #2 on: 08/19/2018 05:51 AM »
It's a bit of cellulose dust.

Offline sanman

  • Senior Member
  • *****
  • Posts: 4310
  • Liked: 633
  • Likes Given: 8
Re: Future Control/Display Interfaces
« Reply #3 on: 08/19/2018 06:41 AM »
According to the linked article above, it's a point of light made by ionizing the air at that spot:

Quote
Using femtosecond lasers (a femtosecond is a quadrillionth of a second, and the lasers transmit bursts that last 30 to 270 femtoseconds), the team can make holograms that are safe to touch. The images are three-dimensional, with resolutions up to 200,000 dots per second. The voxels are light emitted by plasma that's created when the laser's focused energy ionizes the air.

They currently seem to be moving that ionization around at high speed, like a 3D raster beam. Their hardware setup seems to be what's limiting the size.
If they could enlarge it and achieve enough ionized plasma points, maybe they could do a large-resolution voxel image display (voxel is a pixel defined for 3 dimensions rather than 2)

All of this would require a pressurized cabin of course, to supply the air that's being ionized - without that, the display won't work.
Is that a liability in a manned spacecraft?


https://digitalnature.slis.tsukuba.ac.jp/2015/06/fairy-lights-in-femtoseconds/

Quote
Abstract :
We present a method of rendering aerial and volumetric graphics using femtosecond lasers. A high-intensity laser excites a physical matter to emit light at an arbitrary 3D position. Popular applications can then be explored especially since plasma induced by a femtosecond laser is safer than that generated by a nanosecond laser. There are two methods of rendering graphics with a femtosecond laser in air: Producing holograms using spatial light modulation technology, and scanning of a laser beam by a galvano mirror. The holograms and workspace of the system proposed here occupy a volume of up to 1 cm^3; however, this size is scalable depending on the optical devices and their setup. This paper provides details of the principles, system setup, and experimental evaluation, and discussions on scalability, design space, and applications of this system. We tested two laser sources: an adjustable (30-100 fs) laser which projects up to 1,000 pulses per second at energy up to 7 mJ per pulse, and a 269-fs laser which projects up to 200,000 pulses per second at an energy up to 50 uJ per pulse. We confirmed that the spatiotemporal resolution of volumetric displays, implemented with these laser sources, is 4,000 and 200,000 dots per second. Although we focus on laser-induced plasma in air, the discussion presented here is also applicable to other rendering principles such as fluorescence and microbubble in solid/liquid materials.

« Last Edit: 08/19/2018 07:04 AM by sanman »

Online nacnud

  • Extreme Veteran
  • Senior Member
  • *****
  • Posts: 2290
  • Liked: 446
  • Likes Given: 211
Re: Future Control/Display Interfaces
« Reply #4 on: 08/19/2018 07:18 AM »
According to the linked article above, it's a point of light made by ionizing the air at that spot:

Your first links says the light is projected onto paper dust, cellulose, in an optical trap not ionised air emitting light.

From seeker.com

Quote
The system can generate different optical effects, colors and images by using different kinds of materials for the projection particles.

“They could be just about anything,” Smalley said.  “Glass beads, diamonds, cellulose, tungsten — a wide variety of materials. What we've found most effective is a substance called black liquor, which is a byproduct of the paper manufacturing process. It's essentially just paper, cellulose."

Offline sanman

  • Senior Member
  • *****
  • Posts: 4310
  • Liked: 633
  • Likes Given: 8
Re: Future Control/Display Interfaces
« Reply #5 on: 08/19/2018 07:55 AM »
According to the linked article above, it's a point of light made by ionizing the air at that spot:

Your first links says the light is projected onto paper dust, cellulose, in an optical trap not ionised air emitting light.

From seeker.com

Quote
The system can generate different optical effects, colors and images by using different kinds of materials for the projection particles.

“They could be just about anything,” Smalley said.  “Glass beads, diamonds, cellulose, tungsten — a wide variety of materials. What we've found most effective is a substance called black liquor, which is a byproduct of the paper manufacturing process. It's essentially just paper, cellulose."


Dammit, you're right - I guess I was quoting from a different article on a different technology that I also wanted to post:

https://www.popsci.com/secret-interactive-holograms-plasma-and-femtosecond-laser

So that one is ionized air using femtosecond lasers. This one can be made interactive, rather than just being a display - notice how the heart icon turns into a broken-heart icon when the beam is broken. But clearly it needs air for the display portion to work. The air is necessary as a display medium.






« Last Edit: 08/19/2018 08:13 AM by sanman »

Offline TripleSeven

  • Full Member
  • ****
  • Posts: 581
  • Istanbul turkey
  • Liked: 247
  • Likes Given: 1013
Re: Future Control/Display Interfaces
« Reply #6 on: 08/19/2018 08:37 AM »
3D display systems are the holy grail of command and control...whoever comes up with one that 1) works and 2) is practical...changes things in a big way

Offline Hobbes-22

  • Full Member
  • ****
  • Posts: 419
  • Acme Engineering
    • Acme Engineering
  • Liked: 140
  • Likes Given: 89
Re: Future Control/Display Interfaces
« Reply #7 on: 08/19/2018 09:00 AM »
In the automotive industry, there's a trend moving away from physical controls (single-function knobs and buttons) and towards touch screens.
While this enables more flexibility, it has a big drawback: it makes it more difficult to operate the controls blindly. Controls in a car fall into 2 basic groups:
1. controls that have to be usable while driving (lights, wipers, AC, audio)
2. controls that are acceptable if they are only operable while stationary (setting user preferences, satnav etc)

For spacecraft something similar holds. For operating a robotic arm, you need dedicated physical controls because you can't afford to hunt for them. The same goes for maneuvering the spacecraft, emergency operations etc.

Online speedevil

  • Senior Member
  • *****
  • Posts: 2531
  • Fife
  • Liked: 1272
  • Likes Given: 1393
Re: Future Control/Display Interfaces
« Reply #8 on: 08/19/2018 09:28 AM »
Goggles are several orders of magnitude closer to reality, and can actually be readily purchased.

For spacecraft controls, I don't see the point.
We are not going to - probably ever - see such a high information load that very simple displays aren't quite adequate for all spaceflight related matters.

This is due to a combination of low spacecraft performance, avoidance of error, and time to work things out.

Flying 'seat of pants', using information rich displays, when your current smartphone can plot the trajectory much more accurately and reliably than you can, is just not going to happen.
This is not a rapid high speed dodging firefight, it's a case where to get optimal performance out, you need optimal inputs, in precise directions at precise times, in ways humans are very bad at intuiting.

For the case where information rich displays are useful, 3d goggles built into the spacesuit are quite adequate.

Offline TripleSeven

  • Full Member
  • ****
  • Posts: 581
  • Istanbul turkey
  • Liked: 247
  • Likes Given: 1013
Re: Future Control/Display Interfaces
« Reply #9 on: 08/19/2018 09:49 AM »
In the automotive industry, there's a trend moving away from physical controls (single-function knobs and buttons) and towards touch screens.
While this enables more flexibility, it has a big drawback: it makes it more difficult to operate the controls blindly. Controls in a car fall into 2 basic groups:
1. controls that have to be usable while driving (lights, wipers, AC, audio)
2. controls that are acceptable if they are only operable while stationary (setting user preferences, satnav etc)

For spacecraft something similar holds. For operating a robotic arm, you need dedicated physical controls because you can't afford to hunt for them. The same goes for maneuvering the spacecraft, emergency operations etc.

its unclear to me where this is going...I've watched and been a part of the transition in airplanes (and to some extent the bridge of a ship) from purely mechanical controls and displays to heavy "screen" displays ...there is a great line from Clarke...I think its from 2010 the book but he is talking about the controls system on the Russian Lenov...and he talks about how "even in this day and age" (paraphrase) "a satisfying click" of  a mechanical switch

but clearly the trend is toward digital controls...

my old employer hasnot quite yet "digitalized" the Auto pilot flight director controls (the AFDS) but its probably coming :)

Online nacnud

  • Extreme Veteran
  • Senior Member
  • *****
  • Posts: 2290
  • Liked: 446
  • Likes Given: 211
Re: Future Control/Display Interfaces
« Reply #10 on: 08/19/2018 09:54 AM »
I really don't see any advantage in these displays over current tech for the purpose of spacecraft. Interfaces with those display though...

Quote
CEO and Co-founder of CTRL-labs, Thomas Reardon, on stage for the Keynote at the O'Reilly Artificial Intelligence conference in NYC, shares a bold vision for the future of human-computer interaction and how the company uses non-invasive neural interfaces to unlock human potential.

« Last Edit: 08/19/2018 09:55 AM by nacnud »

Offline IRobot

  • Full Member
  • ****
  • Posts: 1280
  • Portugal & Germany
  • Liked: 277
  • Likes Given: 248
Re: Future Control/Display Interfaces
« Reply #11 on: 08/19/2018 07:34 PM »
This could enable cabin space to be freed up, by not having a large physical dashboard panel in front of the crew/occupants.

The display/control interface could be flexibly and dynamically reconfigurable to different dimensions, based on the particular need/application.

The display/control interface hardware could be made more miniaturized, compact and rugged against shock/vibration/stresses associated with liftoff & re-entry or even depressurization, due to its virtually projected nature, as opposed to being a large physical display.

There could be more backup miniaturized hardware available for redundancy, in case the main interface broke down - as opposed to carrying extra physical monitor screens for redundancy.

Virtual projected screens can't be damaged the way physical monitor screens can by bumping into them, etc - nor can they bang into you and cause you injury.
If you want to remove the panels, you can just make astros use some sort of Microsoft Hololens.
https://www.microsoft.com/en-us/hololens

This would even help on an EVA. Or as context-aware helper. You see a few physical switches, the glasses overlap useful tips on each button.

EDIT: I have a MS Hololens on my desk right now, will play around with it during the afternoon :)
« Last Edit: 08/20/2018 07:26 AM by IRobot »

Offline CameronD

  • Full Member
  • ****
  • Posts: 1401
  • Melbourne, Australia
    • Norton Consultants
  • Liked: 450
  • Likes Given: 334
Re: Future Control/Display Interfaces
« Reply #12 on: 08/19/2018 11:05 PM »
my old employer hasnot quite yet "digitalized" the Auto pilot flight director controls (the AFDS) but its probably coming :)

Certainly "Augmented Reality" (using 3D glasses) is being seriously looked into by Boeing (for pilots) and Thales (for ATC).  Coming soon to a cockpit near you. :)

I've tried some of it and it works really well.  Being able to walk around and through a live 3D air-space is amazing!
With sufficient thrust, pigs fly just fine - however, this is not necessarily a good idea. It is hard to be sure where they are
going to land, and it could be dangerous sitting under them as they fly overhead.

Offline Asteroza

  • Full Member
  • ****
  • Posts: 624
  • Liked: 80
  • Likes Given: 2
Re: Future Control/Display Interfaces
« Reply #13 on: 08/20/2018 12:14 AM »
I vaguely remember a concept for light thruster controls where a ball floats in a cube space strongly pinned by magnetic forces, and a person grips the ball. Sensors detect the ball moving in 6 axis movement, and interpreting that as commands for the thrusters. This was intended for close range manual docking/berthing work.

Offline Hobbes-22

  • Full Member
  • ****
  • Posts: 419
  • Acme Engineering
    • Acme Engineering
  • Liked: 140
  • Likes Given: 89
Re: Future Control/Display Interfaces
« Reply #14 on: 08/20/2018 08:30 AM »
Quote
I vaguely remember a concept for light thruster controls where a ball floats in a cube space strongly pinned by magnetic forces, and a person grips the ball. Sensors detect the ball moving in 6 axis movement, and interpreting that as commands for the thrusters. This was intended for close range manual docking/berthing work.

Sounds like a more expensive version of the 3D controllers currently in use in the CAD/3D modelling industry.

Offline IRobot

  • Full Member
  • ****
  • Posts: 1280
  • Portugal & Germany
  • Liked: 277
  • Likes Given: 248
Re: Future Control/Display Interfaces
« Reply #15 on: 08/20/2018 10:41 AM »
Quote
I vaguely remember a concept for light thruster controls where a ball floats in a cube space strongly pinned by magnetic forces, and a person grips the ball. Sensors detect the ball moving in 6 axis movement, and interpreting that as commands for the thrusters. This was intended for close range manual docking/berthing work.

Sounds like a more expensive version of the 3D controllers currently in use in the CAD/3D modelling industry.
I have one of those at home. Besides using it for work, it also works extremely well with Kerbal Space Program and especially for manual docking! It requires some hours of training, it is not straightforward.

Offline Rik ISS-fan

  • Full Member
  • ****
  • Posts: 894
  • the Netherlands
  • Liked: 209
  • Likes Given: 108
Re: Future Control/Display Interfaces
« Reply #16 on: 08/23/2018 11:12 PM »
Just two links:
http://www.esa-telerobotics.net/research
https://www.youtube.com/user/TeleroboticsLab/videos
While NASA launched the Robonaut to the ISS, ESA tried to developed for only a fraction of the cost utilizable haptic robotics. Add a hollow lens and you don't require EVA's anymore. That practice of doing EVA's should stop the next decade. By default haptice external activities from the ground. When delay get's problematic; a astronaut takes control with a wired connection. (hardly any delay => direct action + force feedback)

Edit 3th link: https://meteron.dlr.de/
« Last Edit: 08/23/2018 11:25 PM by Rik ISS-fan »

Offline laszlo

  • Full Member
  • *
  • Posts: 182
  • Liked: 194
  • Likes Given: 39
Re: Future Control/Display Interfaces
« Reply #17 on: 08/23/2018 11:43 PM »
As Hobbes-22 points out, you can't operate the controls blindly. Additionally, in high-g environment, getting your hands up to the controls could be problematic. Similar thing in a dynamic vibrating environment. The see-through aspect of these controls can also cause problems. Is that dot you're looking at part of the control or something that drifted out of somewhere and stuck to the wall behind the projection area?

I see these kinds of controls as disasters waiting to happen. They may look cool in sci-fi, they may work in stationary installations, but they're just trouble waiting to happen in a moving vehicle.

Online nacnud

  • Extreme Veteran
  • Senior Member
  • *****
  • Posts: 2290
  • Liked: 446
  • Likes Given: 211
Re: Future Control/Display Interfaces
« Reply #18 on: 08/24/2018 12:39 AM »
As Hobbes-22 points out, you can't operate the controls blindly. Additionally, in high-g environment, getting your hands up to the controls could be problematic. Similar thing in a dynamic vibrating environment. The see-through aspect of these controls can also cause problems. Is that dot you're looking at part of the control or something that drifted out of somewhere and stuck to the wall behind the projection area?

I see these kinds of controls as disasters waiting to happen. They may look cool in sci-fi, they may work in stationary installations, but they're just trouble waiting to happen in a moving vehicle.

Watch the vid I posted above, you can control a screen, robot or whatever, with just the intention of moving your hand. Actually moving the hand is unnecessary. Mind you the high-g environment you are referring to is very rare, I only know of one instance which came close.

Offline CameronD

  • Full Member
  • ****
  • Posts: 1401
  • Melbourne, Australia
    • Norton Consultants
  • Liked: 450
  • Likes Given: 334
Re: Future Control/Display Interfaces
« Reply #19 on: 08/24/2018 12:46 AM »
As Hobbes-22 points out, you can't operate the controls blindly. Additionally, in high-g environment, getting your hands up to the controls could be problematic. Similar thing in a dynamic vibrating environment. The see-through aspect of these controls can also cause problems. Is that dot you're looking at part of the control or something that drifted out of somewhere and stuck to the wall behind the projection area?

I see these kinds of controls as disasters waiting to happen. They may look cool in sci-fi, they may work in stationary installations, but they're just trouble waiting to happen in a moving vehicle.

Watch the vid I posted above, you can control a screen, robot or whatever, with just the intention of moving your hand. Actually moving the hand is unnecessary. Mind you the high-g environment you are referring to is very rare, I only know of one instance which came close.

Well, rocket launch is one.  The Russians currently use a stick. :)

SpaceX are using them so maybe they have a solution that isn't immediately clear, but that's one problem with the use of touchscreens (in both aviation and spaceflight).  In a high-g/high vibration environment, a physical switch you can position your fingers around and forcibly toggle is just so much easier to do than stopping your fingers skidding across a touchscreen.

And in this environment, I'd think tactile feedback is essential.
 
« Last Edit: 08/24/2018 12:50 AM by CameronD »
With sufficient thrust, pigs fly just fine - however, this is not necessarily a good idea. It is hard to be sure where they are
going to land, and it could be dangerous sitting under them as they fly overhead.

Tags: