Daily Ultra-HDR images from Perseverance Rover

[Holger Isenberg]

The daily Perseverance Rover calibrated color images on https://areo.info/mars20 are now shown in Ultra HDR!

Since sol 1277 my site is publishing them now in this new format. Automatically works with Chrome, Opera, Edge and Brave on Macbooks and iMacs since M1 / 2020 and recent Android phones. On Windows only if an HDR-capable display is attached.

Now the full 12 bits/channel camera data can be seen for the first time on your web browser! Below a try to demonstrate the difference in a screenshot photo comparing standard JPEG (SDR) on the left and new Ultra-HDR on the right. But in reality it's much better, you have to see it yourself!

This is as far I know the first website showing the full capability of the 3 Navcam / Hazcam pairs on because without HDR on the display software and hardware side it was impossible to experience the full dynamic range the 12-bits per RGB color channel the original raw camera data provides. The other color cameras on board, Watson and Mastcam-Z have 11-bits/channel, Ingenuity Helicopter only transmitted in 8 bit.

The 3rd image below is one of the UltraHDR ones. It will only show the HDR effect when clicking on it to view the full size. On other browsers / hardware it will be shown as normal image without the additional extreme high contrast effect you get in a HDR capable system.

[Spiceman]

Pretty awesome !

[Holger Isenberg]

It's preparation for further HDR images from space. For example Artemis where Nikon Z9 cameras will be used, which are currently tested on the ISS. And those produce as raw format 14 bits/channel data, far more than any standard dynamic range display system where 8 bits are standard can show. For example with HDR, the lunar surface and stars are possible to show at the same time without overexposing the surface.

[Holger Isenberg]

Here a best-of selection from the recent 12 months and where I already converted the images into HDR from my https://areo.info/mars20 which shows current color images from Mars Perseverance Rover. Daily automatically added and now in HDR viewable on Chrome, Edge, Opera or Brave browser on Macbooks or Android phones. On other hardware it will show standard dynamic range images.

List of sols with best images, now converted to HDR:

 * https://areo.info/mars20/ecams/0954 clear morning sky
 * https://areo.info/mars20/ecams/0961 afternoon clouds
 * https://areo.info/mars20/ecams/0989
 * https://areo.info/mars20/ecams/1069 same scene in clear weather
 * https://areo.info/mars20/ecams/1085
 * https://areo.info/mars20/ecams/1109
 * https://areo.info/mars20/ecams/1162 white rock
 * https://areo.info/mars20/ecams/1207 valley
 * https://areo.info/mars20/ecams/1218 Watson panorama tiles
 * https://areo.info/mars20/ecams/1223
 * https://areo.info/mars20/ecams/1227
 * https://areo.info/mars20/ecams/1246
 * https://areo.info/mars20/ecams/1277 watson green crystals closeup
 * https://areo.info/mars20/ecams/1288 rover climbing track
 * https://areo.info/mars20/ecams/1301 boulder covered hilltop

[Holger Isenberg]

The free areoHDR app is now available for iPhone and iPad in the Apple App Store! It displays the same Mars color images already available on https://areo.info/mars20, but now in HDR on iPhone since X and iPads Pro since 2018 and iPad Air since M1. It can also be run on Macs since M1. The current version is free, just limited currently to the recent 200 sols: https://apps.apple.com/us/app/areohdr/id6738591240

Some images are a bit off in color in HDR mode, too much red, but will be improved soon. You can toggle the HDR mode in the app settings page (upper left button). The upper right info icon shows the HDR capability of your device, the higher number above 1.0 the better.

[deadman1204]

The free areoHDR app is now available for iPhone and iPad in the Apple App Store! It displays the same Mars color images already available on https://areo.info/mars20, but now in HDR on iPhone since X and iPads Pro since 2018 and iPad Air since M1. It can also be run on Macs since M1. The current version is free, just limited currently to the recent 200 sols: https://apps.apple.com/us/app/areohdr/id6738591240

Some images are a bit off in color in HDR mode, too much red, but will be improved soon. You can toggle the HDR mode in the app settings page (upper left button). The upper right info icon shows the HDR capability of your device, the higher number above 1.0 the better.

i assume its upscaled hd?

[Holger Isenberg]

i assume its upscaled hd?

The terms are indeed a bit confusing. HD (high definition) is about number of pixels while HDR (high dynamic range) is about the range of light intensity differences between darkest and brightest part in the image.

The resolution is for some of the images 20 MP (5120 × 3840 pixels), some are only 1.2 MP (1280 x 960) as the camera is switched to different modes depending on the needs. You can zoom in by twice tapping on the image or pinch with two fingers.

The dynamic range by most of the rover cameras data is 12 bits i.e. digital intensity numbers 0 to 4095, that's 4 bits or 16 times more than the 8 bits (0 to 255) normal dynamic range of images on computer screens today. But with modern screens like on recent Macbooks, iphones, iPads and also Android phones those support 10 bits already and have some tricks with the dynamic background light to increase it even further. And that is now possible to see for Mars images with my App.

[edzieba]

It's worth noting that 'bit depth' and 'dynamic range' are NOT the same thing - bit depth measures the range of numeric values a pixel can be recorded as, whereas dynamic range measures the upper and lower bounds of light levels that that number could indicate, often measured either in stops (logarithmic photographic unit of exposure) or dB (ratio of brightest to darkest value). You can have a high bit depth image with a very narrow dynamic range, or a low bit depth image with a very wide dynamic range (but with noticeable 'banding' artefacts). "x bits = x stops dynamic range" is a shorthand that works within the confines of calibrated SDR digital cinema cameras, but becomes increasingly inaccurate outside of that, and falls down for modern HDR techniques that use pixel binning or multiple exposures.
For modern CMOS sensors such as those used on Perseverence, the lower-bound (outside of issues with the electronic amplification stage where someone is cutting corners to reduce cost) for sensors is only 1-2 photons needed for a measurable signal. This means the upper bound, and thus dynamic range, is usually driven by how many photons saturate the sensor and/or the amplification stage.
Supercam, for example, has multi-exposure HDR capture mode, which indicates its designers were not satisfied with the single-image dynamic range available with the sensor and thus single-exposure (non-stacked) images can be assumed to not be HDR. Stretching one of these images to the range of a HDR display would be a type of false-colour image display: visually impressive, but not particular accurate to what has been captured. The science cameras (e.g. Supercam) do make a data product available with radiometric calibration for images to get actual exposure levels, but IRC no such calibration is available for the NAVCAM images currently used.

[Holger Isenberg]

[...] radiometric calibration for images to get actual exposure levels, but IRC no such calibration is available for the NAVCAM images currently used.

Thanks for the details! I simplified the complicated internals. For the end user it usually means they need more bits than 8 on the camera and on the display to get nice HDR images as otherwise the problems happens you explained.

About the calibration: That's indeed scientifically currently not possible for Navcam and Hazcam because NASA is not publishing the test images taken on Earth by those. And only with having those a scientific model for calibration can be validated.

For the calibration I apply to the Navcam images I used all the existing data about the spectral sensitivity in part provided by the sensor vendors and calculated it as best as possible, but the final validation is not possible due to above.