HDR, well I'll be danged ...

[Saorsa]

I always shoot in raw to preserve as many options as possible for post processing. I've often wanted 'real' HDR but every package I tried seemed to insist on filters that made the image look less than natural. I tried futzing about with sliders but never really managed to get what I wanted, a natural looking image with a wide dynamic range.

I had noticed an HDR setting on my Nikon D7100 and discovered the same thing on the D750. The only trouble was it was always greyed out as unavailable in the menu. The best I could find reading the manual was that it meant some parameter was unsuitable for HDR.

I was playing around with the D750 learning the controls and generally handling and fondling it. I wasn't trying any particular setting, just fiddling with controls and inadvertently left the image quality at jpeg fine. When I went through the menu again, I noticed that the HDR was now available. Hmmmm, says I, I don't know what I changed but lets see how this works. I stepped out of the back door and took two shots across the canal one with HDR on and one off. What a difference that uninteresting image was.

I quickly shot up to our local history park where I knew I had trouble with shadows and bright areas along a path to one of the buildings. Two shots again and here is what I got.

First shot matrix metering



Then, I turned the HDR on and took the same shot.



BAM, just what I was hoping for.

[Manfred M]

If you check your manual, it will tell you that in-camera HDR is only available if you shoot with your camera set to jpeg only. The moment you enable any form of RAW (including raw + jpeg), that functionality gets disabled.

[Saorsa]

If you check your manual, it will tell you that in-camera HDR is only available if you shoot with your camera set to jpeg only. The moment you enable any form of RAW (including raw + jpeg), that functionality gets disabled.

It said it wouldn't record .NEF and I took that to mean that the processed image would not be stored as Raw.

[george013]

It said it wouldn't record .NEF and I took that to mean that the processed image would not be stored as Raw.

A question, does the camera take different shots?

George

[Manfred M]

It said it wouldn't record .NEF and I took that to mean that the processed image would not be stored as Raw.

If you think about it, that makes perfect sense as the camera is blending a series of shots to create an new interpretation of the scene. Raw data is essentially a data dump from the camera sensor without any manipulation.

From a purely practical standpoint, the jpeg files are a lot smaller and there is less data to process as well. You are shooting a burst of shots, so fast processing through the buffer and data pipeline is far less resource intense than doing so with the larger data files. My D800 produces the "maximum quality" jpegs that range from 1/4 to 1/3 the size of the raw files for the same image.

[Dave Humphries]

It said it wouldn't record .NEF and I took that to mean that the processed image would not be stored as Raw.

Nope, .NEF is Nikon's RAW file format, if you look at the file extension of the resulting image, I think you'll find it is .jpg.

[MrB]

I suspect that there are probably quite a number of people who have followed the mantra of "real photographers only shoot in the raw", and have never discovered what the technology built into the modern camera is capable of achieving.

The HDR image is very good but it is also worth noting that, if there are some aspects of the result that you would like to modify, the high quality (low compression) JPEG can stand up to being edited to quite some extent, before there is noticeable loss of quality.

Cheers.
Philip

[Saorsa]

Nope, .NEF is Nikon's RAW file format, if you look at the file extension of the resulting image, I think you'll find it is .jpg.

Yes, I expected the processed HDR image to be stored as jpeg. It is after all a composite, processed image not a raw capture. I did not understand that there could be no .nef(raw) capture at all or that setting raw+jpeg fine would not provide the necessary data for processing the jpeg data as HDR.

The specific statement in the D750 manual (p177) says "It can not be used to record NEF(RAW) images. Flash lighting, bracketing, multiple exposure, and ime lapse photography cannot be used while HDR is in effect and shutter speeds of bulb and -- are not available.

From a software/firmware perspective, there is no reason that the 'normal' exposure could not be stored as both raw and jpeg with the jpeg fine for the 'normal', over and under captures merged in the HDR processing to produce a final HDR jpeg.

My only issue here is a statement which I believed to say that the HDR image would not be stored as .nef, not that I could not set .nef as one of my outputs. Since that is a seperate sentence to the one listing forbidden settings (flash, bulb, etc.) I took it as a separate statement.

[Saorsa]

A question, does the camera take different shots?

George

It appears to take three or more jpeg captures and process them as an HDR image. The shutter sound is a bit different when the HDR flag is set.

[Manfred M]

Brian - my educated guess is that the reason Nikon took the route they did is most likely related to the onboard RAM.

We know that all images are processed to a jpeg for display and histogram purposes, the the amount of RAM (as opposed to the buffer or external storage) is likely limited to roughly the amount of memory required to process the data from a single raw file. I haven't seen information of the image processor chip, but suspect it uses on-board RAM. Processing three jpeg files will take roughly the same amount of RAM as a single raw file. I suspect the practical hardware limitations (and possible processing time as well) likely drove the decision to work with jpegs.

[NorthernFocus]

It appears to take three or more jpeg captures and process them as an HDR image. The shutter sound is a bit different when the HDR flag is set.

If I'm not mistaken it fires three images over a user defined EV range. Similar to bracketing but it blends them into a single image. I find it amazing that the camera can do so even when shooting handheld.

I recently went though a similar exercise but with my Sony a6000. I shot the same scene in RAW and with a 5EV range HDR(jpeg) image. After the fact I was able to process the RAW file in LR with simply the highlights, shadows, black/white points to essentially create the same image. My first thought was, why bother with the HDR if I can get the same way I've always done with shooting RAW. Then I woke up, duuuugh. Why bother post processing and ultimately converting the final image to jpeg if you can get the exact same result SOOC

[MrB]

My understanding of the computing aspects of image processing is very limited, but I thought that a JPEG was a file storage format, specifically for compressing image data for saving into a smaller file size, and that the data cannot be worked on in image processing software until the file has been decompressed into RAM. So it seems to make no sense to me that the camera's processor would take the data from the three recorded images (which are always raw data), process them into three separate image files, compress them to make three JPEG files, then decompress them to process them into the HDR image file, and finally compress that data into the final JPEG file. Perhaps the boffins have programmed a way of using the raw data from the three captured images in quite a different way to make the final HDR image file, which might explain why from my cameras (Pentax), as yet, the three intermediate images are not available for storage on the memory card.

Cheers.
Philip

[IzzieK]

Brian...thanks for this post. It is interesting to note that you can't take HDR with .nef files...but I prefer the response of Dan on how to get the same result. Your test is interesting though but wouldn't go that route...

[george013]

My understanding of the computing aspects of image processing are very limited, but I thought that a JPEG was a file storage format, specifically for compressing image data for saving into a smaller file size, and that the data cannot be worked on in image processing software until the file has been decompressed into RAM. So it seems to make no sense to me that the camera's processor would take the data from the three recorded images (which are always raw data), process them into three separate image files, compress them to make three JPEG files, then decompress them to process them into the HDR image file, and finally compress that data into the final JPEG file. Perhaps the boffins have programmed a way of using the raw data from the three captured images in quite a different way to make the final HDR image file, which might explain why from my cameras (Pentax), as yet, the three intermediate images are not available for storage on the memory card.

Cheers.
Philip

You're right that JPG is a file storage format, and also a way of compression. The image itself is a raster-based RGB-image converted from the individual R,G,B sensor-info. A difference often misunderstood.
I think that proces, RAW-conversion-JPG is done little by little. The JPG-compression is block defined. You can take 1 block of x pixels, convert it and save it. Would be the use of a minimum of memory.
Converting a RAW-image of 4000x 6000 to an 8-bit image will take 4000x6000x3=72MB just for the image, no overhead, additional info etc.

@Saorsa,

It appears to take three or more jpeg captures and process them as an HDR image. The shutter sound is a bit different when the HDR flag is set.

Maybe you're missing the mirror movement.

I'm not sure if I understood it right, but I thought having read that the speed is 2 shots per second. I'm not sure about this.

George

[Dave Humphries]

The book I read; on the D7100 (i.e. not Brian's D750) suggests that in Auto-HDR, the camera takes just two images and blends them produce one jpg image that encompasses the entire tonal range. There is a user controlled setting to cope with different situations, the book suggests that trial and error may be needed to get the absolute best result from a given scene.

I strongly suspect that these sections of manuals and books are left deliberately vague since it is the kind of area in which subsequent firmware updates might change/improve things over time (or they might not!).

I haven't tried it at all, so I cannot give any definitive answers.

[MrB]

Some info. on Pentax HDR:
The HDR Mode on the Pentax K-5 IIs takes three bracketed shots, -3, 0 and +3 EV.
The images can be auto-aligned, so hand-holding is viable.
There are five HDR options: Auto and 4 strength levels (Standard HDR, and Strong 1, 2 and 3). I usually set either Auto or Standard, as they seem produce the most natural result.
Exposure compensation is still available, so the set of bracketed exposures can be moved up to five stops either up or down the EV scale. Useful, e.g. when the scene contains very bright elements.
Although the HDR Mode works to capture only a JPEG HDR result, the middle exposure can be recorded as a raw file, after the HDR JPEG has been saved, by pressing the AEL button.
The K-S2, a more recent Pentax DSLR, also allows the user to select the separation of the bracketed exposures by 1, 2, or 3 EV, instead of just the 3 EV of the K-5.

Cheers.
Philip

[Saorsa]

The book I read; on the D7100 (i.e. not Brian's D750) suggests that in Auto-HDR, the camera takes just two images and blends them produce one jpg image that encompasses the entire tonal range. There is a user controlled setting to cope with different situations, the book suggests that trial and error may be needed to get the absolute best result from a given scene.

I strongly suspect that these sections of manuals and books are left deliberately vague since it is the kind of area in which subsequent firmware updates might change/improve things over time (or they might not!).

I haven't tried it at all, so I cannot give any definitive answers.

It's possible to use any number greater than one to do HDR processing The idea is to have at least one image which preserves highlight detail and one to preserve shadow detail. Two is a minimum and would probably be entirely adequate dependent on the sensitivity of the sensor.

I've been thinking about this in terms of the possible algorithms for internal processing and it's making more sense to me now but I wish Nikon had just said it wouldn't work with any form of raw capture. I'm pretty sure the issue is a bit more than just ram as GrumpyDiver says in post 10. There is a certain amount of processing that has to be done in merging and normalizing the pixel values of the two images. For example the images must be aligned properly and then each aligned pixel site evaluated and normalized. This could be very resource consuming in higher precision modes like TIFF or NEF.

I suspect that the shutter opens once and two or more samples are captured at 8 bit precision. The Buffer obviously has capacity for multiple higher precision images but processing time would be vastly increased with each additional bit.

This would have been a very interesting problem when I was working as a programmer of DSPs and PLCs. Fortunately I'm retired now but I can still complain about how the tech writer can't express something as simple as "HDR operation will work only in Jpeg modes (basic, normal, fine). Any raw mode will disable HDR processing."

[tclune]

The book I read; on the D7100 (i.e. not Brian's D750) suggests that in Auto-HDR, the camera takes just two images and blends them produce one jpg image that encompasses the entire tonal range.

Stupid question: my understanding is that current Nikon cameras are ISO-less. If that is true, why bother taking two photos at all -- why not just take one photo and shift the data, eliminating alignment issues between the two data sets, then do the HDR processing before generating the JPEG at all?

[Saorsa]

Brian...thanks for this post. It is interesting to note that you can't take HDR with .nef files...but I prefer the response of Dan on how to get the same result. Your test is interesting though but wouldn't go that route...

In a lot of situations you can indeed get the same result in processing raw. That is not true in all case however where the range of light is very large or concentrated in a small range.

Processing Raw does start with an image of greater precision which allows the black spot/white spot to work well but HDR actually gives you the benefit of a greater range. If you look at the images I posted, the HDR version shows a lot of detail in the black plastic recycle bin next to the trashcan on the right. I might try it some time but I don't think I could get that kind of detail from a single raw image and still have detail in the really light portions.

Now that I have a better understanding of this I think I will be using it occasionally.

HDR would actually work with Raw files it's just that the computers in the cameras today don't really have the horsepower to deal with the amount of processing required for higher precision. I wouldn't want to wait 5 minutes between shots

[george013]

In a lot of situations you can indeed get the same result in processing raw. That is not true in all case however where the range of light is very large or concentrated in a small range.

Processing Raw does start with an image of greater precision which allows the black spot/white spot to work well but HDR actually gives you the benefit of a greater range. If you look at the images I posted, the HDR version shows a lot of detail in the black plastic recycle bin next to the trashcan on the right. I might try it some time but I don't think I could get that kind of detail from a single raw image and still have detail in the really light portions.

Now that I have a better understanding of this I think I will be using it occasionally.

HDR would actually work with Raw files it's just that the computers in the cameras today don't really have the horsepower to deal with the amount of processing required for higher precision. I wouldn't want to wait 5 minutes between shots

You could try to shoot some pictures with exposure bracketing and merge them afterwards. You've the RAW from the individual photo's and can try to analyse the proces. I guess the rough idea is to overlay the pictures and choise that pixel that's closest to the middle,128, so eliminating the extremes. Just a guess.

George