It is just you and me

[xpatUSA]

Originally Posted by xpatUSA
Fortunately I have access to raw histograms and just did the following to measure the headroom directly without the need for iteration:

Shot a white card such that the live view histogram was just ETTR (cam does not have live blinkies).

Opened the raw in RawDigger and observed the histogram. Sure enough, there was 1EV of headroom below the known raw saturation level.

A quick test of one, with some room for experimental error, but I must now agree that between the raw capture and the sRGB JPEG there is far more "leeway" than simple math implies!

I just converted several raw images in capture nx-d. And then compared the jpg with the "raw". All jpg's have LESS clipping.
I think you can use any converter to compare with the raw: just take care the jpg has been converted from the same picture in the same converter.

If that was a response to my test, I have no comment because my test did not include any conversions and no raw converter was used.

Sorry.

[Manfred M]

George - the firmware that is embedded in the camera and Nikon's raw converter software will not produce identical results, nor should they. The firmware in the camera is quite specific and written for the specific camera model. The raw converter is generic and has to be "tuned" for each specific camera model. Just because both have a Nikon logo does not suggest that they are the same product and will produce identical results.

The issue being discussed is the "headroom" that is available in raw data. The context is quite independent of the software being used; the work that Ted and I are referring to shows that Ole does not have to underexpose by a stop in order to not blow out the highlights. He should be able to shoot without doing so and that will give him better results.

[george013]

George - the firmware that is embedded in the camera and Nikon's raw converter software will not produce identical results, nor should they. The firmware in the camera is quite specific and written for the specific camera model. The raw converter is generic and has to be "tuned" for each specific camera model. Just because both have a Nikon logo does not suggest that they are the same product and will produce identical results.

The issue being discussed is the "headroom" that is available in raw data. The context is quite independent of the software being used; the work that Ted and I are referring to shows that Ole does not have to underexpose by a stop in order to not blow out the highlights. He should be able to shoot without doing so and that will give him better results.

They produce exact the same image, up to now.

That so called headroom is not linear. First a wb correction and then a gamma correction. As I saw, the jpg will be clipping later as the raw, due to its compression. Not earlier.. Maybe due to non comressed values and higher bit depth better results can be gained with the raw.
I think Ole said he didn't underexpose this specific image: he forgot to set it.

George

[William W]

Ole,

What camera and what lens did you use to make this photo?

WW

[Manfred M]

They produce exact the same image, up to now.

That so called headroom is not linear. First a wb correction and then a gamma correction. As I saw, the jpg will be clipping later as the raw, due to its compression. Not earlier.. Maybe due to non comressed values and higher bit depth better results can be gained with the raw.
I think Ole said he didn't underexpose this specific image: he forgot to set it.

George

Still off-topic and still not understanding what happens in data compression, George.

As I do not know the specifics of the compression algorithms used either in the camera or in the external software (i.e. raw converter) I can only draw on my general knowledge of the topics. If you do, please share your sources (rather than making things up).

First of all raw data and a JPEG cannot possibly produce identical images. Raw data is 16-bit (constructed from 14-bit data in most modern higher end cameras) whereas JPEGs are 8-bit. This means in raw, each colour channel can have 16,384 discrete values whereas in JPEGs we are down to 256 discrete values per channel. I cannot see how you suggest the images created can be equal. Nor does it make any sense that there is more "headroom" in a JPEG than with the raw data. Throwing away half the data by going from 16-bit to 8-bit saves a lot of file size; in fact one is actually throwing away over 98% of the colour data...

When it comes to JPEG compression, the algorithm designer, who understands the human visual system has two aspects of the human visual system that he or she can exploit:

1. Human colour vision is most sensitive in the mid-tone range; it is not sensitive to highlights and to a lesser extent shadows, so throwing away data in those ranges can be done without impacting how the final image is perceived. This approach is definitely used in video signal processing where luma information is protected and chroma data is thrown away. Similar approaches can be used in still image processing too.

2. The human visual system can "only" resolve about 10 million individual colours, so combining colour values that are "close" is also going to produce imperceptible results in the final product. Even lowly 8-bit images produce over 16 million colours, so the algorithm designer can play games here with little risk of anyone noticing. All that data we get in raw files is only necessary during the editing stage where we need lots of data to manipulate. Once we are finished, throwing it out in the final image works quite well.

You are right that Ole forgot to do his usual "thing" and deliberately underexposed the image. The point i was making is that in general, overriding the cameras automation by deliberately underexposing ALL street shots is both technically a poor decision and also unnecessary.

[george013]

Still off-topic and still not understanding what happens in data compression, George.

As I do not know the specifics of the compression algorithms used either in the camera or in the external software (i.e. raw converter) I can only draw on my general knowledge of the topics. If you do, please share your sources (rather than making things up).

First of all raw data and a JPEG cannot possibly produce identical images. Raw data is 16-bit (constructed from 14-bit data in most modern higher end cameras) whereas JPEGs are 8-bit. This means in raw, each colour channel can have 16,384 discrete values whereas in JPEGs we are down to 256 discrete values per channel. I cannot see how you suggest the images created can be equal. Nor does it make any sense that there is more "headroom" in a JPEG than with the raw data. Throwing away half the data by going from 16-bit to 8-bit saves a lot of file size; in fact one is actually throwing away over 98% of the colour data...

When it comes to JPEG compression, the algorithm designer, who understands the human visual system has two aspects of the human visual system that he or she can exploit:

1. Human colour vision is most sensitive in the mid-tone range; it is not sensitive to highlights and to a lesser extent shadows, so throwing away data in those ranges can be done without impacting how the final image is perceived. This approach is definitely used in video signal processing where luma information is protected and chroma data is thrown away. Similar approaches can be used in still image processing too.

2. The human visual system can "only" resolve about 10 million individual colours, so combining colour values that are "close" is also going to produce imperceptible results in the final product. Even lowly 8-bit images produce over 16 million colours, so the algorithm designer can play games here with little risk of anyone noticing. All that data we get in raw files is only necessary during the editing stage where we need lots of data to manipulate. Once we are finished, throwing it out in the final image works quite well.

You are right that Ole forgot to do his usual "thing" and deliberately underexposed the image. The point i was making is that in general, overriding the cameras automation by deliberately underexposing ALL street shots is both technically a poor decision and also unnecessary.

I'm on-topic. It all started with this post of you.

Originally Posted by mugge View Post
I started doing that a while ago as many of my images were in the 'overexposed region.' As I do love street scenes it is a safer option to have the camera set for underexposure at least I can recover a lot more images that way. No one likes blown images.

Clipped shadows are impossible to recover. I shoot to "correct" exposure all the time and very rarely clip the highlights. Underexposed areas tend to show digital noise, so that is something that needs to be avoided.

By correct exposure I mean I check the channel histograms and ensure that no channel is clipping. The camera bases the histogram on the JPEG file, so even if you show a bit of clipping, you still have about 1 to 1-1/2 stops of headroom on raw data.

I assume you mean clipped highlights when you wrote clipped shadows.

Can you tell me how you gain that "correct exposure"? Isn't that by starting to examine the result of the measuring of the light meter and correct that if you see clipping, based on your histogram. What is the essential difference by correcting that exposure on the forehand based on experience??

I assume you're referring to a life histogram. If so, did you ever think about it that not everybody has a life histogram? I don't think I have one.
If you use the histogram of a taken shot, the jpg, then you just doing a try and correct cycle.

Now your side roads.

Photography is working visual. As you mentioned a human can only distinguish between 10 million colors, less then 8 bit per channel. Let my monitor use 8 bit per channel, isn't that wonderful? No difference visible between a 8-bit and a 16-bit image.

I know a VERY SIMPLE diagram showing that the converted raw data into a RGB- raster image and the jpg file are the same: BY DEFINITION. The jpg is nothing else as a DISKFILE containing that RGB-rasterimage. Well, with some compression and even that depending on the settings.

And one thing more. Nowhere I said that a jpg has more headroom as a raw. Why insinuating that?

George

[mugge]

Ole,

What camera and what lens did you use to make this photo?

WW

I used Olympus e-m1 mk 1 and 12-40mm 2.8 Oly lens

[William W]

Thanks.

I refer generally to the topic which Manfred has introduced and specifically to the 'headroom' of a Digital camera.

'headroom' is more of a colloquial term than a technical term: in simple terms when used in this meaning, it is basically how far you can push to the right and still be able to recover the highlights in Post Production - OR - blowing out only a few highlights that cause not too much disturbance to the Final Image.

A key factor to remember is that your camera (most modern Digital Cameras) have a dynamic range that exceeds the Dynamic Range of the screen you'll look at and the printed Final Image that you'll view - so the further you can push the Camera's Exposure to the right whilst still being able to recover the highlights, the MORE Camera Exposure you'll have given to the SHADOWS. This is important, because, (as a good general rule), the MORE exposure you have in the shadows the less apparent noise you'll get in the final image Post Production.

You'll notice that I have mentioned "Post Production" quite a lot. That's because it is important, especially for the type of image you've captured - it is an image with a reasonably wide Dynamic Range - AND – with Blacks and Whites across that wide DR.

Note that I mentioned that modern Digital Cameras have a wide Dynamic Range: that’s good for this type of scene because you can be assured that the camera will capture the detail across the scene’s dynamic range.

HOWEVER, the important point is the bit about the screen or the print not having the same dynamic range as the camera – so in Post Production, if you want to see all that detail in the scene you’ll need to ‘squash it up’ to get it into the DR of the screen or the print.

The kicker is – the more exposure that you have for the darkest shadow you have to squash from the bottom (the shadows) the more detail you’ll have when you start the ‘squashing up’ . . . now this is basically the same as what has been already stated, but I’ve stated it a little differently.

***

Now we get to a technique to use to expose as far to the right as possible: there are a few.

Using the histogram and the blinkies (if you have them) and then Chimpng is one method. I rarely use that.

What I have done is trial my cameras in typical lighting scenarios. I have three basic lighting scenarios:

. Front lit hard light

. Backlit hard light

. Soft light (interior) or cloud cover/open shade outside

Using those three typical lighting scenarios I have trialled several exposures, using EXPOSURE BRACKETING – and from there I then selected the image file, exposed as far to right as possible from which I could still recover the highlights.

Then it was a matter of relating that file’s exposure to the Camera’s TTL Meter's suggested exposure.

For example, for a typical front lit, hard light scene, my camera might have suggested: F/16 @ 1/250s @ ISO200. (And remember that I have BRACKETED several the exposures for this same scene).

Let’s assume that I find that the exposure F/16 @ 1/125s @ ISO200 (one stop overexposed to the cameras suggested exposure) has highlights that I can recover in Post Production.

Let’s assume that the exposure F/16 @ 1/100s @ ISO200 (one and one third stops over the camera’s suggest exposure) had highlights that I could NOT recover.

Let’s further assume that I run several tests on several front lit scenes and I can ALWAYS recover the highlights from the file which is one stop over the camera’s suggested exposure – then I will conclude that this camera has 1 stop of ‘headroom’ for front lit hard light scenes.

So I am now armed with a rule - PROVIDED THAT I ALWAYS USE THE SAME METERING MODE for this type of Lighting Scenario, I know that I can push the exposure one stop more than what the camera’s TTL Meter suggests and I can be reasonably confident of being safe.

Now I might choose NOT to do that for all my shots – for example for front light hard light scenes where the shadow detail is on little consequence, I might choose to think “I don’t like post production anyway - so long as the (foreground) main Subject has good detail I don’t care all that much about the (background) shadow detail.

***

An important point about using the Histogram and the Blinkies: as I mentioned, I don’t use them very often, but I do sometimes. I mainly use Canon DSLRs and on those cameras the Histogram and the Blinkies are driven by the JPEG File.

IMPORTANTLY – the JPEG Image File is driven by the IN-CAMERA JPEG Parameters. I think that many (most) cameras are built this way.

SO if you use the Histogram and/or the Blinkies for Chimping and as a guide as to how far you can push to the right, you must be aware that changes to the JPEG IN-CAMERA Parameters will affect what you see on the histogram and in the Blinkies.

The main reason why I don’t use the Histogram or the Blinkies all that often is because I capture ‘raw + JPEG (L)‘ and I often use the JPEG Image SOOC (Straight Out of Camera), and I sometimes change the JPEG Parameters to suit the SOOC image that I want.

Therefore, for me, it is easier just to know simple rule like: “on this DSLR for Front Lit stuff and so long as I use EVALUATIVE METERING I can push one stop and I will be safe.”

Actually I’ve always argued that, that type of rule is simpler for 99% of shots that most people make, and I have never seen the passion for habitually interrogating the Histogram and Chimping: if the shot is that important and I am uncertain, I’d rather set a bracket (typically ± ⅔Stop) and fire away.

Histogram Gazing and Chimping up and down, as an HABITUAL FUNCTION, is recipe for missing the action.

WW

[xpatUSA]

Thanks.

I refer generally to the topic which Manfred has introduced and specifically to the 'headroom' of a Digital camera.

'headroom' is more of a colloquial term than a technical term: in simple terms when used in this meaning, it is basically how far you can push to the right and still be able to recover the highlights in Post Production - OR - blowing out only a few highlights that cause not too much disturbance to the Final Image.

WW

Re: colloquial v. technical, Bill, the term "headroom" is used (but not defined per se) in the Standard for Sensitivity ISO 12232:

"Note: Equation (5) provides 1/2 “stop" of headroom (41 % additional headroom) for specular highlights above the signal level that would be obtained from a theoretical 100 % reflectance object in the scene, so that a theoretical 141 % reflectance object in the scene would produce a focal plane exposure of Hsat."

The note is referring to saturation-based ISO speed, a method less popular in these days of REI and SOS. Point being that usage in an ISO Standard renders the term fairly official, I would have thought.

On the other hand, the well-respected Doug Kerr is usually careful to put quotes around the the term which, to me, does imply colloquialism:

http://openphotographyforums.com/for...itivity.18387/

Not disagreeing with your post's technical content at all, BTW.

As to using the camera histogram this article from the authors of RawDigger clearly disassociates image brightness (as shown by the histogram) and degree of exposure:

https://www.fastrawviewer.com/blog/i...esent-exposure

In fact, it is all too common for a darkly-developed image to be declared as "under-exposed" and vice-versa!

[Manfred M]

I assume you mean clipped highlights when you wrote clipped shadows.

No, I meant shadows. While there is headroom at the highlight end, there is little to none at the other end of the histogram. Once my camera shows clipped shadows, those values cannot be recovered

Can you tell me how you gain that "correct exposure"? Isn't that by starting to examine the result of the measuring of the light meter and correct that if you see clipping, based on your histogram. What is the essential difference by correcting that exposure on the forehand based on experience??

My view is very similar to what Bill wrote in #28.

Correct exposure is determined by the lighting and the subject. When I shoot, try to get the best data I can in my capture so that I have the ability to pull out the image when I get it into my computer. Evaluative metering does the job without any overriding on my part 95% of the time. Back lit scenes, hard light, snow scapes, night shots, etc. take a bit more care and attention and in those cases I will (depending on the subject) bracket, use exposure compensation or shoot in manual mode.

In those cases I use my histogram (the one that shows each colour channel individually) to ensure that I do have good data. To me that is the critical aspect of getting the exposure right.

I assume you're referring to a life histogram. If so, did you ever think about it that not everybody has a life histogram? I don't think I have one.
If you use the histogram of a taken shot, the jpg, then you just doing a try and correct cycle.

The cameras that I use most often do not have a live histogram. When I start shooting in a new lighting situation, the first thing I do is look at the light and adjust my exposure based on experience; based on what the histogram shows me. I then take a test shot to see how close I am to getting the exposure I am looking for and tweak my settings accordingly.

Photography is working visual. As you mentioned a human can only distinguish between 10 million colors, less then 8 bit per channel. Let my monitor use 8 bit per channel, isn't that wonderful? No difference visible between a 8-bit and a 16-bit image.

My computer screen is 10-bit, so I run it in that mode. It has more subtlety than what I see in 8-bit. Graphics cards have been supporting 10-bit screens for many years.

The 16-bit data is not required for final output, but rather to give the editing tools the maximum amount of data to work with. This gives cleaner results without introducing artifacts. As I do a lot of printing, a printer produces even fewer colours than a screen does (although it does have a wider range), so being able to compensate for the way my printer / paper reproduce is quite important in creating an optimal

I know a VERY SIMPLE diagram showing that the converted raw data into a RGB- raster image and the jpg file are the same: BY DEFINITION. The jpg is nothing else as a DISKFILE containing that RGB-rasterimage. Well, with some compression and even that depending on the settings.

As I and many others have pointed out, that diagram is wrong. Please go back and understand the processes you are writing about instead of just making things up.

JPEG is a file format. It contains data that can be reconstructed into an image by software that is designed to do so and feed it to the computer's video hardware / software. In addition to the data, it also contains a header that tells the software how to interpret / decode the stored data. Other image formats like TIFF. PNG do very much the same thing.

So far as I understand it, there are two distinct parts to compression in a JPEG. The first is accomplished by eliminating data based on the amount of compression specified. This includes cutting 16-bit data to 8-bit and eliminating colour data. There is also some standard data compression (i.e. like what happens with ZIP files) as well.

And one thing more. Nowhere I said that a jpg has more headroom as a raw. Why insinuating that?

I re-read your posting several times and that is how I understood what you had written. Obviously there was a disconnect between what you had written and how I read it.

[xpatUSA]

No, I meant shadows. While there is headroom at the highlight end, there is little to none at the other end of the histogram. Once my camera shows clipped shadows, those values cannot be recovered

Some people try to use different terminology, presumably to avoid such confusion. I've seen "footroom" and "bottoming", for example.

My view is very similar to what Bill wrote in #28.

... Back lit scenes, hard light, snow scapes, night shots, etc. take a bit more care and attention and in those cases I will (depending on the subject) bracket, use exposure compensation or shoot in manual mode.

In those cases I use my histogram (the one that shows each colour channel individually) to ensure that I do have good data.

Unfortunately, some cameras only show "luminosity" in the live histogram, such that a clipped color channel can be quite hidden. So a 'separate colors' histogram is a good thing to have, because clipped colors in the converted image mean false colors. Equally, "bottomed" color channels can often mean false colors (due to over-saturation, for example).

https://www.cambridgeincolour.com/tu...istograms2.htm

Although my live-view Sigmas (not the DSLR) can indeed show a 'separate colors' histogram it is after-the-fact in the post-shot LCD review.

[george013]

.....
I re-read your posting several times and that is how I understood what you had written. Obviously there was a disconnect between what you had written and how I read it.

Your post 22

......
The issue being discussed is the "headroom" that is available in raw data.......

My answer post 23

That so called headroom is not linear......

Your post 25

......
Nor does it make any sense that there is more "headroom" in a JPEG than with the raw data.....

One of us must miss something.

George

[george013]

Does nobody see the problem Ole is facing. It's not exposure settings. It's that due to sun and a lot of white, high contrast etc the dynamic range of the scene is bigger as the dynamic range of the sensor/camera.
A very common situation. That's one of the reasons you've a EC button on the camera.
A second discussion would be: how far do you go. That could be personal, a habit

George

[William W]

Does nobody see the problem Ole is facing. It's not exposure settings. It's that due to sun and a lot of white, high contrast etc the dynamic range of the scene is bigger as the dynamic range of the sensor/camera.

This statement is absolutely unsubstantiated and certainly is not referenced by any definitive measurements of the Camera’s DR nor the Scene’s DR.

Additionally, there is only an extremely limited analysis possible considering the only image is a Post-produced Black and White conversion without EXIF data and the commentary provides scant details of the shooting conditions, lighting, location and general description of the scene.

This statement is potentially unhelpful and confusing to many readers.

WW

[george013]

This statement is absolutely unsubstantiated and certainly is not referenced by any definitive measurements of the Camera’s DR nor the Scene’s DR.

Additionally, there is only an extremely limited analysis possible considering the only image is a Post-produced Black and White conversion without EXIF data and the commentary provides scant details of the shooting conditions, lighting, location and general description of the scene.

This statement is potentially unhelpful and confusing to many readers.

WW

See post 5

I shoot mostly in AP mode and when I'm in street scenes I normally have camera set at one stop underexposure. This time I did not

A normal situation.

George

[William W]

Aside -

Re: colloquial v. technical, Bill, the term "headroom" is used (but not defined per se) in the Standard for Sensitivity ISO 12232 . . . [etc]

Ted,

I appreciate all the content of Post #29. The links are helpful and I have added them to my library.

'headroom' - I first learnt about 'headroom' when studying sound engineering. When Digital Photography came into the mainstream, in any commentary I have generally used the term colloquiallal.

Maybe 'colloquial' is more a description of me: I sometimes purposely avoid getting into technical discussions.

Doug Kerr is very smart and very knowledgeable.

WW

[xpatUSA]

... 'headroom' - I first learnt about 'headroom' when studying sound engineering. When Digital Photography came into the mainstream, in any commentary I have generally used the term colloquially.

Maybe 'colloquial' is more a description of me: I sometimes purposely avoid getting into technical discussions.

Well, while I was posting, I had quick look for a definition of photographic headroom but to no avail. Plenty about audio headroom, though. So, colloquial it is, eh?

Doug Kerr is very smart and very knowledgeable.

Long ago, Doug helped personally with trying to measure the luminance of watch hands (before I knew everything).

See https://tcktek.blogspot.com/2011/06/exposing-lume.html

The interesting part of the exercise was that I learned from him the whole relationship between a subject luminance and it's brightness in a developed image.