Histogram: Luminosity and Color

[Glenn NK]

- so to deliberately bias the exposure upwards, risking capturing data in a sensor's non-linear region, only to have to bias it downwards in post-processing, risking issues getting the gamma right, just doesn't make any sense to me (unless it's a high dynamic range scene that requires it).

I've tried to find some information on this biasing in the non-linear region of a sensor (perhaps some graphs) but haven't been able to find any through a search. Do you have something on this?

Glenn

[Manfred M]

I've tried to find some information on this biasing in the non-linear region of a sensor (perhaps some graphs) but haven't been able to find any through a search. Do you have something on this?
Glenn

Hard to do without the spec sheets on the sensors, but I do feel that Colin is likely right on this one as these devices will not be 100% linear through the entire usable range.

Generally the manufacturers will trade off a bit of non-linearity for extra performance. Remember that the signal from the sensor goes through a very simple amplifier circuit before being output to the A/D converter, and this will be another source of non-linearity being introduced over and above those from the photodiode performance.

he non-linearity will be more pronounced at both ends of the range, so one would expect these effects both shooting ETTR as well in underexposed images that have not been clipped too badly at the black end.

[ajohnw]

There seems to be a bit of a misunderstanding in this area. Sensors are basically linear devices. Neglecting noise the only thing that can make them none linear is electrical leaks from the wells. This page gives a decent overview of what goes on and also explains apparent none linearity which in real terms is down to them being linear where as we see approximately logarithmically - the light level doubling nature of stops in other words.

http://www.ronbigelow.com/articles/bits-1/bits-1.htm

Actually I feel he has missed the point of 14bit A/D. The increased resolution at the dark end leaves more scope for expanding in a logarithmic fashion at the dark end of things without posterisation. It needn't mean that the sensor itself actually has more dynamic range. More likely less noise so that the numbers read are more meaningful. Why have they done this - things like D light and jpgs. They need that if it's to work as it has to increase contrast as well as brighten up the dark end. Good for from raw too for the same reason.

He also electronically speaking miss uses quantisation errors.- it's a term that relates to a/d and d/a converters. As does linearity errors but if those are larger than the expected accuracy of the signal .............. It's also possible to have none linear A/D's as well because they are loosely based round comparing a signal with the voltage on a resistor network. I suspect they would be telling us about that if they were doing that - A true logarithmic converter is possible in theory at least but probably not in practice.

John
-

[Inkanyezi]

I don't think we should delve too much in the relative linearity of "linear" devices. They are all basically linear, but there is both a threshold and a roof for their linearity, where the threshold depends on various kinds of noise and the roof depends on the voltage.

The three groups of sensels are working together, and when reaching the roof, one of them will always do so first, then the other and last the third, effectively responding in a non-linear manner. When all three have hit the roof, we say that they have clipped, but colours do clip separately. To me, this is a non-linear region of the sensor. When all three are capturing data below the roof, they are linear. Mostly the red will hit it first, then the blue and last the green, but that order can change depending on the colour of the light that hits them.

Unfortunately, the blinkies will often not kick in until all three are clipped, so relying on them only, may force the sensor into the non-linear region, where the petals of roses and poppies only get a flat red, and where colours mixed with red will not get the right proportions of it. RAW shooters with decent conversion software can see this and respond to it, while jpeg shooters will never know what caused it. Your RAW conversion software might show blinkies for single channels.

[ajohnw]

Don't know about that Urban. Once clipped they are gone. JPG's often have a none linear response at both ends when drawn on a log graph but that's after processing. The reason I posted the link is that it shows why some might think they have none linear regions because we look at them in a logarithmic fashion as a 1 stop down means halving light levels. So lots of counts at the bright end and very few at the dark end but what ever the count is they are all stops.

There will be some leakage of some sort from the wells in a sensor so that will add some degree of none linearity.

The reviewers don't help as they could actually show the numbers produced by the cameras. This one might interest you. Go to the picture controls widget and select ISO in the drop down at the top. I set 100, 400, 1600 and 1280. The "dynamic ranges" shown are interesting. For some reason they don't do this on many cameras. They also seem to have gotten cagey about raw dynamic range. It has been a common feature especially on Canon in the past.

http://www.dpreview.com/reviews/nikond7000/14

That is entirely contrary to popular belief.

John
-

[Colin Southern]

Your comment makes me think your and my concept of ETTR are different, as I ETTR in almost every photo that I capture and it's not because of a perceived problem. It's simply to achieve the ideal exposure using the histogram as the guide. If you and I have different concepts about ETTR, that would explain at least part of our disagreement about what is best for Maria at this stage in her learning curve.

I think we'd then have to define "ideal exposure". ETTR basically means "over-expose, but not to the point where highlight information is lost" (so that more shadow information is captured). I'd argue that if you're not needing to reduce the exposure in post-processing then it's not ETTR, if you are (and assuming that it's not a metering issue) then it is.

With modern sensors and predominately reflective scenes at or around base ISO there's just no need for ETTR.

[Colin Southern]

I've tried to find some information on this biasing in the non-linear region of a sensor (perhaps some graphs) but haven't been able to find any through a search. Do you have something on this?

Glenn

You could try Googling for sensor response graphs. I just know it from experience when creating light meter profiles for my cameras (it's evident on the graph).

[Colin Southern]

I don't think we should delve too much in the relative linearity of "linear" devices. They are all basically linear, but there is both a threshold and a roof for their linearity, where the threshold depends on various kinds of noise and the roof depends on the voltage.

The three groups of sensels are working together, and when reaching the roof, one of them will always do so first, then the other and last the third, effectively responding in a non-linear manner. When all three have hit the roof, we say that they have clipped, but colours do clip separately. To me, this is a non-linear region of the sensor. When all three are capturing data below the roof, they are linear. Mostly the red will hit it first, then the blue and last the green, but that order can change depending on the colour of the light that hits them.

Unfortunately, the blinkies will often not kick in until all three are clipped, so relying on them only, may force the sensor into the non-linear region, where the petals of roses and poppies only get a flat red, and where colours mixed with red will not get the right proportions of it. RAW shooters with decent conversion software can see this and respond to it, while jpeg shooters will never know what caused it. Your RAW conversion software might show blinkies for single channels.

I think Urban sums it up well (better than I could anyway). In summary, it's a response area to be avoided; definitely the point at where there's more risk than reward with ETTR (which is usually not beneficial to a real-world reflective photo at base ISO anyway).

[ajohnw]

Interesting suggestion Colin - on histograms I also found this on the same search

http://www.brisk.org.uk/photog/histo1.html

And this that shows the response of a 5D sensor. The D3 is about as well.

http://www.brisk.org.uk/photog/5drespiso100.html

John
-

[Mike Buckley]

ETTR basically means "over-expose, but not to the point where highlight information is lost"

That explains our different thinking, as that's not at all what I thought it meant. That might be my fault. It also might be that there is no industry standard on what it means. I do know that I'm far from the only person who doesn't think of it as over exposing.

[Inkanyezi]

I think we're getting high over the head of the OP here. I also tried making light meter profiles, but I got lost somewhere, when I tried to define response to colour.

When using a white reflective subject (terry towel) to find out dynamic range, I found non-linear response at both ends (in RAW). Also, depending on light source, clipping occured differently in the three respective channels. But as I never had problems with white subjects, I also tried to find out how the meter responded to colour. Basically, it is colour blind, but the image sensor is not. A saturated bright red, which has virtually no green or blue, reads about 1½ stops less than white when clipping. Neither blue nor green channel will clip as readily as red does, if I don't expose it 1.3 stops less than the +2 for a white reading. So to err on the safe side, I set compensation to +0.3 for the red poppy, to have it as bright as possible without clipping and retain structure.

Of course this matches an incident reading perfectly.

When using the method for exposing to the right outlined by Chuck Gardner, one is on the safe side. The towel will show a few very small blinkies in a few places, and one can be bold enough to let it blink a bit more when shooting RAW, but not all over. If shooting jpeg, it shouldn't blink more than a few small dots. On the other hand, it might be simpler to just take an incident reading with a calibrated meter, or a spot reading with your own calibration for the purpose.

I think the white towel raindance is a good excercise for anyone who isn't acquainted to what "exposing to the right" would mean. When you look at the histogram, if it is the brightest subject in the frame, you will find that the histogram ends just in the lower right corner. Whether to do it as a routine could be more questionable, but as a way of learning exposure, it is very pedagogic.

[ajohnw]

The thing that I notice about many of these posts is that the OP doesn't have any histograms to shoot to. Few people have. Only the ones in the preview which are a bit tiny - see the link I posted on that aspect. Canon true but I don't think they are always reliable. It then comes down to are the blinkies reliable. Well on an E-P3 I know that they are very reliable but as yet I don't know about the ones in the E-M5, or the D7000 other than that one appears to be lacking but need to take more shots to be sure.

In a nutshell there is as always a need to use the camera and find out.

Out of interest I have seen one histogram that appears to be useful in a camera. Pure luminance, very large and overlaid on the preview image. Curiously the same camera has highlight and low light spot metering. They were also the 1st to introduce tilting rear screens on dslr's Leaves me wondering why they don't retain all of their their good ideas. Mind you these days they do have histograms in the viewfinder. The colour of the ends change when there is any clipping. I do intend to find out how good that is but often so far I am more worried about the exposure i need - couldn't have it with the levels on either.

John
-

[Colin Southern]

That explains our different thinking, as that's not at all what I thought it meant. That might be my fault. It also might be that there is no industry standard on what it means. I do know that I'm far from the only person who doesn't think of it as over exposing.

It comes down to definitions; Normally an ETTR'ed shot will look over-exposed on the review screen (what I would call over-exposed) but not over-exposed in-terms of blown highlights where the sensor can't record any more detail due to saturation (so not "over-exposed" as far as the camera is concerned).

Probably a good example would be a simple black / white / middle gray card; a normally metered exposure would put the middle gray patch peak in the middle of the histogram - the white patch 2 stops up, and the black patch 2 stops down. That's what I would call a normal or standard exposure. If one decided to "ETTR it" they'd increase the exposure by around 2 stops (the highlights won't blow because there's about a 2 stop safety margin at base ISO anyway) (well there is for Canon DSLRs, and Nikon DSLRs are probably the same). With the exposure increased by 2 stops the middle gray patch will now look white, and the black patch will now look middle gray, with 4 times as much info captured for each level.

Obviously, looking at the review screen, the card will look over-exposed (hence what I would call over-exposed) and it would then be necessary to reduce the exposure by 2 stops in post processing to shift the tones back to where they should be. Kinda like digging a hole and then filling it in again because the only benefit will be less noise in theory, but in practice, it's noise that normally won't be visible anyway (because the noise floor is so many stops down it's practically in another time zone), but - depending on exactly how far the initial exposure was pushed and the colour temperature of the light it was shot under etc, the white patch might not be white anymore, but you can't change the overall colour temp to make the white patch correct because that throws out the other two patches that may already be correct colour wise.

Obviously it's not a major issue with a simple card -- but it IS an issue when one has subtle memory colours like skin tones. Hence my standard lecture about ETTR offering more risk than reward a lot (but not all) the time in that it risks highlight colour issues if not done correctly, all to preserve shadow detail that isn't significantly noise anyway. As (I think it was Urban) pointed out, exposure was more of a compromise 10 years ago, but that was 10 years ago - today's sensors are much better. ETTR (or "dynamic range matching" is probably a better term for it) is really only needed when the dynamic range of the scene is close to that of what the sensor can handle, which in-turn probably only happens most when shooting at high ISOs or with significant backlighting. The rest of the time my suggestion is to just forget about it.

[ajohnw]

I think we're getting high over the head of the OP here. I also tried making light meter profiles, but I got lost somewhere, when I tried to define response to colour.

When using a white reflective subject (terry towel) to find out dynamic range, I found non-linear response at both ends (in RAW). Also, depending on light source, clipping occured differently in the three respective channels. But as I never had problems with white subjects, I also tried to find out how the meter responded to colour. Basically, it is colour blind, but the image sensor is not. A saturated bright red, which has virtually no green or blue, reads about 1½ stops less than white when clipping. Neither blue nor green channel will clip as readily as red does, if I don't expose it 1.3 stops less than the +2 for a white reading. So to err on the safe side, I set compensation to +0.3 for the red poppy, to have it as bright as possible without clipping and retain structure.

Of course this matches an incident reading perfectly.

When using the method for exposing to the right outlined by Chuck Gardner, one is on the safe side. The towel will show a few very small blinkies in a few places, and one can be bold enough to let it blink a bit more when shooting RAW, but not all over. If shooting jpeg, it shouldn't blink more than a few small dots. On the other hand, it might be simpler to just take an incident reading with a calibrated meter, or a spot reading with your own calibration for the purpose.

I think the white towel raindance is a good excercise for anyone who isn't acquainted to what "exposing to the right" would mean. When you look at the histogram, if it is the brightest subject in the frame, you will find that the histogram ends just in the lower right corner. Whether to do it as a routine could be more questionable, but as a way of learning exposure, it is very pedagogic.

Most camera light sensors are rgb to get round the colour problem as much as they can. f you want the view from a person who actually works on them try this, actually I think they are up to 2k sensors now, not sure. There is a lot of software making decisions about metering. Find some one who is hacking Canon code and talk to them.

http://chsvimg.nikon.com/history/scenes/14/index.htm

John
-

[Inkanyezi]

It comes down to definitions; Normally an ETTR'ed shot will look over-exposed on the review screen (what I would call over-exposed) but not over-exposed in-terms of blown highlights where the sensor can't record any more detail due to saturation (so not "over-exposed" as far as the camera is concerned)....

That is not my experience of exposing to the right. On the contrary actually. All cameras I have used will blow highlights in clouds or small bright white areas, and when I expose those to the right, usually the exposure is less than the evaluative metering in the camera. So an ETTR shot of a subject that needs it will come out darker than one shot with evaluative metering without any compensation. There's mostly no problem with large white areas, but when they are a small part of the image. The simplest way to cope with it is to draw the curve just as Nikon's D-Lighting, lifting its left side somewhere close to the end. Exposing to the right will actually do just the same as Active D-Lighting, holding back exposure to save highlight and at the same time lift shadows. Evidently there's contrast loss in middle tones and highlights, and the image will cover a larger dynamic range than the standard treated jpeg. Often this strategy works well. However it isn't a one size fits all.

Of course exposure to the right is unnecessary and could be counter-productive with a subject that spans much less than the sensor can capture, and that has no bright highlights. Then one might have to make the image darker in PP, even though a normally exposed image would have no issue with noise. If, due to colour, the sensor then is in the non-linear mode, one colour clipping, there will be an uncorrectable colour cast.

[Mike Buckley]

Now that I see the confusion, I would be willing to bet that the term, ETTR, has as many definitions as the terms, low-key and high-key. People are using the terms, thinking that everyone is using the same definition when in actuality people are using very different definitions.

[Manfred M]

Now that I see the confusion, I would be willing to bet that the term, ETTR, has as many definitions as the terms, low-key and high-key. People are using the terms, thinking that everyone is using the same definition when in actuality people are using very different definitions.

Actually Mike, all of the terms are quite clear. Unfortunately, a lot of people done understand and hence misuse them.

ETTR was coined by Michael Reichman of Luminous Landscape in this article back in 2003:

http://www.luminous-landscape.com/tu...se-right.shtml

[DanK]

I hate to weigh in at this point, after all the detailed discussion, but I actually think this can be simplified. Let me try:

1. Moving the histogram to the right increases the S/N ratio. That is often a good thing to do, but it is not always costless.
2. Whether one wants to move it, and how far, is a matter of judgment.
3. One factor that ought to influence #2 is how bad noise is likely to be if you don't shift the histogram. That will depend on a bunch of things, but in particular, the camera body, the ISO, and how much of the histogram is far to the left.

E.g., Colin, your case is low noise (base ISO, good camera, studio lighting), mass in the midtones, and concern with rendering of the midtones, right? So you don't shift. In other cases, noise in the shadows may be a big deal, and shifting makes sense. And, of course, if the histogram shows that the image subsumes most of the dynamic range of the sensor, you can pretty much forget about it unless you intend to combine exposures.

Re camera bodies: I have two, a 50D and a 5D3. The 50D is a much noisier camera, so I worry much less about the location of the histogram with the 5D3, particularly at low ISOs and when the mass of the histogram is not too far down from the right. Consistent with what Colin says, with the 5D3 under good lighting conditions, being somewhat below the top of the histogram doesn't seem to make much if any practical difference. But that is exactly when it shouldn't.

I don't personally care whether people use ETTR to mean "shift to the right" or "shift all the way to the right," except that people may need to clarify their use just to communicate.

For the great bulk of the photography I do, I don't think it needs to be any more complicated than this.

Am I missing something?

[Mike Buckley]

Actually Mike, all of the terms are quite clear. Unfortunately, a lot of people don't understand and hence misuse them.

That's exactly what makes them very unclear. People shouldn't have to look up the original source of a term to determine if it is being used correctly. Not only that, but the term itself is unclear because it's apparently not accurately describing the situation. It seems to me that the term should have been OETTR (over exposing to the right).

As an example of sources of confusion, consider the two following excerpts at Wikipedia separated by only two relatively small paragraphs:

• "The principle is also applied in film photography..."
• "This technique is only relevant for use when shooting in a raw image format..."

[Colin Southern]

That is not my experience of exposing to the right. On the contrary actually. All cameras I have used will blow highlights in clouds or small bright white areas, and when I expose those to the right, usually the exposure is less than the evaluative metering in the camera.

I agree most metering will blow clouds -- but I wouldn't call that type of scene typical ETTR. Typical ETTR is more adding +EC to a reflective scene to remove the typical 2-stop safety margin and get 4x shadow information and 2-stop better signal to noise ratio. If anything, I'd call the scenario you describe as ETTL (shifting the exposure left) (but lets not go there!).