A rather stunning ETTR processing vid; pulls a picture out of basically a white sheet

[Scott Stephen]

http://www.youtube.com/watch?v=wuwfNhyXDGQ&feature=plcp

I had seen this a while back, and I found it again. This person shot something so extremely overexposed that it was just about a white screen, then she brings it back down with Lightroom.
I was flabbergasted.

[Colin Southern]

It's not really anything particularly tricky to do; looking at the histogram there doesn't appear to be any clipping -- so all of the information is in tact - and it's only a "white screen" because our monitors can't display the detail that shifted into those regions.

All that's been done (in essence) is to stretch the tonal range out so that the darks are where they should be.

ETTR can bite one in the bum though -- the histogram in-camera is (a) based on a JPEG (usually with a contrast curve applied) and (b) usually only a composite of the channels that usually doesn't indicate when only 1 channel is blown -- and can often result in parts of the data being captured by non-linear regions of the sensors response curve (ie "eating severely into the safety margin") to the point where you can get a weird colour shift in the highlights that's nigh on impossible to correct in post-production.

I'd venture to suggest that deliberately exposing an image like this at the moment of capture is far from best-practice; and considering the extreme dynamic range capability of modern cameras (usually around 6 more stops than monitors can display and 8 more than can be seen in a paper print) there's just no need for it.

[Carl in Louisiana]

Scott, That was an awesome video! Thanks for the link. Short but lots of very useful info.

[FrankMi]

A very informative video on how to use a number of the tools in Lightroom to achieve various results. Good stuff to be aware of!

[Manfred M]

This is a pretty extreme example of ETTR and I don't think this is necessarily particularly good practice. In this case, the photographer has deliberately thrown away a good deal of the dynamic range her camera is capable of recording and has then synthesized an acceptable image.

Let's all remember what ETTR is all about and that is to use the portion of the exposure with the lowest signal to noise (SN) ratio. There is simply more data in light values that dark values, which means the S/N ratio is higher and a cleaner, lower noise image will result.

The whole concept of ETTR came out in 2002 or 2003, when sensor capabilities were less evolved than they are today. While the math hasn't changed, the S/N ratios are far cleaner in modern cameras, so I'm not completely convinved that the benefits of this extreme example. The author has crammed her exposure into perhaps 20% of the sensor's range. Even with the large amount of data available in a RAW file, I can't but help thinking that a significant part of the resullting image is based on interpolated data.

I'm not sure where the optimal cutoff for this technique is. At some point, the benefits or reduced noise have got to be outweighed by some loss of quality. Certainly the histogram shown in the original ETTR article is nowhere near as extreme as in this video.

http://www.luminous-landscape.com/tu...se-right.shtml

[Glenn NK]

http://www.youtube.com/watch?v=wuwfNhyXDGQ&feature=plcp

I had seen this a while back, and I found it again. This person shot something so extremely overexposed that it was just about a white screen, then she brings it back down with Lightroom.
I was flabbergasted.

Thanks Scott - that was pretty interesting. I've seen a few of her videos before, but not this one. I find Laura Shoe's descriptions very easy to follow and understand.

Actually, the image was not overexposed; it was exposed completely to the right but there was no clipping at all (you likely know that LR will show how much and which channels are clipped if any). If you haven't set your 600D up to get the LCD jpeg to more closely reflect the clipping of the RAW file in LR, I would suggest it. I have my Contrast set to minus 2 on both bodies - some people even use minus three.

But having said that, I was flabbergasted too.

Thanks for finding the video again, and showing us.

I have a few questions/comments on subsequent posts that I don't quite follow:

1) Wouldn't the dynamic range determined by the scene and not the camera? There were no shadows and only very minimal blacks at all to start with in the scene, so there was little dynamic range for the camera to capture. I just shot a pure white computer screen (background for a cad program), and the result was a narrow spike in the middle of the histogram - with no dynamic range at all.

2) How could an exposure be crammed into 20% of the sensor's range? If there is no dynamic range to start with, you're stuck with what you have, are you not?

3) How would the benefits of reduced noise (by using ETTR) be outweighed by loss of quality, and where would the loss of quality come in - I'm not understanding this either.

Glenn

[Colin Southern]

Hi Glenn,

1) Wouldn't the dynamic range determined by the scene and not the camera?

In reality, the effective dynamic range of the scene (ie the bits we want to record detail for) either fits within the sensors capability or it doesn't. If it does (and does so comfortably as is the case here) then in theory the photographer can expose it anywhere in the sensor's range that they want to - but - there are both pros and cons associated with where it's placed; if it's severely under-exposed (with under-exposure in this case being defined as an exposure placed significantly "lower" than were normal metering would put it) then it's being placed closer to the noise floor and thus noise may well be revealed when the exposure is "corrected" in post-processing. On the other hand, if the image is over-exposed (with over-exposure in this case being defined as an exposure placed significantly "higher" than where normal metering would put it) then although one gets it further from the noise floor (a good thing in theory), one also pushes it closer and closer to the non-linear regions of sensor response which will in most cases affect one channel before others - which will inturn introduce a colour cast affecting only highlights that's very difficult to null in post processing.

ETTR is a good-sounding theory, but in practice it's not that cut and dried; if the dynamic range of the scene is modest (as it was in this example) then it's - I'm going to come right out and use the "S" word - stupid to push all the values up 2 or 3 stops to minimise noise that wouldn't have been apparent with a modern camera anyway -- only to have to move them back down the same amount in post-production (to where they were in the first place!) with the associated risks of difficult-to-correct colour casts.

On the other hand, if the dynamic range of the scene is large then one has no option but to use ETTR (assuming that they're not going to shoot a bracket of exposures or use a GND filter). The example contained in the video though is just a seemingly good recovery (even though there's nothing particularly "magical" going on) of an extremely bad exposure.

How could an exposure be crammed into 20% of the sensor's range? If there is no dynamic range to start with, you're stuck with what you have, are you not?

Assuming that there's no clipping going on (and from a technical point of view, no DR compression due to operating in a sensor non-linear region of the response curve) then you're right - it's not "crammed into 20% of the sensor's range" - it's simply "placed in the top XX% of the sensor's range".

How would the benefits of reduced noise (by using ETTR) be outweighed by loss of quality, and where would the loss of quality come in - I'm not understanding this either.

Noise would never have been a problem with a normal exposure in this case anyway, but to answer the question - assuming not operating in non-linear response curve regions and assuming no clipping - then there would have been ZERO loss of data; just the opposite in fact -- ETTR captures MORE data, not less -- but keep in mind that it's capturing it in a linear fashion ... what we see on our monitors has a lot of that discarded due to gamma conversion - BUT - (and as you can see it's a BIG but), just because we can't see it (due to the fact that we're looking at a gamma encoded image - and the fact that our monitors aren't displaying enough levels in that region for us to be able to distinguish - plus the extreme lack of contrast) - DOESN'T MEAN TO SAY THAT THE DATA ISN'T STILL ALL THERE IN THE ORIGINAL GAMMA 1.0 ORIGINAL FILE THAT'S BEING WORKED ON. Or in other words, "it looks more impressive than it really is"; there's no "magic" going on - it's simply shifting the data values back to regions where they look best (and should have been in the first place). Or looking at it another way it's really nothing more than showing how to offset the results of a technique that didn't need to be used in the first place!

It's a bit like shooting someone in the foot to show them how good your first aid skills are!

Hope this helps

[revi]

<snip>
I have a few questions/comments on subsequent posts that I don't quite follow:

1) Wouldn't the dynamic range determined by the scene and not the camera? There were no shadows and only very minimal blacks at all to start with in the scene, so there was little dynamic range for the camera to capture. I just shot a pure white computer screen (background for a cad program), and the result was a narrow spike in the middle of the histogram - with no dynamic range at all.

2) How could an exposure be crammed into 20% of the sensor's range? If there is no dynamic range to start with, you're stuck with what you have, are you not?

3) How would the benefits of reduced noise (by using ETTR) be outweighed by loss of quality, and where would the loss of quality come in - I'm not understanding this either.

Colin took care of most while I was typing the first version

Just about the loss of quality when pushing ETTR too far too the R:
When the light level the sensor see gets close to saturation, the signal/response curve isn't linear any more. In other words,
an increase in light level of 1% will give an increase in sensor response that is <1%. This would in itself not be that much of
a problem, if it would touch the three channels at the same time.

But in most cases, one channel is more intense than the others and only the most intense channel will show the non-linearity.
That means that the ratio between the colours for a given spot in the image changes, and thus the colour. But this will only
appen for the parts that are close to over-exposure. That means we'll get 'localised' colour casts.

As an example, lets take a gray gradient under tungsten light: gray, so all three channels 'should' be the same intensity in the
final image. But tungsten light has a lot of red. So if we expose way too far to the right, we'll blow the red channel first,
followed by the others. But even before we clip the red channel, we'll get in the non-linear zone of the response curve. That
means that the lightest parts of our gradient will 'see' less red than the darker parts. And we'll get a cyan cast in the lightest
parts of our gradient... And as the cast is only in part of the colour, and not in the whole image, it's going to be tricky to
correct.

Remco

[Scott Stephen]

Just to clarify a little, since my OP was so brief: I found this while searching for Lightroom tutorials. I think LR is a wonderful tool and I am certain I still only use a fraction of what it is capable of doing.
I was not looking for ETTR or ETTL information at all, and I am certainly not arguing in favor of ETTR (nor against it, for that matter).
I stumbled upon another short vid which happened to be ETT-LEFT in the extreme, the main point of which was to show viewers why they should use RAW. If I ever find that again, I may post it for its PP information (and for pointing out the value of shooting RAW).

[Manfred M]

Glenn - Interesting updates. I noticed I made one omission in my original response that would put my original comments and questions into a clearer light.

The amplifier section of a sensor is analogue and the main characteristic of any amplifier is that its response curve is that it is not completely linear. From what I recall from those electrical engineering courses from long ago is that when you measure amplifier output, the extremities of the curves are distinctly non-linear, but the middle portion does exhibit fairly linear behaviour.

From a practical standpoint, that means the best and cleanest part of the range is really the middle part of the range and the extreme darks and extreme lights are where the non-linearity will start showing up.

This means that while you are theoretically reducing the amount of noise in the image shown in the video through ETTR, when you go to the extreme in this example, you are potentially trading off a theoretical noise reduction for colour fidelity issues. Noise tends to be more of an issue with dark colours (higher S/N ratio), the image in question really does not have a lot of dark tones, so the extreme approach is probably doing more harm than good. Bring the exposure closer to the mid-point of the histogram, you are going to be using the more linear part of the amp’s range, giving you truer colours, at the expense of some noise reduction that may not be required.

The example does show that you can recover a lot of detail from an overexposed RAW image, but it does not cover off potential side effects of doing so.

[Glenn NK]

This means that while you are theoretically reducing the amount of noise in the image shown in the video through ETTR, when you go to the extreme in this example, you are potentially trading off a theoretical noise reduction for colour fidelity issues. Noise tends to be more of an issue with dark colours (higher S/N ratio), the image in question really does not have a lot of dark tones, so the extreme approach is probably doing more harm than good. Bring the exposure closer to the mid-point of the histogram, you are going to be using the more linear part of the amp’s range, giving you truer colours, at the expense of some noise reduction that may not be required.

I would humbly suggest that the reduction of noise by using ETTR is NOT theoretical (particularly when I frequently need ISOs in the range of 640 to get a reasonable combination of f/stop and shutter speed - many botanicals seem to hide in shady places ).

I agree that the effects of non-linearity are not theoretical either, but are they significant? Is there some data to support this, as I've never seen/heard any comment on this aspect.

So, perhaps it comes down to which effect is most significant (or deleterious).

There are so many experts in the field of digital photography and imaging that are advocating ETTR, whilst (I love that British word) very few are against it (I actually found one).

I'm going to go with the flow. I've been using ETTR since I first heard of it and I cannot find or see any colour fidelty issues. If colour fidelity issues become recognized and found to be significant, I will stop using ETTR.

Glenn

[pnodrog]

Below is a indicative illustration of zones compared to the reasonably linear response of a silicon sensor with a 12bit AD converted output. All the zone charts I have seen show equal length steps with the tone values doubling with each step. The illustration varies the length to reflect the space tone values may occupy related to the possible data conversion.




From a numeric point of view there seems to be tremendous room for detail to be held in zones 9 and 10 so it appears that ETTR is a valid approach. Theoretically zones 9 and 10 occupy even more room than is shown but it conveys the concept of the highlight headroom providing clipping does not start to happen.

Warning – the above assumes the AD conversion is linear and stored as RAW values but to speed up the conversion manufactures may not bother to completely resolve the higher values or may use a reference curve during the AD conversion. Why use so much time and storage for highlights as it is basically redundant precision? Examining the RAW file may give a clue.

I am curious as to how the designers approach it, so if any of you can provide factual insights I would be pleased to know.

[revi]

I would humbly suggest that the reduction of noise by using ETTR is NOT theoretical (particularly when I frequently need ISOs in the range of 640 to get a reasonable combination of f/stop and shutter speed - many botanicals seem to hide in shady places ).

But why would you use ETTR when increasing the ISO? Basically, ETTR is over-exposing wrt your meter reading (or rather correcting your meter reading), so you should be able to get the same effect by lowering your ISO value, leaving time and F-stop the same (and then NOT correcting your meter reading, of course).
Example: say you over-expose 1 stop @ ISO 640, so why not use ISO 320 with the same time/F-stop as you would use for ISO 640?

And, is 640 a multiple of your camera's base sensitivity? I think it was Colin who wrote about the way intermediate sensitivities were handled by the camera, and that it could increase the noise level in the final image.

[Glenn NK]

I started photography shooting with Kodachrome. In order to get decent prints from slide film, overexposure had to be avoided, so we tended to underexpose. So I'm quite familiar with the tendency of users in my age bracket to avoid higher exposures - if anyone had suggested ETTR in 1962, he would have been "put away". I sometimes wonder if this tendency to avoid high exposure isn't still around. I also wonder if many judge exposure and colour by the image on the camera's LCD - I think this would be a serious mistake.

ETTR is NOT overexposing - it's exposing so that the important highlights DO NOT clip. Specular highlights are a special case (sunlight reflecting off water is particularly problematic) - I find that specular highlights clip no matter what the exposure, so I don't try to save them. My eyes can't see any detail in the specular highlights of wave-tops so I don't worry about them.

Using ETTR, the RGB histogram on the camera may show slight clipping, but the RAW file should not be clipping. I set the in camera contrast to minus three - this results in an LCD JPEG that more closely reflects what the RAW file will show (not perfect but close). I only shot RAW - this approach probably will not work with JPEG.

I use the meter as a guide only; I take a shot, check to see where the exposure is on the RGB histogram, and adjust the camera to push the exposure to the right. If the next shot clips, I adjust the exposure again. These aren't action shot images - they are macros shot with a tripod where time is the least of my worries. Sometimes I erase three images on one setup before I get the exposure where I want it. As long as the images are not clipped, they are not overexposed. If I'm taking "action shots" of my granddaughters, I don't use ETTR (to avoid blowing out a face).


The following are the noise plots for my cameras:

https://www.box.com/s/zj3mfdcqclw9omxpfkb3

https://www.box.com/s/bpvazy6awia6oibdaftu

This may explain why I use 160, 320, 640 and 1250.

Glenn


EDIT:

Someone posted this on another forum yesterday:

http://www.petapixel.com/2011/05/02/...ng-dslr-video/

And it was suggested that it should apply to still images as well as video (both use the same sensor, etc. don't they?).

[Inkanyezi]

But why would you use ETTR when increasing the ISO? Basically, ETTR is over-exposing wrt your meter reading (or rather correcting your meter reading), so you should be able to get the same effect by lowering your ISO value, leaving time and F-stop the same (and then NOT correcting your meter reading, of course).
Example: say you over-expose 1 stop @ ISO 640, so why not use ISO 320 with the same time/F-stop as you would use for ISO 640?

There is a lot of confusion about meter readings and what ETTR actually means. For us old farts that started out with the zone system half a century ago, it is a bit clearer than mud, but for most people that didn't ever think about zones, a meter reading represents an exposure, and setting your exposure to a longer time or larger aperture would be over-exposing or exposing more to the right.

But your averaged meter value could very well be exposing to the right, while any adjustment to more exposure would inevitably blow the highlights. Or, on the other hand, if your averaged reading will actually cause highlights to blow out, as often happens, you must adjust your exposure to minus in order to expose to the right, ETTR.

It is a bit simpler to understand if you accept at least the theory of zones and use spot metering. When you spot meter a middle tone, your reading should be close to an actual "correct exposure", although without visualising the image, you cannot know. When you wish to expose to the right, you must know just how much plus compensation a spot reading will need, to place it as far right as possible without running over. So you point your spot meter to a highlight where you wish to retain structure, and compensate with a plus value that truly represents this placement in the zone system. This actual value may differ at different ISO settings, but generally, there is a particular number of f-stops needed to compensate, in order to place your highlight as close to the right side of the histogram as possible, without flowing over.

With a higher amplification of the analogue output, there is a possibility that the number of f-stops to correct will decrease, so that if you for example may use +3 compensation at base ISO of 200, you can still do so up to let's say ISO 800, but must then gradually decrease the plus compensation with higher ISO numbers, so that you might have to use only +2 at ISO 3200 to expose to the right.

Be aware, that it is not the amount of light that makes your photon wells flow over in conversion, but amplification of their signal, and this signal is certainly within the linear range when overflow occurs. Hence you cannot, as suggested get this effect at a lower ISO and the same exposure, because a lower ISO will mean that you amplify the signal less, and your exposure will not be to the right, but fall farther left in the histogram. It will certainly still be in the linear region, and the s/n ratio will be about the same, although you will have less data in the image and waste a lot of data on the unexposed right highlight region of the histogram. Visually, your image might be the same anyway, as there is certainly enough data also in the middle of the tone scale.

If you apply it on the image in the video, you could have exposed it at least two stops less, and it would have required less fiddling to find an image in the rather contrast-less frame. Expanding the histogram would be easy with the curve tool only, and the end result would be about the same visually.

I think that the example is not very typical of an image exposed to the right. Far more often, the total amount of values in the scene exceeds what can be registered on the chip, and you might have to decide whether to deliberately blow hightlights to save shadow detail or choose to make your deep shadows completely black, in order to retain tonality in the brightest areas. If you have one of the better sensors, you might not have to choose one or the other, but can do all the wizardry in your computer.

And even though these possibilities exist when working from a linear raw file, I still think the zone thinking has a value in my everyday application of photography. I can mostly get clean and nice images in jpeg directly out of my camera by knowing just how different values of exposure will register in the final image. Contrary to the suggestion that you could use lower ISO, whenever I think that I can stretch out tonality to get more punch from rather poor contrast, I can also increase ISO, as there is no evident loss, as long as I keep everything sufficiently far to the right without raising the noise floor too much.

[Manfred M]

Glen – Obviously I was not clear enough in where I was coming from; I’m not arguing the fact that ETTR will tend to produce more noise-free images, but rather what other impact to overall image quality will be introduced by shooting that way, especially when the data seems to be at the extreme right side of the histogram.

Looking at the link to some of Jeff Schewe’s analysis and work and the one thing that struck me about his images where he was shooting very far to the right that while the image looked interesting artistically, the colours did not look right. I suspect that this is probably a result of some of the data coming from the non-linear part of the sensor output.

In most devices, the highest level of linearity / accuracy tends to be in the middle portion of the device’s range and accuracy tends to drop at the extremities. So biasing the results so that they are weighted to the right of the histogram’s centre line would make sense, but how far is too far?

The underexposure you refer to (and my heavy duty film days were in the early 70s) was also to improve saturation a bit and shooting at 1/3 of a stop underexposed tended to result in landscape shots with a bit more “pop”. You are quite right, reversal films were very unforgiving when you overexposed them.

[pnodrog]

The underexposure you refer to (and my heavy duty film days were in the early 70s) was also to improve saturation a bit and shooting at 1/3 of a stop underexposed tended to result in landscape shots with a bit more “pop”. You are quite right, reversal films were very unforgiving when you overexposed them.

Manfred I always exposed Fuji velvia film a 1/3 of a stop under and was pleased with the look until I wanted to scan them many years later. (Dmax far to high)

Came across an article on the direction of sensor technology as viewed by Aptina.

http://www.aptina.com/products/technology/DR-Pix_WhitePaper.pdf

[Manfred M]

Thanks Paul - it's an interesting read. Aptina obviously has this in development for some time (the white paper has a 2010 publication date). It will be interesting to see how this pans out in a real life photography setting. They are cramming more electronics into an already crowded sensor component, so that will limit the minimum size of the individual photodiode site. A tradeoff like this could result in degradation of the normal light performance, so I wonder how the image quality will measure up against the competition. With the extra circuit elements, I would think that this would introduce an additional sources of current leakage, i.e. noise into the circuit.

The other thing I wonder about is how the switch from normal to low light operation would be handled. The simplest way would be through operator intervention as I suspect (almost) simultaneous operation would be a bit challanging, certainly for use in normal photography where a high degree of accuracy in colour reproduction is required. Surveillance and other industrial applications are likely to be more tolerant of these types of issues.

[pnodrog]

Manfred the dual sensitivity approach reminded me of what Fujifilm tried by combining different size pixels on the Fuji S2 pro. At the time its dynamic response was better than the rest. It is interesting to watch progress. I think that Aptina do intend to make it simultaneous - I was only quickly reading the paper to try and get a bit more of a general understanding of the technology.

I was trying to find out what approach sensor designers are taking to adapt a basically linear detector response to better match the requirements of photography. In the paper was a reference to a logarithmic detector but no info as to how it was done. One method I am now aware of is for a linear sensor to have a steep AD conversion slope for the darker values, a transition slope for mid tones and a flatter slope for the highlights. So the RAW data was composed of values derived from three distinct slope changes not a curve. Switching to a new references during conversion is probably faster and more precise than trying to generate a variable reference curve.

[revi]

<...>

Someone posted this on another forum yesterday:

http://www.petapixel.com/2011/05/02/...ng-dslr-video/

And it was suggested that it should apply to still images as well as video (both use the same sensor, etc. don't they?).[/B]

That was basically what I was thinking of when asking about your ISO setting. Note what they say there also: for most cameras
the base ISO is 100, Canon being an exception with base at 160 (as seen in the graphs you referenced).

As a side note, I find the opening image a touch misleading, in that the base gray gets a lot lighter with higher ISO values (in the video itself
the background is more constant, and the noise less visible!).