Need some help - overexposed RAW/DNG

[ScoutR]

Hi everyone:

I just switched from shooting in jpg format to RAW and then converting to DNG when importing from the camera. I find that I am having to decrease exposure on almost every shot. Up until now whenever I have been out it has been pretty dull and foggy, so I just blamed the weather. Today however I had some really good light, and again, every shot has come out overexposed. I am pretty sure I can fix them up, but before I start fooling around willy nilly (like I've been doing up till now) I have a few questions.

1. Is this kind of thing to be expected when shooting in RAW format? Is it normal to have to make exposure adjustments.

2. Seeing as I am working with an uncalibrated screen, I need to know if I can trust the histogram. For instance, if the histogram says the photo is overexposed, then it is overexposed, and my screen calibration has nothing to do with what the histogram says.

3. Might I be better off to keep the file in RAW format rather than convert to DNG? I will experiment with this anyway, but would like some opinions.

4. The photo below is an example. This was shot in RAW format converted to DNG and then resized and exported as a jpg so I could post it here. I did not do any PP. On my screen it is overexposed and the histogram bears this out. Every shot from today is the same - way overexposed and washed out.

I'm not sure if you can get any metadata information from the file as posted. If you can, is there anything in there that you can see that might help me avoid this in the future. If you can't see the metadata, do you know how I can get it to show up so you can take a look.



Thanks
Wendy

[Colin Southern]

Hi everyone:

I just switched from shooting in jpg format to RAW and then converting to DNG when importing from the camera. I find that I am having to decrease exposure on almost every shot. Up until now whenever I have been out it has been pretty dull and foggy, so I just blamed the weather. Today however I had some really good light, and again, every shot has come out overexposed. I am pretty sure I can fix them up, but before I start fooling around willy nilly (like I've been doing up till now) I have a few questions.

1. Is this kind of thing to be expected when shooting in RAW format? Is it normal to have to make exposure adjustments.

2. Seeing as I am working with an uncalibrated screen, I need to know if I can trust the histogram. For instance, if the histogram says the photo is overexposed, then it is overexposed, and my screen calibration has nothing to do with what the histogram says.

3. Might I be better off to keep the file in RAW format rather than convert to DNG? I will experiment with this anyway, but would like some opinions.

4. The photo below is an example. This was shot in RAW format converted to DNG and then resized and exported as a jpg so I could post it here. I did not do any PP. On my screen it is overexposed and the histogram bears this out. Every shot from today is the same - way overexposed and washed out.

I'm not sure if you can get any metadata information from the file as posted. If you can, is there anything in there that you can see that might help me avoid this in the future. If you can't see the metadata, do you know how I can get it to show up so you can take a look.



Thanks
Wendy

Hi Wendy,

A few pointers ...

- DNG is RAW; same data, just a different way of organising it. Won't make any difference to the issues you're facing.

- The in-camera histogram is based on the in-camera JPEG thumbnail conversion - it usually has a fairly strong contrast curve applied and thus can't be relied upon overly for over-exposure decisions.

- Be sure to turn on highlight alert so that areas of blown highlights blink on the cameras review screen - it's worth it's weight in gold.

- Be sure that the issue you're facting is over-exposure. In the image example you posted the exposure looks OK as best I can tell, but it is flat looking due to an incorrect black point being set. (Send me the RAW file for a final analysis though if you like).

- Be careful about the metering mode that you use. For scenes like the above you need to be using Evaluative.

- Keep in mind that you may need to dial in a bit of EC ("Ecposure Compensation") from time to time.

- Be sure that you haven't accidentally applied some EC.

- What shutterspeed / ISO / F-Stop was this shot at?

Hope this helps

[Dave Humphries]

Hi Wendy, Colin,

I checked the EXIF data above;
Nikon D3000 - Wendy, if same as my D5000; the "blinkies" Colin refers to are available via one of the reveiw screen modes on the camera (called "Highlights"), you cycle through them with the up/down pads (left/right does previous/next image).

1/60s @ f16, iso200 at 55mm (82mm FF)

Auto Exposure
No Exposure Bias (EC) is applied
Metering on Multi segment (not spot)
Not sure if you had any scene modes applied though, as that may not show up here and it could cause this trouble.

The pic doesn't look like it should need anything special in terms of EC, wierd. Must dash.

Cheers,

[ScoutR]

- The in-camera histogram is based on the in-camera JPEG thumbnail conversion - it usually has a fairly strong contrast curve applied and thus can't be relied upon overly for over-exposure decisions.

Thanks Colin: Just to make sure we are on the same page, I'm referring to the histogram when I view the photos in Lightroom. The reason I want to make sure I can rely on the histogram, is so that I can make adjustments based on the histogram regardless of what my uncalibrated screen shows. For example if my screen is too dark or too bright I would overadjust, but if I can use the histogram as a tool, it will help me decide if my screen is way off. (I think, maybe, it might....)

Be sure that the issue you're facting is over-exposure. In the image example you posted the exposure looks OK as best I can tell, but it is flat looking due to an incorrect black point being set. (Send me the RAW file for a final analysis though if you like).

See below for a snippet of the original histogram and exif data, which I fianlly found. I really appreciate any help you can provide. I can send you the original also if that will help.

- Be careful about the metering mode that you use. For scenes like the above you need to be using Evaluative.

I can see from the Exif data that metering mode was set to "pattern" I will pay more attention to this in the future. Kelby's Digital Photography books are on my priority list.

- Be sure that you haven't accidentally applied some EC.

looks OK there, no EC was applied.

Below is an adjusted version and the histogram to go with it. I think I might have gone too far in the other direction, but to me and on my screen it looks more like what I saw when I was taking the picture.

[ScoutR]

I checked the EXIF data above;
Nikon D3000 - Wendy, if same as my D5000; the "blinkies" Colin refers to are available via one of the reveiw screen modes on the camera (called "Highlights"), you cycle through them with the up/down pads (left/right does previous/next image).

Thanks Dave, I will check this out.

1/60s @ f16, iso200 at 55mm (82mm FF)

Auto Exposure
No Exposure Bias (EC) is applied
Metering on Multi segment (not spot)
Not sure if you had any scene modes applied though, as that may not show up here and it could cause this trouble.

Good Grief man, how do you find this stuff. I just spent half an hour looking for the very same info in Lightroom. Anyway at least I know where to find it now.

The pic doesn't look like it should need anything special in terms of EC, wierd.

That is wierd. When you have a minute, please take a look at the adjusted version that I posted and let me know if it seems dark to you. Everyones feedback on this would help me a lot. It would really suck if my monitor was out by so much. I went through the advanced calibration, and according to all the supplied graphics, it's about as close as I can get to what they are asking for in the guides.

Really appreciate your help and feedback on this.

Thanks for taking the time
Wendy

[Colin Southern]

Hi Wendy,

With digital photography there is often a difference between "technically correct" and "visually correct" (in terms of captures).

Digital sensors are linear devices and what that means is a full HALF of all information captured is captured in the top 1 stop of the sensors range; so in most cases (there are exceptions, but I won't go into that much detail at this stage) you want the histogram to be touching the right-hand edge (as it appears to be) - but - without a "spike" right at the end (which you don't have).

However - if you're capturing, say, 11 stops of dynamic range - but your scene only needs about 4 to describe it (a typical reflective scene), then you can end up with what should be shadows biased more towards midtones, which from a technical point of view is great -- it just doesn't look very good!

Do a test - open the file in the RAW converter - don't touch the exposure, but bring the blacks slider up, and the brightness (which is essentially a mid-tone adjustment) down - and I'll bet that your image will look pretty good (if so then you don't have over-exposure with this image).

If you're looking for VISUALLY correct then you might have to dial in a bit of EC - but from a technical point of view this will throw away a LOT of information (that in this case you probably won't need, but if you had to use a high ISO and you threw away a lot of info you could well end up with a lot more noise).

If you'd like to send me the file though I can have a much better look at it. Also - if you want to do accurate work, then there's just no getting away from a monitor calibrator and profiler.

In the meantime though you might like to check out Seans black/white scales to give you an idea of how your black/white points are set.

[ScoutR]

With digital photography there is often a difference between "technically correct" and "visually correct" (in terms of captures).

Digital sensors are linear devices and what that means is a full HALF of all information captured is captured in the top 1 stop of the sensors range; so in most cases (there are exceptions, but I won't go into that much detail at this stage) you want the histogram to be touching the right-hand edge (as it appears to be) - but - without a "spike" right at the end (which you don't have).

However - if you're capturing, say, 11 stops of dynamic range - but your scene only needs about 4 to describe it (a typical reflective scene), then you can end up with what should be shadows biased more towards midtones, which from a technical point of view is great -- it just doesn't look very good!

So does this mean that there will probably always be certain adjustments that have to be made to every photo from the same shoot. For instance every photo yesterday had the same washed out quality. If I took 1 photo and made the adjustments then I would just apply the same to all the photos. After that each 1 might need individual tweaking depending on the photo, but it's considered normal when shooting RAW that there might be, depending on conditions, certain adjustments that will have to be applied.
If so, would this be considered equivalent to what was done with film at the processing stage, but with film, unless you were doing your own processing you never got to see what things looked like right out of the camera?

Do a test - open the file in the RAW converter - don't touch the exposure, but bring the blacks slider up, and the brightness (which is essentially a mid-tone adjustment) down - and I'll bet that your image will look pretty good (if so then you don't have over-exposure with this image).

Yes, this helps a lot. It's very close to what I got when decreasing exposure. I still find that a minor decrease in exposure helps, but not the full to 1.5 stops that I was applying yesterday. Perhaps next time if I set my camera to the proper exposure mode for what I'm shooting I will get better results. . I hope...

I don't mind having to do some tweaking and if I'm understanding what you said in the first couple paragraphs, it sounds like it is to be expected, but these photos from yesterday were very disappointing. I was so excited that the sun (finally) came out and the lighting and colours were really very dramatic. Then I looked at the photos, and it's just not what I saw.

If you're looking for VISUALLY correct then you might have to dial in a bit of EC - but from a technical point of view this will throw away a LOT of information (that in this case you probably won't need, but if you had to use a high ISO and you threw away a lot of info you could well end up with a lot more noise).

In the future I will be paying more attention to getting the proper exposure. I really don't want to loose information, even if I can afford to.

If you'd like to send me the file though I can have a much better look at it. Also - if you want to do accurate work, then there's just no getting away from a monitor calibrator and profiler.

I don't know if this will work, but I have uploaded the original to my ftp site and provided a link. Let me know if you can work with this.
http://ftp.execulink.com/~wfsweb/Images/DSC_0183.dng

Regarding the monitor calibration, I hear you and it really does not make sense to do all this work, only to find that all adjustments are wrong for printing or viewing on any other monitor. That's just silly, but... based on the monitor calibration guides I've seen so far - the one one this site, and the advanced calibration that comes with my video card - calibrating the monitor would be like asking me to tune up my car.
I have a Viewsonic VX2433wm and it does not seem to matter what I do regarding brightness and contrast, the middle square in the first step of the cambridge tutorial you (and Dave) have pointed out to me always looks darker than the background. I do see a difference between the 3 grids but I cannot get the middle one to blend in and match the background.

Sorry for rambling on, but it's really important to get this sorted out before I go much farther with this. I think you know this monitor. I realize it is not a professional monitor, but do you think there is enough room for calibration on this monitor, that it would be worth it for me to find a professional to do it for me?

Wendy

[Dave Humphries]

Hiya Wendy,

I would say the first histogram does look overexposed to me.

The revised picture looks much better, although there's something not quite right with the foreground bushes though, possibly a touch magenta in WB, but then of course they are in shade and hence a different colour temperature to the sunlight farm buildings - which look ok. Histo #2 looks OK to me too.

It is quite a nicely sharp picture too, although perhaps just a tad too much noise, I find my D5000 can show it in the sky too under certain conditions (like these) and other times it's unnoticeable.

Cheers,

[ScoutR]

Thanks Dave: I am so relieved to know that the second one looks better. At least I know I'm heading in the right direction and that my monitor is not totally off.

Perhaps the noise is because I decreased the exposure instead of adjusting the Black Point and Brightness as Colin suggested. Lots of things to work on. I'm still experimenting to see how the camera records things and then (with everyones help here) trying to make sure that I'm making things better and not worse.

Thanks again
Wendy

[Colin Southern]

I would say the first histogram does look overexposed to me.

I downloaded the raw (?) file (was DNG file extention, but said TIFF, so not sure if it's really a RAW) and it was over-exposed (as indicated by highlight alerts and loss of detain in a small portion of the sky and and building roof).

I just wanted to draw the distinction between what's ideal technically -v- visually ...

The histogram bunched up to the right like that is unlikely to ever be correct from a visual point of view, but that's not particularly significant because that's what post-processing is for. But from a perspective of an ideal capture it's great, so long as the highlights aren't blown (which you often don't get a great indication of from the histogram) (with the above being a great case-in-point).

I guess what I'm trying to say (in 1,000,000 words or less) is "yes - this was over-exposed, but we can't always assume that a shot is over-exposed from an "ideal capture" perspective just because the histogram is bunched up to the right. All it really means is most of the values in the scene are captured in a region where there's more bits available to describe them you're further away from the noise floor - both of which are good things

Interestingly, the shot was a perfect "sunny 16" exposure - so I suspect that the blown highlights are due to specular reflections from the roof and direct (-v- reflected) light from the clouds (sorta kinda).

Bottom line is if you want to expose scenes like this from a visual perspective (ie scenes with small portions of bright light) then you'll need to dial in -ve EC.

[Colin Southern]

So does this mean that there will probably always be certain adjustments that have to be made to every photo from the same shoot. For instance every photo yesterday had the same washed out quality. If I took 1 photo and made the adjustments then I would just apply the same to all the photos. After that each 1 might need individual tweaking depending on the photo, but it's considered normal when shooting RAW that there might be, depending on conditions, certain adjustments that will have to be applied.

The correct answer is "it depends" - sorry to be vague, but it depends on how far from visually ideal the shots are - how close you want them to be, and if the lighing in the scene is changing from shot to shot. Light bounding off objects in a scene like this won't change a lot - but - if you're getting specular reflections (eg "bright glare / direct reflections") or bright cloude in the shot then these ARE likely to change from shot to shot.

In the future I will be paying more attention to getting the proper exposure. I really don't want to loose information, even if I can afford to.

The "number 8 wire approach" as us Kiwis call it is simply to dial down the exposure if you see blinkies, and increase it if the scene looks dark on the camera and the histogram stops well short of the right hand side.

Regarding the monitor calibration, I hear you and it really does not make sense to do all this work, only to find that all adjustments are wrong for printing or viewing on any other monitor. That's just silly, but... based on the monitor calibration guides I've seen so far - the one one this site, and the advanced calibration that comes with my video card - calibrating the monitor would be like asking me to tune up my car.

It's not actually as hard as it sounds - at a basic level, all you need to do is follow the instructions that come with the package.

Sorry for rambling on, but it's really important to get this sorted out before I go much farther with this. I think you know this monitor. I realize it is not a professional monitor, but do you think there is enough room for calibration on this monitor, that it would be worth it for me to find a professional to do it for me?

I use one at home and it calibrates / profiles just fine. You could pay someone to do it (I suppose) - so long as you were convinced that they knew what they were doing. Personally though, I'd suggest just buying a spider III and "be done with it" We should be able to help you with the process if you get stuck.

[mrsoto]

Hello
Here is a test using RawTherapee nothing from Adobe

I believe that the color temperature registered by the cameras is incorrect, in this case I did use clouds as a gray card

This photo has all the information you need but blacks are little white

Later,
mrsoto

[mrsoto]

Look at sky. Your sensor appear to by dusty

[Colin Southern]

Hello
I believe that the color temperature registered by the cameras is incorrect, in this case I did use clouds as a gray card

To my eye your version looks a little too warm.

@ Wendy - I've given it a Photoshop 30-Second Makeover (TM!) - I've mucked the sharpening up a little, but hopefully it still looks OK.

[ScoutR]

I downloaded the raw (?) file (was DNG file extention, but said TIFF, so not sure if it's really a RAW) and it was over-exposed (as indicated by highlight alerts and loss of detain in a small portion of the sky and and building roof).

I will have to investigate the TIFF thing. I noticed it too when I tested the link. Thought it was something on my end, but you got the same results. DNG extension but said it was a TIFF. I will check it out. I'm thinking (hoping) something with my ISP changing the format because it was such a large file. Either that or I've done something in Lightroom that is messing with the formats. Lightroom looks pretty staitforward though, so I don't think it's there.

I just wanted to draw the distinction between what's ideal technically -v- visually ...

The histogram bunched up to the right like that is unlikely to ever be correct from a visual point of view, but that's not particularly significant because that's what post-processing is for. But from a perspective of an ideal capture it's great, so long as the highlights aren't blown (which you often don't get a great indication of from the histogram) (with the above being a great case-in-point).

I guess what I'm trying to say (in 1,000,000 words or less) is "yes - this was over-exposed, but we can't always assume that a shot is over-exposed from an "ideal capture" perspective just because the histogram is bunched up to the right. All it really means is most of the values in the scene are captured in a region where there's more bits available to describe them you're further away from the noise floor - both of which are good things

Very helpful, and I understand what you are saying. You should write a book.

Interestingly, the shot was a perfect "sunny 16" exposure - so I suspect that the blown highlights are due to specular reflections from the roof and direct (-v- reflected) light from the clouds (sorta kinda).

Aaahaaa! I had to look up "Sunny 16" but according to the rule exposure should have been f16 @ 1/200. I was using aperature priority f16 and camera gave me 1/60. That explains a lot to me and also, hammers home the importance of setting the proper exposure mode and then exposing for the right part of the scene. It's one thing reading all this stuff, but for me practical examples work wonders. Next Project - memorize exposure settings on Camera and be able to find them automatically!!!!! (farsighted and don't want to take glasses on and off )


Thanks
Wendy

[ScoutR]

The "number 8 wire approach" as us Kiwis call it is simply to dial down the exposure if you see blinkies, and increase it if the scene looks dark on the camera and the histogram stops well short of the right hand side.

I couldn't find a definition for the "number 8 wire approach" but I will be checking out the "blinkie' thing on the camera as soon as I get offline.

I use one at home and it calibrates / profiles just fine. You could pay someone to do it (I suppose) - so long as you were convinced that they knew what they were doing. Personally though, I'd suggest just buying a spider III and "be done with it" We should be able to help you with the process if you get stuck

.

Well, I'm feeling a bit better about the monitor calibration now. I was stressing a bit when I got feedback that the first photo looked fine. It looked so bad on my end, I figured I had the calibration thing all screwed up. Based on subsequent feedback, I'm thinking that I am in the ballpark with the calibration tool that came with the video card, and I feel a little more confident about trying out the Spider III when I can work it into my budget.

I have a couple more posts to reply to and then I'm off to check the blinkies

Wendy

[ScoutR]

Look at sky. Your sensor appear to by dusty

Thanks MrSoto, I see the spots and I will check it out.
Thanks also for taking the time to do the rework. I like what you have done, and the price of the software is certainly right. I already have Lightroom though, and need to get up to speed on it. My mind would crumble if I tried learning anything else right now.

Thanks again
Wendy

[ScoutR]

[/QUOTE]

@ Wendy - I've given it a Photoshop 30-Second Makeover (TM!) - I've mucked the sharpening up a little, but hopefully it still looks OK.

WOW! Now that is bright, but certainly more like what i saw as opposed to the original shot. The colours are soooooo much better. I think I'd keep Mrsoto's sky though.

Aside from the sharpening did you just work with the Black point, brightness and contrast? I don't do sharpening yet, so don't even try to explain it to me. It's one of those controls that seems to be pretty sensitive so I leave it alone. I just let Lightroom do the sharpening when I export to jpg.

Thanks to You, Dave and Mrsoto (hope I didn't miss anyone) for all the time you spent on this. Many valuable lessons have been learned and I do appreciate it very much.

Wendy

[Colin Southern]

Aaahaaa! I had to look up "Sunny 16" but according to the rule exposure should have been f16 @ 1/200. I was using aperature priority f16 and camera gave me 1/60. That explains a lot to me and also, hammers home the importance of setting the proper exposure mode and then exposing for the right part of the scene. It's one thing reading all this stuff, but for me practical examples work wonders. Next Project - memorize exposure settings on Camera and be able to find them automatically!!!!! (farsighted and don't want to take glasses on and off )

Oops - quite right - not sure how I managed to come up with 1/60th being "sunny 16" @ F16 @ ISO 200

[ScoutR]

Oops - quite right - not sure how I managed to come up with 1/60th being "sunny 16" @ F16 @ ISO 200

No need for sad faces. I hope you don't think after all the help you have given me that I was trying to point out that you got the rule wrong. Just the opposite actually, the "Sunny 16" rule was appropriate for the scene, and that's what I thought you were saying. After looking up the rule, I came to realize how my sloppy exposure methods caused the the camera to screw up.