Why Auto White Balance isn't

[Manfred M]

Urban / Remco – I’m not quite sure what either of you are arguing here. If you have a dispute with the term AWB, then perhaps you have a point and a better name like “automated colour cast removal tool” might be a more appropriate name for the functionality. If you are suggesting that it is impossible to create an algorithm that works 100% if the time, then I would have to agree with you as well. Does it have anything to do with the actual colour temperature; well perhaps not, because it has the same drawbacks as a reflected light metering system; it is based on some assumptions that do not always hold.

On the other hand if you suggest that a technique that provides an overall neutral colour balance to an image is not a valid approach, then I will have to strongly disagree with you. This technique has been around for a very long time. When I was doing colour darkroom work as a hobbiest over 30 years ago, my Durst Color Analyser worked exactly this way. It would take an overall reading of the colour negative and averaged everything out and allowed me to produce prints with an overall neutral colour. Just like the AWB function in a camera, it would be fooled by specific lighting conditions or specific subject matter. There were spot colorimeters available that allowed one to target skin tones and neutral targets, much like modern PP works. In fact, some of the basic functions in Lightroom and ACR / Photoshop seem to work exactly the same way

Just like the autofocus, metering and many other camera functions, these can be fooled and are not 100% reliable, the same can be said for AWB. That being said, it works adequately for most people, most of the time, so what is the issue? Frankly I have enough shots that work that even working from the basic RAW image does not give me a correct colour balace because of the large number of different light sources; but in the end, the images do work.

[xpatUSA]

[although] that works reasonably well for sunlight and incandescent lamps, it fails for fluorescent and phosphorescent light sources. The latter show a spectrum with peaks and holes (i.e. certain colours are very intense, others are absent).
And the glow-in-the-dark stuff is phosphorescent, and has a discontinuous spectrum. So there is no 'correct' white balance setting, the best you can hope for is getting close to the visual impression on both the glow stuff and the rest of the watch (not guaranteed at all!).

The most common glow material these days is doped Strontium Aluminate, such as Luminova (the kind you hold in a strong light to 'charge up'). As can be seen, the spectral emission curves at right are quite continuous, with no peaks and holes:



OT, but fireflies have quite a broad continuous spectrum:



As has been pointed out, there is no white balance for a light source per se. However, the camera or the image in PP demands a WB to be set, right or wrong. This is quite a dilemma.

As I think about it today, perhaps there is an approach that is more scientific than "getting close to a visual impression". Given a spectral emission curve, is it not possible to determine a device-independent color in L*a*b* space? Perhaps directly from the dominant wavelength, in the case of watch phosphors. Armed with that, adjust image WB until the Lab color picker says the same? Or maybe determine the XYZ values, convert those to L*a*b* and then do it?

BTW, watch lume shots are taken in the dark, or with the rest of the watch in very low light usually with very poor color rendition which is rarely corrected. The glow is the thing!

An 8-day aircraft clock re-lumed with Super-Luminova:

[AB26]

Please define getting it right in camera.

At 08h00 this morning I was doing my own bit of experimenting as a result of Urban’s post. Shooting took place in a little more than a minute. No cloud only direct early morning sunlight.

I set up my camera, with a Sigma 70-300mm lens, on a tripod, removed the UV filter, fitted the lens hood, set up the subject and waited for the sun to rise high enough to shine on the subject. Next, set exposure (Shutter speed 1/1250sec, Aperture F5.3, ISO 200). Shoot Jpg to get the shots SOOC.

Downloaded in PhotoScape then resized and sharpened - that is all.

First shot: Pre-set WB, Second shot: Auto WB, Third shot: 3800K, no 4 at 4200K. Now preview and dot down what my eye tells me what is closest to what the scene looks like in that moment in time.

Five hours later and I look at the images again and I cannot remember what the scene looked like when I did the shoot, It could be either 2 or 4. My notes tell me that Pre-set was closest to what it actually looked like. The most accurate colour rendition of what my eye saw at that moment in time.

That is what I call getting it right in camera. The essence of the mood in that moment in time rendered as close as possible to what it was at that moment.

How do you reproduce the colour in a magnificent scene if you cannot recall what it really looked like? How do you record accurate colours if you do not know how to get it right in camera?

1. Pre-Set


2. Auto


3. 3800K


4. 4200K

[Andrew1]

To my understanding of AWB the first two photos posted by Urban show the camera did exactly what it was supposed to do. It compensated for the average colour of the reflected light in the photo. If you are a professional wall photographer then it is important to note the results for future use. For me it does not represent anything close to my normal subject matter so I can disregard the results entirely. I'm sure a colour cast will raise its head sometime but if I'm shooting jpeg I can see the results right away and adjust as required. If I'm shooting raw I will attend to it later as does everybody else who gets that involved. It's an easy fix early in the routine and then just move on to the next adjustment. AWB by design will work for a majority of the people a majority of the time. The real lesson here is that as with any tool it's up to the user to know how to properly use the tool, not blame it when something does not come out as expected. I recall a similar complaint in another thread about blue hair which was easily explainable if the photographer had an understanding of the process.

[SteveF]

This is an interesting thread. AWB is, in the main, adequate for me outside. However I have the luxury of some high end kit not more than 5 months old. I rather enjoy experimenting with white balance at times and at others I just get so frustrated!

AWB is, or should be better than we find it nowadays. I'm sure the manufacturers have the ability to white balance every damn pixel induvidually or certainly the three RGB ones together in any area of the sensor. Take my S100 Canon P&S for instance. On absolute auto (green box mode) it does white balance for different parts of the scene but put it in program or manual and you lose that ability re white balance.

I rarely shoot raw ( and if you saw me dithering at the computer you would understand why ) and have had this over-riding desire to produce images straight from the camera to print and shooting outdoors I can do that with relative ease. No, I'm not perfect but, if you're shooting AWB pretty much the same type of subject (cycling) week after week you soon learn what works.

So, recently shot a dance school that I'm sub-contracted to locally and truthfully it's a white balance nightmare. 240 students in eight hours with 28 groups on top of individual portraits of every student demands no lights (portable) so we relied on the mirrors and available light. I custon WB every hour during the day and set my camera to that setting as well as shooting jpeg+raw as I knew under the circumstances there was a lot of pressure on to get completed in the day.

The day shots were fine - jpegs popped out with the custom WB doing it's job. AWD might have done just as well and here comes the rub ... I didn't trust the AWB to 'get it right'. Like a lot of you here I don't have the faith in the camera 24/7 in all conditions. But I should have, right? Same reason re the S100 above.

When it came to the evening classes in the week I custom WB under flouros didn't i? Not a bright move, of course! My CWB was going to be at a different cycling of the flouros to the actual picture taking click. So, back to raw and the eyedropper. Cannot even apply the setting globally as they would all have some different timing to the others in the cycle of the wave of herts thingy wave. We're 50hz in the UK. Even using a shutter speed of 1/25th didn't get all the two peaks in judging by some of the colours we got.

To top this off if I have used AWB on site I might have shortcut most of the work as AWB would mostly have gotten some right but I robbed myself of the opportunity because I didn't trust AWB ... so, why is it that manufacturers still have this issue with AWB? Is AWB set at the point of exposure on a half press of the shutter or actually when the shutter fires?

[AB26]

Thanks Steve, for that post. It emphasizes what Urban is trying to make clear in this thread. Auto WB isn’t.

After reading Steve’s post I grabbed my camera and rushed into my pitch dark garage switching on the only fluorescent light in the garage. Mounting the camera on a tripod and taking shots of my wife’s white car.

No matter what I did with WB and Shutter speed, every second shot taken in succession is to blue. Again, more success with pre-set WB. I can get whatever tint I want by changing the WB setting – from 2500K to 3800K (thereafter it gets to yellow) but the WB seems to be all over the place with every second shot. Then I fitted a flash changed WB to flash and walaa, Bobs your Aunty, my wife’s white car is white.

The moral of this story: get to know your equipment and the limitations it has. What I have learned from this exercise is not to attempt a shoot under fluorescent light. You need an alternative constant light to get it right.

If anybody ever has to ask me to do a shoot under fluorescents without alternative lighting I will very politely refuse to do the shoot.

Yahoo, I have learned something – THANK YOU!!!! Now I can go to bed with a feeling of gratitude.

[Manfred M]

Andre - what you are experiencing will occur with older transformer style ballasts where the line (mains) frequency can have the effect that your are seeing. Shooting at a shutter speed where you get two full cycles (1/30 s for North America (60HZ line frequency) and 1/25 s for much of the rest of the world (50 Hz line frequency) will mitigate this issue.

Modern electronic ballasts run at much higher frequencies (200Hz to 20 000HZ) and you should not run into this issue, but of course, there are still a lot of old style ballasts out there.

That being said, I agree with your view of not shooting under fluorescents without alternaitve lighting; they tend to have terrible light far a lot of colour photography is concerned; especially skin tones. One of my Speedlights came with a green gel to help with the green spike and I know you can get CTG gels to improve colour balance issues as well. I've never used them but saw them on my local photo shop sitting there with the CTO and CTB gels.

[NorthernFocus]

...Don't get caught up in reality - it rarely if ever exists....

People seem to have blown by Glenn's comment so I'll bring it forward.

There are two aspects to this discussion. One is the true color temperature of light as scientifically measured. The other is human perception. Unless one is pursuing the "true color" of something in the context of scientific study, the first one is not relevant. Most of us who are capturing/creating images are only concerned with human perception. By definition we're dealing with a subjective topic.

Not to pop anyone's bubble but those out there who are in the SOOC camp who think you are accurately capturing lighting/colors in a scene are mistaken. What you are re-creating in the camera via settings, etc. is your perception of the lighting/colors at that moment in time. That is unless you're using a calibrated grey/white card as a reference rather than tweeking things to make the resulting image look "right".

I tried using a grey card for a while and invariably the scenes looked flat relative to the image in my mind's eye. The only time I've really found it useful is shooting indoors under artificial lighting.

Let's all be real about this. Some of us simply prefer to use different tools to re-create our particular versions of reality.

Now, all that said, one thing that does drive me absolutely bonkers is how different camera bodies by the same manufacturer can come up with such different color renditions when shooting the same scene with AWB

[jcuknz]

My question is ...AWB .... if it is so bad why does it work so well for me, except when shooting under artifcial light?

Perhaps I belong to the 'vaste majority' and have twice failed the 'dots' colour test set by my employers, plus a horrendously unfair and unrealistic MOT test as a yachtie

Perhaps as a result of those experiences I don't bother about colour ... if it looks OK then it is AOK.

But judging from the entries to #20 [Editing -- seashore] I wonder about people accepting stuff which is obviously wrong to me ... unless they were being deliberately careless.

EDIT #21 Good one Manfred

[Inkanyezi]

I see that logs are put on the fire and stoking taking place. I'd like to add a few cents worth on the deviation of light spectra, ballasts and fluorescents.

Fluorescents come in various flavours, and those sold in my part of the world nowadays are almost invariably three-phosphors, which have very good colour rendition. They do have spikes in their spectrum as well as dips, and so does natural daylight. The only light sources that have a continuous smooth spectrum are the heat emitters, black body radiators. The sun is a nuclear reactor with ionising radiation inside, exciting gaseous substances that are solid at more modest temperatures. Sunlight is filtrated in our atmosphere. On a sunny day, our sky is blue, dispersed blue from the sunlight, light that is spread in the upper layers of our atmosphere. The sun itself, intensely white, appears yellowish against that blue backdrop. The light that reaches us, when there are no clouds, is a mix of direct light from the source, and diffused mostly blue light from the entire sky. In the shadows, mostly the blue light from the sky as well as some reflected light is present, so they appear bluish in photographs, and in open shade, light is rather blue, with a "colour temperature" in the region of 10000 K.

So, only incandescent lamps have a smooth even spectrum with no holes or spikes. They are also the norm for the K scale, up to 4999 K. Above that, the norm is "daylight", a spectrum that is not totally smooth, a spectrum that has a few dips and more than one top, and that is stronger in blue and weaker in red than the black body radiation.

How about the camera sensor, its Bayer filter, tricolour screens and the very colour sensitive cells in the retina of our eyes? Do they emit or respond to "colour" in the same way, or do they really depend on the "smoothness" or "freedom of spikes/dips" in the light source? All of them do have some unevenness in their emission or response to various wavelength, and colour itself is not a wavelength, but a figment of imagination. All of them also respond to, or emit, other wavelengths than their "colour". All our vision cells respond, to a varying degree, to all wavelengths in the visible range, with their peak sensitivity around the "colour" that they represent. Same with all colour filters in camera sensors. They let all wavelengths through to a varying degree, with the peak at the wavelength representing their "colour". Keep in mind that colour is not a physical property, but a figment of our minds, a response in our brain to mixed optical phenomena received by our eyes.

So to get "correct" or "pleasing" colour, it is not necessary to have a smooth spectral curve, and the K scale is not wholly applicable to fluorescent lamps. It is fully valid for heat emitters, black body radiators, incandescent lamps - but alas, not for daylight. We use it anyway, but the spectral curve of "daylight" is quite different from heat radiator spectral curves.

On ballasts and cycles then. Not only magnetic ballasts cycle the light output at twice the mains frequency (2x60=120 cycles in USA, 2x50=100 cycles in Europe and many other places in the world). Also many electronic ballasts share this property, due to poor smoothing of the rectified DC that drives their HF oscillators. This is not entirely a bad thing, as better smoothing is a problem for the power supply, taking the entire output only from the top of the voltage curve, leading to tremendous current spikes. So all screw-in type of fluorescent lamps (or LED) also cycle at 100 or 120 Hz just as FL lamps with magnetic ballasts. Only one manufacturer of larger ballasts has compensated for the spikes inherent in using only the top part of the voltage curve, Vossloh Schwabe. They include a magnetic ballast in series with their electronic HF ballasts, and with proper compensation those pose no problems to the supply grid.

Good three-phosphor fluorescents or better provide excellent colour rendition for photography, but must be custom balanced, as there is no camera that has a preset WB for such lamps. Their "K" value will usually not render colour well, so adjusting by the K scale might fail. Only older, "single colour" FL lamps are impossible to white balance well, and those will never render skin tones in a pleasing fashion. After about 10000 hours of service, the colour rendition of FL tubes has degraded beyond good colour renditions, and FL lamps that are mounted in a casing that does not provide adequate cooling will always get too hot and give poor colour rendition, regardless of the quality of the FL lamp.

To avoid banding with focal plane shutters, exposure times should preferably allow at least three full cycles of the light output, i.e. if the lamp pulsates with twice mains frequency, 1/30 second or longer might be required.

[NorthernFocus]

..Perhaps I belong to the 'vaste majority' and have twice failed the 'dots' colour test set by my employers, plus a horrendously unfair and unrealistic MOT test as a yachtie ...

Twenty percent of all males are "legally color blind" by US standards. Which I believe means can't pass the dots test. And everyone else has varying perceptions of the color spectrum.

[AB26]

I see that logs are put on the fire and stoking taking place.

Sir Urban,

If only, those whom put logs on the fire to stoke it, can appreciate the value of your post. It is beyond my understanding why a discussion on WB always has to turn into a dogfight.

From what I can understand from your post is it is not an attempt to change any minds, it is rather informative and should be archived as technical teaching material. It is of great value to those whom seek to improve their photographic skills.

Can we please have the “Fluorescent” WB function also removed from our cameras?

[xpatUSA]

As has been pointed out, there is no white balance for a light source per se. However, the camera or the image in PP demands a WB to be set, right or wrong. This is quite a dilemma.

As I think about it today, perhaps there is an approach that is more scientific than "getting close to a visual impression". Given a spectral emission curve, is it not possible to determine a device-independent color in L*a*b* space? Perhaps directly from the dominant wavelength, in the case of watch phosphors. Armed with that, adjust image WB until the Lab color picker says the same? Or maybe determine the XYZ values, convert those to L*a*b* and then do it?

Apparently, XYZ co-ordinates can be obtained from a spectral power curve, but it ain't easy. See here for a green LED example.

[Inkanyezi]

Apparently, XYZ co-ordinates can be obtained from a spectral power curve, but it ain't easy. See here for a green LED example.

The link points to another technical issue, which relates to three-colour mixing for obtaining white light.

I think that in technical discussions about colour, sometimes it is valuable to discern colour from spectral output, as colour is our perception, that has been treated by the brain, while spectral properties are physical properties of the radiation involved. Light often is mixed, and we are seldom interested in the exact composition of spectral lines. The RGB mixing, which takes place in three-phosphor lamps as well as computer screens of different types and LED colour blenders, is a special case, where we indeed use discontinuous spectra in order to obtain a discontinuous spectrum that simulates "natural light" with a "continuous" spectrum. Mind that receptors in the eye are mainly sensitive to the same three spectral bands.

Watch photography, when you want to record the luminous output from the nightglow numbers and hands, should not be balanced to the output of the luminiscent material, but to the estimated colour perception of the eye. Hence it is most likely that we'll find the "correct" white balance somewhere along the K scale. With a calibrated monitor, the easiest method might be comparing the output of the monitor to the actual glow of the watch. As it is only one single colour, it shouldn't be too difficult. Just adjust the K slider until it looks right.

Of course the colour coordinates may be used in PP software to obtain a likeness, but the method seems a bit overkill for practical use. We are rather accustomed to adjusting images visually, and the visual impression should be enough, as long as we are not performing some scientific task, where exact physical properties should be recorded.

Discontinuous light spectra sometimes do pose problems of colour perception, and it relates to the different reflective properties of various substances and dyes. We have a phenomenon called metamerism, related to different colour perception under different lighting. This is a problem when mixing paint for repairing damage on cars, or for matching clothes and accessories, like a jacket and skirt with a blouse and finding a matching handbag and shoes. Sometimes colours that look right under incandescent are very different in daylight and the other way around. And when patching up some minor damage to the car paint, it might look okey in the garage, but when you get it out on the street, those dots look like freckles. To match colours for different light sources is an art.

[xpatUSA]

Watch photography, when you want to record the luminous output from the nightglow numbers and hands, should not be balanced to the output of the luminiscent material, but to the estimated colour perception of the eye.

Thanks for the advice on watch photography. However, the link uses the CEI x,y and z co-ordinates which are acknowledged as the best approximation of LMS response of the average eyeball (albeit adjusted so the negative bit goes away). The link determines a device-independent color in XYZ form which, as the link explains, is the normalized derivation of those x,y and z values each multiplied by the luminous material's spectral emission and integrated of course. Once calculated, XYZ is readily transformable to any color space (after all, it is an ICC profile PCS).

Hence it is most likely that we'll find the "correct" white balance somewhere along the K scale. With a calibrated monitor, the easiest method might be comparing the output of the monitor to the actual glow of the watch. As it is only one single colour, it shouldn't be too difficult. Just adjust the K slider until it looks right.

My point is that a particular XYZ color transformed to, for example, sRGB (gamut considerations apart) gives exact target RGB values, or HSV, or L*a*b*, which is a whole lot better than playing with a slider until something "looks right". And the monitor need not even be calibrated if the color picker is using the image data, like PhotoShop Elements for example.

Of course the colour coordinates may be used in PP software to obtain a likeness, but the method seems a bit overkill for practical use. We are rather accustomed to adjusting images visually, and the visual impression should be enough, as long as we are not performing some scientific task, where exact physical properties should be recorded.

The purpose of my clock image earlier was scientific - an attempt to determine luminance from a digital image. Knowing no better then, D65 was used, a blind guess and quite unscientific.

[Inkanyezi]

Sorry if I caused confusion, while the intention was to clarify. I understand the purpose of finding coordinates for the luminous material on the face of the clock. Of course its exact coordinates is a good way to render it at any WB setting for the rest of the image. After all, the output is the same whatever "white" your vision is tuned for and maybe the difference is negligible whether we use D65, D50 or any other setting, as it is a single colour, although it must be translated into a mix of two or three.

I mainly wanted to point to the difference between radiation, spectral output, and colour, perception. When comparing the output on a monitor with the actual glow of the luminous material, it is evident that the clock face emits a single colour, but the monitor will display a colour mix, in order to tickle the same nerves in our visual system.

[xpatUSA]

No problem, Urban,

I understand the difference between spectral power and spectral luminosity.

Today I found a paper here where they actually showed the CEI x,y co-ordinates for some experimental phosphors!



The doping caused peaks in the spectrum, unlike that for Luminova previously shown. However, all three x,y's were out-of-gamut for sRGB according to Bruce's fine calculator. Middle one was the worst:



Changing Y didn't help

So, my proposed "simple" method could not be guaranteed to work, since the raw conversion would involve transformation of out-of-gamut sRGB colors. One remaining hope is Dave Coffin's XYZ dcraw conversion to get the color, assuming that his conversion is correct. Although his output is 8 or 16-bit, it would be easy to normalize the numbers.

FYI, my original method is posted here.
.

[xpatUSA]

Andre - what you are experiencing will occur with older transformer style ballasts where the line (mains) frequency can have the effect that your are seeing. Shooting at a shutter speed where you get two full cycles (1/30 s for North America (60HZ line frequency) and 1/25 s for much of the rest of the world (50 Hz line frequency) will mitigate this issue.

I use a Philips TL950 lamp as an overhead light for my table-top work (plus a couple of LED floods for subject lighting). It has a claimed CRI of 98 but ya know how that goes. Here's the spectrum:



Spikes galore - but of relatively narrow bandwidth which could explain the amazingly high CRI. Of course, with appropriate doping, the spikes can be arranged to avoid or aid the CRI metrics which is why Ra is becoming more popular. Having said that, the lamp is always on over my bench, even when taking shots.

Main point of my post is that the lamp-holder buzzes away loudly at 60Hz. However, when I turn the lamp off, it glows quite visibly for a short time afterward due to the persistence time of the phosphors in the lamp. So that may be why I have never noticed the modulation effect referred to by many.