Setting White Balance with IR filters

[pnodrog]

Could not help myself.

A few years ago I googled "scene recognition exposure" and came up with very little info. Tried it just now and far more information seems to be available. I'm going to start by reading THIS.

P.S. I have now read it and it was as useful as advertising.

"The New Nikon D4 DSLR uses a newly developed Advanced Scene Recognition System and 3D Color Matrix Metering III’s 91,000-pixel RGB sensor that is tasked with measuring each scene’s light properties, color spectrum and brightness levels, and analyzing it against the camera’s built-in image database to realize even more accurate autofocus, auto exposure, i-TTL balanced fill flash, Active D-lighting and auto white balance results."

I doubt that any of the manufacturers are going to tell us exactly how the initial exposure is determined.

[xpatUSA]

If the image is saved as a RAW file the only issue is exposure. If a sharp IR pass filter has been used there should be minimal exposure captured in the Blue and Green channels. With a Bayer colour filter the metering will possibly expect the green channel to provide about 50% of the exposure reading. It will depend on the exposure algorithm what bias this makes to the camera's interpretation of the exposure. If the exposure meter has built-in scene recognition it may realise it is caused by a IR filter rather than the subject being say just a red painted wall.

Interestingly some camera manufacturers make claims they incorporate scene recognition software but I have never viewed any specific information relating to it. Not that I have been interested enough to searched for it. Regardless you are discussing an exposure problem and any use of WB will only be significant if the exposure determining algorithm takes it into account.

Hopefully one of you technically driven photographers will research it all out while I'm out taking photographs....

While you were away, Paul ...

We should remember that in-camera metering sensors have the same spectral response as ordinary light-meters, voila:



Photopic response refers:

http://hyperphysics.phy-astr.gsu.edu.../efficacy.html

This is because photographic exposure works using lux and lux has no IR content.

So, if I point my converted DSLR at a scene dripping with IR, the metering sensor will not see the IR - and therefore will recommend a very much higher exposure than is optimal for the IR or even full-spectrum. In a scene with IR in it, I usually have to apply around -2EV exposure compensation so as not to blow the IR-reflective parts of the image.

Apologies if you already knew all that!

[xpatUSA]

I think that those who actually do know what they are doing and then advocate setting a specific WB when using an IR Filter (as opposed to leaving the WB on AUTO) are mainly doing so for the ease of Post Production continuity and workflow. I expect this reason is not always apparent and perhaps not often not explained adequately.

I doubt that not setting a specific (i.e. leaving it on “Auto” White Balance) actually limits the outcome possibilities, however probably doing so will make the road to some possibilities longer and/or more tedious.

***

If you set a specific WB (i.e. not Auto WB) before you begin, this will allow (probably) easier post production, especially if you want to have continuity of that output, between scenes or between brackets of one scene.

There is also a view, based upon theory, to set a specific 'warm' WB, (for example "Incandescent" if Degrees K is not available) there will be less likelihood of clipping. I quite like the theory but I have never tested the real life value of it.

By coincidence today I took a Custom white balance with my converted camera of some foliage in bright sun and then examined the raw histogram and also the RGB conversion of that same raw data.



No clipping -red maybe 2/3EV headroom.

Duh, the response below -5EV is from the masked pixels which I neglected to turn off ...



No adjustments, default settings only, observe that the red in the review histogram is badly clipped.

So, IR/full-spectrum photography is nowhere near as simple as internet lore makes it out to be ...

[dje]

While you were away, Paul ...

We should remember that in-camera metering sensors have the same spectral response as ordinary light-meters, voila:



Photopic response refers:

http://hyperphysics.phy-astr.gsu.edu.../efficacy.html

This is because photographic exposure works using lux -well, the candela really- and lux has no IR content.

So, if I point my converted DSLR at a scene dripping with IR, the metering sensor will not see the IR - and therefore will recommend a very much higher exposure than is optimal for the IR or even full-spectrum. In a scene with IR in it, I usually have to apply around -2EV exposure compensation so as not to blow the IR-reflective parts of the image.

Apologies if you already knew all that!

Thanks Ted, very illuminating (not irradiating!). I hadn’t really thought about that before. Presumably the metering sensors have a filter that mimics the efficacy response? I wonder what they do with mirrorless cameras which don’t have a separate metering sensor?

Dave

[xpatUSA]

Thanks Ted, very illuminating (not irradiating!).

Excellent quip, Dave!

I hadn’t really thought about that before. Presumably the metering sensors have a filter that mimics the efficacy response?

I imagine so. Here's a mention of the CIE response:

https://solarlight.com/product/pma21...opic-detector/

As a Foveon owner, I can say that the middle (green) layer comes similar to that response, too.

I wonder what they do with mirror-less cameras which don’t have a separate metering sensor?

Dave

Mirror-less cameras generally have a UV/IR blocking filter over the sensor. That filter may or may not emulate the CIE response. On my SD15 camera it certainly doesn't, with it's sharp dichroic cut-off slopes.

And on the Sigma Quattro series, Foveon don't even attempt to measure true illuminance, they use the top layer whose response hardly resembles the CIE curve.

I'll pass an opinion that in-camera metering, especially mirror-less, barely resembles that of good light-meters and is probably supported computationally by fudge-ware.

Pardon my cynicism ...

[dje]

Ted I've looked at the green channel spectral response of a few DSLR's and one mirrorless (Sony A7R) and they were all very similar and not too much different to the Photopic response. The green channel filters had a peak around 525nm compared to 555nm for the Photopic curve. It's possible that with mirrorless they just use the green channel for metering, perhaps with a bit of adjustment, but who knows!

Dave

[xpatUSA]

Ted I've looked at the green channel spectral response of a few DSLR's and one mirrorless (Sony A7R) and they were all very similar and not too much different to the Photopic response. The green channel filters had a peak around 525nm compared to 555nm for the Photopic curve. It's possible that with mirrorless they just use the green channel for metering, perhaps with a bit of adjustment, but who knows!

Dave

Agreed, Dave.

Here's an early Foveon response without and with a filter for test purposes.



It can be seen above that the green channel with filter is a very close match indeed, not that that particular model of sensor was ever used in a mirror-less camera. Also see figure 7 (coincidentally) in the link below which shows both Foveon and Bayer - but with no IR filter on the Foveon.

http://kronometric.org/phot/sensor/f...nt%20Paper.pdf

Later Foveon models were used in Sigma's compact cameras DP1 etc. and I can see that the green channel in combination with the necessary IR blocking filter and peaking at 550nm would be good enough for luminance measurement. Furthermore, one of their papers about video capture specifically mentions the use of that channel to get Y as in Y'CbCr ...

[pnodrog]

While you were away, Paul ...

We should remember that in-camera metering sensors have the same spectral response as ordinary light-meters, voila:



Photopic response refers:

http://hyperphysics.phy-astr.gsu.edu.../efficacy.html

This is because photographic exposure works using lux and lux has no IR content.

So, if I point my converted DSLR at a scene dripping with IR, the metering sensor will not see the IR - and therefore will recommend a very much higher exposure than is optimal for the IR or even full-spectrum. In a scene with IR in it, I usually have to apply around -2EV exposure compensation so as not to blow the IR-reflective parts of the image.

Apologies if you already knew all that!

No apologies needed. I claim no knowledge whatsoever. However traditional methods of exposure measurements probably have little relevance to engineers designing exposure assessment based on scene recognition. Scene recognition exposure evaluation appears to be based on distribution of RGB levels within the frame rather than the overall lux.

Unfortunately the camera manufacturers are rather coy about the specifics but we do know for many camera's the exposure system is no longer a simple lux based device.

Regardless I agree that it is an exposure problem and any metering system will need some sort of reasonably significant correction to be of any use.

[xpatUSA]

No apologies needed. I claim no knowledge whatsoever. However traditional methods of exposure measurements probably have little relevance to engineers designing exposure assessment based on scene recognition. Scene recognition exposure evaluation appears to be based on distribution of RGB levels within the frame rather than the overall lux.

Interesting conjecture. My full-spectrum camera has no such metering and doesn't have a CFA, so I'm not familiar with it.

Unfortunately the camera manufacturers are rather coy about the specifics but we do know that, for many cameras, the exposure system is no longer a simple lux based device. <>

So, is it now a complex lux-based system as opposed to simple? Or is measurement of illuminance at various areas of the sensor not done at all in that system, I wonder?

[xpatUSA]

So, is it now a complex lux-based system as opposed to simple? Or is measurement of illuminance at various areas of the sensor not done at all in that system, I wonder?

A bit of research while it's raining:

Nikon's guy Mike Corrado isn't a lot of help with his marketing-speak, but he does mention that a) the sensor is a "color matrix" and b) measures brightness:

https://www.nikonusa.com/en/learn-an...d-advanced-srs

A bit more detail here:

http://www.nikon.com.hk/en_HK/learn_...the-RGB-sensor

"The RGB sensor works together with a diffractive optical element, a special filter that separates light into different wavelengths and projects a clear image onto the sensor."

Brightness gets a mention there, too.

Since brightness is often measured in lumens or alternatively cd/m2, we end up with an estimate of lux at the metering sensor regardless of all the other AI-type fancy stuff. In which case, Nikon's fancy metering sensor will not see IR and will still recommend too much exposure, IMHO.

[Manfred M]

Nikon's fancy metering sensor will not see IR and will still recommend too much exposure, IMHO.

Correct, in my experience and the same can be said for any modern DSLR with its discrete metering module. These cameras are designed to measure visible light and the manufacturers have integrated both IR and UV cutoff filters into the sensor stack to keep the image free of anything other than visible light.

I remember that Leica's first foray into a higher end digital camera, the M8 had a technical issue relating to the back of the lens to sensor distance. They did not incorporate either an AA filter nor a IF cutoff filter into the filter stack to makes sure that the filter stack was thin enough to prevent a collision. These cameras had a well known problem of IR "polluting" the reds. Leica eventually "fixed" the problem by supplying a lens-mounted IR cutoff filter to owners of that camera.

[pnodrog]

"The RGB sensor works together with a diffractive optical element, a special filter that separates light into different wavelengths and projects a clear image onto the sensor."

This appears to be a contradictory statement. I am aware of RGB sensors and have come across diffraction gratings, prisms etc in laboratory equipment I have used. What they mean by "a clear image" is a mystery.(maybe green only?) I would guess that the statement has been written by PR people and is not a serious attempt to explain how it works. Sounds good but is probably just a fob off. Explanations by people who really know what they are talking about sometimes need to be long but are seldom confusing.

Anyway with the lack of any sound explanation as to how the exposure system works it is useless to speculate how to best correct the indicated exposure.

Suck it and see but don't trust the histogram too much. Hopefully the red channel does give a clue.

[Manfred M]

Paul the real issue in my view is that the material that is published about how cameras (and lenses) work is usually put out by marketing departments (likely vetted by the company's legal department), so are generally "dumbed down".

My understanding is that both Canon and Nikon do measure the R, G and B values independently in their DSLRs and use an algorithm to calculate the "correct" exposure. I can only assume that the other camera manufacturers follow suite.

When it comes to mirrorless cameras, the readings are based on the light hitting the sensor, so are by default they are using individual colour channels.

[xpatUSA]

Found some info about Canon metering here:

https://digital-photography-school.c...ur-eos-camera/

Unfortunately, it contains this text which I find hard to swallow:

"Canon introduced Intelligent Focus, Colour and Luminance (iFCL) evaluative metering on the EOS 7D. It has been included on every new EOS camera since then (excluding One series models). The premise behind this new system is that exposure meters that react to the brightness of the subject only are more sensitive to the red light than the human eye. They may think that a red-coloured subject is brighter than it really is and return a false reading that underexposes the subject."

Luminance = CIE Y' = 0.3Red+0.6Green+0.1Blue (roughly, color spaces vary and linear Y is different). But none of them have red brighter than green ...

[xpatUSA]

Found some info about Canon metering here:

https://digital-photography-school.c...ur-eos-camera/

Unfortunately, it contains this text which I find hard to swallow:

"Canon introduced Intelligent Focus, Colour and Luminance (iFCL) evaluative metering on the EOS 7D. It has been included on every new EOS camera since then (excluding One series models). The premise behind this new system is that exposure meters that react to the brightness of the subject only are more sensitive to the red light than the human eye. They may think that a red-coloured subject is brighter than it really is and return a false reading that underexposes the subject."

Much more about the Nikon system here -including an actual 1005-segment layout diagram.

http://www.mir.com.my/rb/photography...onF5/metering/
.

[Manfred M]

Much more about the Nikon system here -including an actual 1005-segment layout diagram.

http://www.mir.com.my/rb/photography...onF5/metering/
.

The Nikon F5 film camera was introduced in 1996. This article is ancient information and frankly I have no idea as to how that technology compares to current digital camera models. The processing power would be negligible when compared to current models.

[xpatUSA]

Originally Posted by xpatUSA Much more about the Nikon system here -including an actual 1005-segment layout diagram.

http://www.mir.com.my/rb/photography...onF5/metering/
.

The Nikon F5 film camera was introduced in 1996. This article is ancient information and frankly I have no idea as to how that technology compares to current digital camera models. The processing power would be negligible when compared to current models.

It would appear that the reference has absolutely no relevance to today's cameras, so I withdraw it ...

[pnodrog]

It would appear that the reference has absolutely no relevance to today's cameras, so I withdraw ...

I think it does have relevance. The description seems to be the at the start of the scene recognition approach. Certainly the processors are much more powerful now. However remember the processor for a film camera only had the relatively simple tasks of focus and exposure. The really heavy processing demand only arrived with digital sensors.

The late 80's early 90's were the beginning of the "fuzzy logic" and later "AI" etc. Some very sophisticated applications were starting to appear at that stage. A lot of the ground work for what many people now think of as complexed modern computing was actually started back then.

[Manfred M]

I think it does have relevance. The description seems to be the at the start of the scene recognition approach. Certainly the processors are much more powerful now. However remember the processor for a film camera only had the relatively simple tasks of focus and exposure. The really heavy processing demand only arrived with digital sensors.

The late 80's early 90's were the beginning of the "fuzzy logic" and later "AI" etc. Some very sophisticated applications were starting to appear at that stage. A lot of the ground work for what many people now think of as complexed modern computing was actually started back then.

Agreed and that generation of cameras are the building blocks that current designs are based on.and absolutely. Some of the rarely used metering and focus modes on modern cameras are left-overs from that era.

My main issue with some of the links Ted posts is that often they are quite old and may or may not be relevant when looking at current technology.

The lenses of that era at least had an IR focusing mark to guide photographers on off-setting the actual focus more appropriately. I did all of my IR work in the film days and I seem to remember that the mark was applicable for B&W work, but not for colour as colour IR had a huge visible light component in the images and using the IR mark resulted in a somewhat softer image.

[xpatUSA]

My main issue with some of the links Ted posts is that often they are quite old and may or may not be relevant when looking at current technology.

More old information that may not be relevant to current technology, Paul.

"The new innovation in the D3/D300 autoexposure system is the introduction of a small diffraction grating or Diffractive Optical Element (DOE for short) in front of the autoexposure lens. The DOE bends the light passing through it by an amount proportional to its wavelength, or color. The amount of this bending is precisely controlled, such that the red, green, and blue wavelengths are shifted by an amount equal to the spacing of the columns of pixels in the RGB sensor."