Camera calibration and icc profile creation workflow

[bipulmohanto]

Hi! I would like to have my own pipeline for color management. So far I have taken RAW image with Canon, converted into TIFF( without any data correction) then applied white balance (e.g. gray world) and demosaiced (e.g. bilinear interpolation).
* I would like to know, after demosaicing when I get the color image, won't I call it camera RGB values(which is device dependent )?
** I want to final image for sRGB color space, should I call this sRGB a PCS (profile connection space). Do I need to convert my camera RGB to sRGB?
***now my main problem is, how can I create an icc profile from this camera RGB image? I do not want to use any professional software actually. I know the name of 4 color management module and their differences but do not know how to create or use them.

I read several literature(still idea unclear), I would like to know how to actually do it and build the pipeline by myself. Please help.

[xpatUSA]

<>
** I want to final image for sRGB color space, should I call this sRGB a PCS (profile connection space).

No, you should not. A PCS is device-independent, sRGB is not. In an ICC profile, the PCS is usually XYZ or CIELAB.

Do I need to convert my camera RGB to sRGB?

I do not understand the question, sorry. What is "camera RGB"?

***now my main problem is, how can I create an icc profile from this camera RGB image?

To create a profile, you shoot a color-patch target under known lighting, usually converting raw to TIFF. Then you run a program which opens the TIFF, detects the patch colors and produces either a file of type .dcp or of type .icc/.icm. Each type of file is called a "profile", either camera or ICC. Which one you use will depend on your workflow and editor.

I do not want to use any professional software actually.

Hmmm . . . .

I have used CoCa:

http://www.dohm.com.au/coca/

I grew tired very quickly of having to make a new profile for every possible scene and lighting, and stopped using it.

[Manfred M]

I'm not quite sure what you are trying to achieve here.

I use a custom camera profile, work with an sRGB / AdobeRGB compliant computer screen that has been calibrated and profiled. My workspace is dimly lit (under 40 lux) in a neutral coloured room and I use manufacturer supplied printer / paper profiles when I print on my "pro" photo printer.

The raw data from your camera has no colour space, but the moment you turn that raw data into an image file, you have to assign one (I tend to work in either the ProPhoto RGB colour space or L*a*b* colour space).

I'm not quite sure where you are trying to go with this as some of the things that you say are not all that clear to me.

Regardless of anything else you need to take a picture of a colour test target so that you can develop a colour profile for your camera. I tend to shoot either in daylight or with flash (usually studio flash) and have found that a single camera profile works well for these lighting conditions. I used to use a separate profile for these, but found that the results were identical, so I only use one.

If I were to shoot under a continuous light source, I might think about making a profile for that as well, but as I don't, I have no need to do this. Likewise, if I were to shoot with some weird light source I might consider doing a separate profile, but for the type of photography I do, i have never found the need to do so. I don't think there is a need to do a colour profile for every different shooting scenario as the effort involved does not provide me with any tangible benefit. The same comment goes for custom printer / paper profiles; for the type of work I do, the paper manufacturer supplied profiles are more than adequate and the amount of money I would have to spend to get a photo-spectrometer to match what the manufacturer does would buy me another mid-range camera body.

Ted is also correct, the profile depends on what your editing software requires. I use the .dcp with the Adobe products I use and the .icc with Capture One.

If I were to suggest one place where I see the least benefit to my workflow, it would have to be the camera profile. The default values are so close that people would not be able to tell when I use my custom one versus the generic ones supplied with the editing software.

[DanK]

Bipul,

My reaction is similar to Manfred's. Why do all this?

My experience has been that my Canon bodies have not needed a custom camera profile, although if I shot in odd mixes of light, I might decide otherwise. I use a color-calibrated monitor and ICC profiles provided by the companies whose papers I use. Like Manfred, I use a wide-gamut monitor, and I switch it to Adobe RGB when editing for prints, but I switch it to sRGB when editing for the web, as sRGB is the standard for computer displays. I don't use a color checker, but I do often include a spectrally neutral card (I use a whiBal) in one shot to get a good starting point for white balance. This seems to work well.

Dan

[dje]

Hi! I would like to have my own pipeline for color management. So far I have taken RAW image with Canon, converted into TIFF( without any data correction) then applied white balance (e.g. gray world) and demosaiced (e.g. bilinear interpolation).
* I would like to know, after demosaicing when I get the color image, won't I call it camera RGB values(which is device dependent )?
** I want to final image for sRGB color space, should I call this sRGB a PCS (profile connection space). Do I need to convert my camera RGB to sRGB?
***now my main problem is, how can I create an icc profile from this camera RGB image? I do not want to use any professional software actually. I know the name of 4 color management module and their differences but do not know how to create or use them.

I read several literature(still idea unclear), I would like to know how to actually do it and build the pipeline by myself. Please help.

Bipul we need more info on what you are trying to do before we can provide much help.

eg

what are the names of these color management modules you talk about?

when you talk about a profile from a "camera RGB image" are you talking about a shot of a Color Checker 24 patch target?

The basic raw processing approach takes the white balanced and demosaiced data and applies a camera profile to convert this to a PCS of either XYZ or LAB. The raw processing software then converts this to a standard working color space such as Adobe RGB etc. Is it this camera profile you wan to create?

Dave

[bipulmohanto]

Hi Dave,
Thanks for your reply. I am very new in this field actually, that's why my understanding may be very silly. Your first point "color management modules"- I am actually working with Canon EOS 5DS R camera and my main target is generating a good ICC profile for this camera which can I use for rest of the images (raw) taken under the same lighting condition. When you called color management module, did you mean like LittleCMS, Argyll? If so, I could not select any particular module, but I have some knowledge with Argyll binaries.

With "camera RGB" I meant the RGB channel values which I get after demosaicing. Am I wrong in this terminology? I have a Rez Checker Nano color checker. And I shot the raw image with my Canon EOS 5DS R. Then apply white balance on raw data and demosaic.

so when I will create an icc profile, this should be like this: take a color checker photo raw format>> apply white balance algorithm>> apply demosaic algorithm >> send this image to color management module (e.g. Argyll CMS) which will then extract the patches, input ground truth data for each of the patches from spectophotometer ...>> the color management module finally will create a camera profile (ICC profile)

then I can apply this ICC profile on other images (do I need to apply same white balance and demosaic algorithm as I applied on the first color checker image?) taken under the same lighting condition. I know adobe photoshop can apply custom icc profile on images.

Thank you once again.

[dje]

Hi Dave,
Thanks for your reply. I am very new in this field actually, that's why my understanding may be very silly. Your first point "color management modules"- I am actually working with Canon EOS 5DS R camera and my main target is generating a good ICC profile for this camera which can I use for rest of the images (raw) taken under the same lighting condition. When you called color management module, did you mean like LittleCMS, Argyll? If so, I could not select any particular module, but I have some knowledge with Argyll binaries.

With "camera RGB" I meant the RGB channel values which I get after demosaicing. Am I wrong in this terminology? I have a Rez Checker Nano color checker. And I shot the raw image with my Canon EOS 5DS R. Then apply white balance on raw data and demosaic.

so when I will create an icc profile, this should be like this: take a color checker photo raw format>> apply white balance algorithm>> apply demosaic algorithm >> send this image to color management module (e.g. Argyll CMS) which will then extract the patches, input ground truth data for each of the patches from spectophotometer ...>> the color management module finally will create a camera profile (ICC profile)

then I can apply this ICC profile on other images (do I need to apply same white balance and demosaic algorithm as I applied on the first color checker image?) taken under the same lighting condition. I know adobe photoshop can apply custom icc profile on images.

Thank you once again.

Hi Bipul

Thanks for clarifying your aim. Your general understanding of the basic process seems sound enough.

If you are using Photoshop/ACR or Lightroom to process your raw files, you will need a .dcp profile rather than an ICC profile for camera calibration. (or you can like most people use those supplied by Adobe). Some of the converters such as RAWTherapee (free), Capture One, and AfterShot Pro use an ICC profile for camera calibration.

It's not clear to me what raw processing software you are using, or are you trying to produce your own? If you are, be prepared for a long and difficult development road!

Argyll is fairly powerful and highly regarded although it has a fairly steep learning curve I'd say. It doesn't produce dcp profiles, only icc. The other software you could look at to produce profiles is DCAMProf. Have a look at this link for some good detail on camera profile creation.

I'm not familiar with the REZ Checker target but you would need to check that it can be used with the CMS software you use.

As for your last question, the camera profile can be used for all images taken with roughly the same illuminant. White balance needs to be adjusted for the particular illuminant for each image. The RAW file EXIF contains details of the WB multipliers the camera selected when taking the shot.

Some people like to have several camera profiles for a few different illuminants. Adobe uses a dual illuminant profile which includes matrices for D65 and CIE A. An interpolated matrix is calculated from these depending on the actual color temperature of the illuminant.

One other source of code for raw conversion is open source Libraw. I haven't used it but it does contain a large number of code modules for various tasks.

Dave

[Manfred M]

You can't get around the white balance in the process, but it is not a particularly significant part of the exercise. The ColorChecker that I use and the RezChecker you use have significant real estate on the cards dedicated neutral patches that are used to determining the white balance.

In the limited testing with my cameras. I found that the white balance does not seem to matter in the end as my software determines the white balance corrections as it generates the camera profile.

This exercise is all about determining the camera's response over the three colour channels given a specific spectral input. Daylight / Studio flash, both of which have similar spectral curves have a full spectral response. Other light sources, especially fluorescent lights or LED lights can have some very strange spectral curves and a custom camera profile may be warranted for these. My view is that having a good profile for terrible lighting will end up giving you technically correct but still awful lighting in your images (which is why I do my best to avoid then).

[bipulmohanto]

Hello Ted, sorry for my ignorance, I am really new in color management, but I found this forum at least which is really nice and helpful.
from the first point, which now I think I understood wrong, "final image for sRGB color space"- I wanted to have an icc profile under a certain lighting condition using the color checker (I am using Rez checker nano). Then apply this new profile on other images (non color checker) taken under the same lighting condition (D50) to see they are better visualized with respect to color. I thought applying the new icc profile on other images would convert it into sRGB color space.

"camera RGB" I meant by camera RGB is the color which I get after white balance and demosaic algorithm. It generated color first time after demosaic and I thought that could be called an RGB (but RGB of camera color space).

"how can I create an icc profile from this camera RGB image"- If you not mind, so raw to icc profile creation should be like this actually: taking raw image of color checker>> do white balancing >> demosaicing >> image in TIFF >> Import the image in CoCa software which will use dcraw library for image importing, scanin from argyl to detect the patches, take the ground truth data from spectometer and colorprof for creating a nice camera profile and save it.

Now before I apply this ICC profile on other images (taken under the same lighting condition) may be using photoshop 6, should I convert all the images beforehand (the same white balance and demosaicing)? Or only applying this color profile on other raw images automatically do the whole process?

Thank you very much for your answer, and the CoCa software. It is working perfect.

[Manfred M]

H
Now before I apply this ICC profile on other images (taken under the same lighting condition) may be using photoshop 6, should I convert all the images beforehand (the same white balance and demosaicing)? Or only applying this color profile on other raw images automatically do the whole process?

Ted is not a Photoshop user, so may not be able to help.

The profile is applied during the raw conversion stage and in Photoshop the default is Adobe Camera Raw (ACR). You select your profile here (as shown by the arrow) and make your custom profile your new default. Your CS 6 ACR will look a bit different than my CC 2019 screen, but the process is the same.

As you can see, I use a custom my custom D810 profile as my default. The difference between it and Adobe Standard are slight. When I used the D800 there was a noticeable difference in the red channel.

[bipulmohanto]

Hi Bipul

Thanks for clarifying your aim. Your general understanding of the basic process seems sound enough.

If you are using Photoshop/ACR or Lightroom to process your raw files, you will need a .dcp profile rather than an ICC profile for camera calibration. (or you can like most people use those supplied by Adobe). Some of the converters such as RAWTherapee (free), Capture One, and AfterShot Pro use an ICC profile for camera calibration.

It's not clear to me what raw processing software you are using, or are you trying to produce your own? If you are, be prepared for a long and difficult development road!

Argyll is fairly powerful and highly regarded although it has a fairly steep learning curve I'd say. It doesn't produce dcp profiles, only icc. The other software you could look at to produce profiles is DCAMProf. Have a look at this link for some good detail on camera profile creation.

I'm not familiar with the REZ Checker target but you would need to check that it can be used with the CMS software you use.

As for your last question, the camera profile can be used for all images taken with roughly the same illuminant. White balance needs to be adjusted for the particular illuminant for each image. The RAW file EXIF contains details of the WB multipliers the camera selected when taking the shot.

Some people like to have several camera profiles for a few different illuminants. Adobe uses a dual illuminant profile which includes matrices for D65 and CIE A. An interpolated matrix is calculated from these depending on the actual color temperature of the illuminant.

One other source of code for raw conversion is open source Libraw. I haven't used it but it does contain a large number of code modules for various tasks.

Dave

Hi Dave,
Your suggestions giving me more and more clear view over the color management. So far I did not use any raw processing software, after using Libraw for image reading I developed simple code for grayworld algorithm (w.b.) and bilinear interpolation. Then I was in dark how I can generate the camera profile. I actually wanted to create my own pipeline and compare how it is behaving with respect to the commercial software. I will keep in mind your suggestion about DCP and ICC profile.

If you do not mind, could you suggest me any good literature or book, or tutorial to increase my knowledge. I have read some literature, but those I found more theoretical then practical.

Have a good weekend.

[bipulmohanto]

Ted is not a Photoshop user, so may not be able to help.

The profile is applied during the raw conversion stage and in Photoshop the default is Adobe Camera Raw (ACR). You select your profile here (as shown by the arrow) and make your custom profile your new default. Your CS 6 ACR will look a bit different than my CC 2019 screen, but the process is the same.

As you can see, I use a custom my custom D810 profile as my default. The difference between it and Adobe Standard are slight. When I used the D800 there was a noticeable difference in the red channel.

Hi Manfred, I am really so thankful to you and everyone, these questions were killing me and I found nobody around me to help on color management, so I just googled and googled. Now I have more clear view on color management.

Have a very good weekend.

[Manfred M]

Hi Manfred, I am really so thankful to you and everyone, these questions were killing me and I found nobody around me to help on color management, so I just googled and googled. Now I have more clear view on color management.

The material out there is often contradictory (a lot of it is simply incorrect) and either too technical or not technical enough. It really depends on where you are trying to go with this from a photographic standpoint.

The part that technical people often seem to forget is that modern photography deals with three separate areas; physics, physiology (i.e. the way the human body works and in photography the eye / brain interactions) and psychology. If you concentrate on one of these aspects without considering the others, your work will suffer.

No matter how well we can nail down the physics / technical side, we are fighting the other two aspects. Our biggest enemy is physiology and a characteristic of how the human visual system works called chromatic adaptation. No matter how imperfect our colours are, so long as they are reasonably close, we will think that they are correct and without the experience and training. All the work you are looking at doing on nailing the camera profile can easily be fooled by other issues in your process flow. The biases in your computer screen, the light level in the room you are working in, the screen brightness, the colour of the walls and your desk all have a greater influence on what you see than the camera profile you are looking at.

The psychology also plays a major role. Many people prefer the look of slightly warmer tones in portraiture than strict colorimetry would suggest. Likewise there are regional difference in our preferences; North Americans tend to prefer a slight red bias whereas a number of East Asian cultures will tend towards a slight yellow tint.

One of my photographic mentors who is well connected in the international photographic community has a mantra that I have come to accept: "Colour is an opinion".

The final issue is related to physiology. Not everyone sees colours the same way. In general there is a significant part of the male population (and a smaller portion of the female population) that have defects in their colour vision. In general, our colour vision deteriorates as we age.

If you want to get an idea as to how good your colour vision is, you might want to try this test that x-Rite has published:

https://www.xrite.com/hue-test

[bipulmohanto]

physics, physiology (i.e. the way the human body works and in photography the eye / brain interactions) and psychology.[/B]

Thank you for sharing, your view will really inspire new people who are interested working in this field. I will keep mind your words.

[dje]

If you do not mind, could you suggest me any good literature or book, or tutorial to increase my knowledge. I have read some literature, but those I found more theoretical then practical.

Try these Bipul

https://www.argyllcms.com/doc/ColorManagement.html

https://www.ludd.ltu.se/~torger/phot...profiling.html

https://www.adobe.com/content/dam/ac...ec_1.4.0.0.pdf

https://www.adobe.com/support/downlo...g/dng_sdk.html

http://dougkerr.net/Pumpkin/#Optics

http://www.color.org/iccmax/index.xalter

https://en.wikipedia.org/wiki/CIE_1931_color_space

Dave