The RAW/JPEG Discussion

[Glenn NK]

Manfred:

I occasionally read comments from Canon users that DPP converts images better than Adobe (CS and LR), but every time I've attempted to use DPP, I quit in frustration. While it may convert well (and I agree that it should - who else knows the output from a Canon sensor better than Canon?), it is a total pita to use.

The (Canon) guys that wrote the program are more likely to be digital/electronic engineers than photographers, and this just doesn't give them credibility when it comes to actually working with images.

Perhaps Nikon software is more intuitive, but the best photographers whose work I see regularly are all using CS whatever version (and about 1/2 of them are using Nikon).

Glenn

[Colin Southern]

ACR does not seem to do as good (by that I mean accurate) a job as View NX2 / Capture NX2 in converting RAW images out of Nikon cameras. I have noticed that several other Nikon shooters on this site have come to the same conclusion. I find this perfectly plausible, Adobe uses a different RAW conversion engine than Nikon does, and it has to be generic, by design. Adobe has to work reasonably well for all brands of cameras, whereas the Nikon software has to work well only with Nikon products. I would suspect that Nikon engineers working with Nikon software designers would get that part right, even if the resulting software is not as slick, or does not have all the bells and whistles as ACR / Photoshop. The best way to go for Nikon shooters who want to go to a level beyond jpeg is to use the Nikon supplied software to make the conversion to TIFF and then edit the files in Photoshop.

Hi Manfred,

I'm about done with this thread, but I will just pass on one final observation ...

You're right in that that - out of the box - ACR has only what I'll call "ballpark colour accuracy" (keeping in mind also though that they do supply a number of generic profiles under the camera tab that I'd be willing to bet that 99% of folks aren't even aware of - including most of those who criticise the quality of the product) (not "aiming that" at you or anyone else by the way ... just an observation I've made through reading thousands of posts on the topic of ACR).

If folks are SERIOUS about accurate colour then they SHOULD be creating camera / lens / lighting profiles to use in ACR (or NX2 or any other package if it supports them) - and that goes for Nikon as well as Canon users (and other brands too of course). Again (just like processing presets) - folks really need to be investing in a little education and a little time to create such things as camera profiles and processing presets; the leverage that making those investments returns is phenomenal and - in my opinion anyway - would result in far greater acceptance of and reliance on and far less criticism of ACR.

[Simon Garrett]

Just to add to the comments from Manfred and Colin,

I would agree that Nikon software (View NX / Capture NX) gives results close to the out-of-camera jpeg, but not necessarily that they're more accurate than Adobe (Lightroom/ACR) when using Adobe's "Camera" profiles. But I know some people prefer the Nikon rendering.

However, some other aspects of Adobe raw conversion are now much better than Capture NX2, which has advanced little since it was released 4 years ago. In particular, Lightroom 4 / ACR7 provides significantly better tone mapping and noise control than Nikon software. I have examples of Lightroom 4 recovering detail in highlights and shadows I just can't get with NX2.

[Manfred M]

Colin, I’m quite aware of the tabs and while it does not have all of my lens / camera combinations, it has most of the major ones. I was using DxO 6 before Adobe came out with the profile functionality and found that in general it is more complete set of lens / body combinations and does a better job with the lens corrections. I did some testing on CA a while ago, and while Adobe gets rid of most of it, DxO got rid of all of it in my samples. This was on RAW images only, Nikon does a great job in-camera for jpegs. The downside of DxO is that it is not as user friendly as the Adobe software and is a standalone package and is not blazingly fast.

With respect to your comments on the ACR profiles, I certainly see the distortion correction, CA correction and drop off correction work quite well for the preinstalled profiles. Colour profiles, I’m rather suspicious about. There aren’t any profiles for my Nikkor 80-400mm lens for either of my cameras in ACR or Lightroom, so I dove into the instructions on creating my own.

I downloaded the Adobe profiler software and printed out the target; so far so good, but the actual profiling didn’t appear look at anything other than the geometry data gathered from the target. The colour of the target paper and the lighting used was not base lined in any way. The procedure that Adobe lays out will certainly do the lens corrections, but I don’t see anything that really makes sense regarding the colour corrections. I didn’t bother actually creating any profiles yet, because DxO has most of them. One of these days I am going to get around to creating custom profiles for two of my lenses where neither software package has profiles on; the Samyang 8mm fisheye and the Sigma 150-500mm lens.

That being said, the proper place to do any colour correction / profiling would be on the camera and lens assembly lines. I would expect that the camera’s sensor would be calibrated during assembly testing and this would be burned into the camera firmware. Same comment on lens colour issues; Canon and Nikon, certainly over the past decade or two have been manufacturing lenses to very high tolerances so that there is very good colour consistency across the entire product line. The right place for any corrections would again be during lens testing during manufacture, with any correction data integrated into the lens firmware. The camera bodies do carry lens profiles information, so I suspect that this would certainly be the case.

So this time round, I am confused. Adobe publishes a package that is “close”? For the money they are charging it should be right on.

[Macmahon]

...

I downloaded the Adobe profiler software and printed out the target; so far so good, but the actual profiling didn’t appear look at anything other than the geometry data gathered from the target. The colour of the target paper and the lighting used was not base lined in any way. The procedure that Adobe lays out will certainly do the lens corrections, but I don’t see anything that really makes sense regarding the colour corrections.
...

Manfred,
The procedure recommended for creating a camera colour profile involves shooting a standard colour chart. Your printing the target would potentially introduce yet another variable - the print profile - which is likely to render any camera calibration suspicious at best. X-rite, for one, supplies a standard version of the Gretag-Macbeth colour target which the Adobe software can calibrate from. It works. Well.

...
That being said, the proper place to do any colour correction / profiling would be on the camera and lens assembly lines. I would expect that the camera’s sensor would be calibrated during assembly testing and this would be burned into the camera firmware. Same comment on lens colour issues; Canon and Nikon, certainly over the past decade or two have been manufacturing lenses to very high tolerances so that there is very good colour consistency across the entire product line. The right place for any corrections would again be during lens testing during manufacture, with any correction data integrated into the lens firmware. The camera bodies do carry lens profiles information, so I suspect that this would certainly be the case. ...
.

I guess the issue here is how absolute "... very high tolerances ..." are. In the case of high-end printers you could make the same argument, but I've never used a printer whose colour fidelity was not improved by building a unique profile for it, that accounts for any marginal variations that may have arisen in manufacturing. Absolute consistency between individual copies of the same product coming of an assembly line would be nice, but is just a dream, I suspect.

That being said camera manufacturers do get pretty good colour output from camera sensors, just as 'generic' print profiles made by printer or paper manufacturers are pretty good. The correction data manufacturers use is presumably derived from calibration tests on a sample of product. I would guess that Adobe has no better process at its disposal, in terms of deriving camera Raw conversions, other than to measure and build from a sample of product just as the manufacturers must. The result is an average. Each individual copy of a camera, or printer, or lens, (or printer paper) is somewhere on a distribution. The average correction may be good enough for many users. Others will still see a need to do their own calibration and colour profiling.

Cheers

Tim

[Colin Southern]

I downloaded the Adobe profiler software and printed out the target [snip] The colour of the target paper and the lighting used was not base lined in any way.

Thank god! The last thing we need to be doing is profiling camera colour based on a user-printed reference! (eek!).

The easiest way to produce camera profiles is to simply use a product like colour passport - takes about 1 minute.

[Colin Southern]

I would guess that Adobe has no better process at its disposal, in terms of deriving camera Raw conversions, other than to measure and build from a sample of product just as the manufacturers must. The result is an average. Each individual copy of a camera, or printer, or lens, (or printer paper) is somewhere on a distribution. The average correction may be good enough for many users. Others will still see a need to do their own calibration and colour profiling.

Adobe create the base camera profiles for ACR by shooting a standard 24 patch colour checker. The thing folks need to keep in mind though is that the resultant "standard" profile has to be - by definition - "one size fits all" (for a given camera) - the problem is - in reality (and somewhat surprisingly) we actually don't want 100% accurate profiles because how cameras capture and process data is only an approximation of how human eyesight works. As a case in point, a profile that's ideal for portraiture would vary significantly from one that was idea for landscape -- so in reality, IT'S JUST A STARTING POINT (something many seem to struggle with) -- it's up to us photographers to ultimately fine-tune the colour process ourselves; this isn't a reflection of the quality of the product - it's just the nature of the beast we call colour management.

[Sponge]

It looks like Adobe has come out with a 'lossy' version of DNG so now we can add a middle option to the discussion

http://www.dpreview.com/news/2012/10...ssy-DNG-option

[Colin Southern]

It looks like Adobe has come out with a 'lossy' version of DNG so now we can add a middle option to the discussion

http://www.dpreview.com/news/2012/10...ssy-DNG-option

Perhaps they're just playing catch up with JPEG, which sets the benchmark when it comes to lossy!

[ajohnw]

What are the misconceptions you allude to?

Glenn

Basically

1) Camera manufacturers take steps to ensure that dynamic range isn't lost during conversion to jpg
2) That a cameras dynamic ranges seems to be specified in test reports based on the one obtained with jpg's.
3) Any attempt at using the entire dynamic range of a camera in a jpg must result in a departure from reality camera jpg or raw.
4) The extra dynamic range available in raw in real terms just allows for more exposure latitude for later correction.
5) As I have shown the compressed detail can be restored from a jpg just a readily as from a raw file. The last shot I posted has restored more than the raw shot has. Fortunately it wasn't based on the preset used on the raw so lacks the over exposed spot on the right cheek.
6) Detail loss in jpg's? Anyone can take a shot with the finest setting and coarser ones and zoom in and see the results and see which wins. Pixels or detail. Mostly irrelevant anyway at final reproduction sizes. I seen to have recovered more hair from the jpg anyway even though it's has a lower exposure level than the raw file.

If some one is doing commercial work for a publisher things may be a little different but in that case they might want 48bit colour.

My workflow - funflow really. Like many.
I only use raw on shots that need extensive modification where for one reason or the other the camera has made a really poor job of it. Usually dynamic range. Perhaps the most useful aspect of that is fake hdr. One thing for sure. It would be pointless trying to use presets. Wedding and studio shots are a different matter. Conditions are known and controlled. Less so at weddings. Sensible pakages allow pipes to applied to jpg's as well anyway so again there is no difference really. Time? My PC takes less than 2.5 sec to convert a raw. Say I take 100 shots and batch convert assuming I have selected the right preset /pipe. It's easy to work out how long that will take. Say I use a raw package that has a raw preview facility - much quicker thumbnail preview but a preview doesn't really show all that is needed to judge a shot. So I shoot raw plus jpg. I seldom use a package that scans and updates picture files along with raw thumbnail preview in the fist instance. I generally plug the camera in and connect using a file manager, identify the file series via variable size thumbnails and move them to where ever including raw. Then I click on the start of a series and up pops a jpg viewer that I hope continues not to thumbnail raw files. In that I can view fitted to window or anything up 100%. If I right click I can move to or use open with. All of this is virtually instantaneous. Click on next and up pops the next jpg. If one show a problem that I can't cater for from jpg or if raw would look to be easier I use the file manager to isolate the raw file. I have it's reference number visible in the jpg viewer. At some point when things are more finalised I might point a package that catalogues pictures at them. Mixed feelings though. I use several packages for different things and some of them have their own cataloguing system built in which is irritating. So irritating I will probably stop using them if I find I can't turn that off. Fortunately they can all be used directly via open with or better still as plugins to an editing package I use. I have looked around at photo cataloguing software but haven't really found one that suits yet. In many ways for my use there may well be no point in using one anyway.

Just how many times do I have to use raw working like this? Not very often. How often do I modify shots in one way or the other? Very often but a lot depends on the camera. Off a compact nearly all of them. 300D will shoot either raw or jpg and not both without installing a hack. Curiously I have never installed it and only shoot jpg's with it. The vast majority need no mods at all. 5D? Both but to be frank I hardly use it and it's doubtful if I ever will out of the house. It's a MK1. I like the linear part of the camera curve even though it's a mere 7 stops in that respect. Pen? If my masochistic attempt to use it for all forms of photography continues more shots will need processing but a lot less than a compact.. Taking the lady shot (s) as an instance I'm fairly sure it would make a better job of the exposure than the raw conversions in jpg mode. Much depends on lighting. Taking something like the following needs work to bring the detail in the blacks out. Burnt in on this one. Before I calibrated my monitor. It's a camera jpg of course. Overcast day and raining.



And at the other extreme fake hdr from raw needed due to back lighting that would cause other cameras problems

As shot



Result. I could have kept more water detail but was trying this out to cope with a problem that is extreme on a Pen



Most of my photography is like this. Happen to be there and want to take a photo. If I post any in a competition on here it will either be unusual in some respect or an attempt at what I think of as photographic art extracted from something I have shot.. I generally do nothing to a shot unless I like it or others will want to look at them. The duck for instance is gauging whether I am likely to feed it. I like that aspect of it. I also like processing shots. Raw has some severe limitations really as have jpg's. Dynamic range is limited by the medium that a shot is finally shown on. Jpg's often have to have a distorted tone curve to show what is there. Raws are interpolated so in real terms detail is lost. The worst aspect of jpg's really is repeated modification and saving. Saving at 100% quality levels is one way round that.

Just lost this lot. I spilt tea all over the keyboard. Auto store recovered it. Good job. I started at about 10-30am and posted at 5-30pm. No point going through it all again either really.


-

[Manfred M]

Colin, after reading your latest response, it occurred to me that we are writing about two totally different functions in ACR; I was looking at the camera / lens profiling and you are looking at the camera profiles only.

Because of my jpg + RAW shooting, I don't use the ACR camera profiles, as I set the similarly named profiles in the camera, but I do use the camera / lens profiles when I don't want to bother with DxO and do a quick and nasty raw import that does a decent job with CA, etc.

[ajohnw]

I posted some info about a package called Argyll that will calibrate monitors, printers, scanners and cameras. The only reasonably priced route to calibrating a printer is a scanner. My main interest was in calibrating my monitor but I was reluctant to buy a new calibrator etc. I like to do things on the cheap when I can. I bought a calibrator from a mac user who was forced to upgrade due to OS changes and used Argyll. This is a command line tool but does have a GUI called DispCalGui available but but only for monitor calibration. Argll's web site lists which calibrators are supported - most of them actually as they are often just rebrands.

Calibrating the other items needs a test card and the data to go with it. The data specifies the actual colour of the patches. It seems it's reliable for about 2 years. Both Argll's and DcRaw web sites mention a source of test cards. They can be bought either batch calibrated or calibrated individually ( a lot more expensive). The equipment needed to read colour patch values accurately is extremely expensive.

The complication with the cards is that scanners are usually calibrated on glossy ones and cameras on matt. Many sets of instructions for camera calibration can be found on the web. Saves yet another long post. These range from waiting for a clear day and taking shots at true noon to using flash. Either way reflections have to be avoided so many people use gloss cards. Obviously auto white balance isn't a good idea here.

There are more or less line by line instructions for using Argyll under the documentation on it's web site so it should be easy for people to see if they can cope with it. There are likely to be instructions elsewhere as well. A lot of the problems with colour is colour space jargon but in real terms if some one just copies a set of instructions there isn't really any need to under stand them.

On the other hand some calibrators come with a test card. Capabilities vary.

There shouldn't really be any need to colour calibrate a lens. Distortion, chromatic aberration and vignetting are another matter. More an more companies are doing that in the camera. Pity they don't reduce the cost of the lenses to match the savings. This will make Sigma's etc life more difficult eventually. Fortunately some software allows manual correction for all 3. There is no chance of having a standard CA profile that fits all lenses for instance unless the manufacturers calibrate each and every lens so maybe these packages are worth having around. Rawtherapee is one. Others rely on using software to generate the correct numbers for a lens but having used the simple manual method I wonder if that is worth the effort. Unlike me though do it before cropping. It's a problem I have only experienced recently due to using a Panasonic lens on an Olympus Pen. Olympus do little if anything to images in the camera. Panasonic do a lot. More recently they may even play around with raw files. At some point manufactures might start using deconvolution based on lens errors in the camera. Olympus have added a remove vignetting option to the menu's in my replacement E-P3. Curious as most of there current lenses don't have a problem. The standard 14-42mm zoom unusually has virtually zero at all apertures. And so it goes on and on. I wonder what comes next. It was all so much simpler when one company made good equipment and another made crap.

-

[Manfred M]

There shouldn't really be any need to colour calibrate a lens. Distortion, chromatic aberration and vignetting are another matter. More an more companies are doing that in the camera.

The in-camera argument is certainly correct for jpegs, although there is some suspicion that both Sony and Panasonic do this in RAW as well. Nikon has been taking out CA in camera in jpegs for several generations and I have read that Canon has started to do so with the 5D Mk III as well.

Colour differences in lenses even within the range put out by the same manufacturer have been debated ever since I have been involved in photography, and this is a characteristic of the glass compounding process. This is often more noticable in third-party lenses because the majors seem to have been much better at controlling this across different lens models than they used to.

Olympus have added a remove vignetting option to the menu's in my replacement E-P3. Curious as most of there current lenses don't have a problem

Pretty well any fast lens I've ever used, especially wide angle ones do have a certain amount of drop off, especially wide open. It does drop off as you stop down. It's something one doesn't always notice until you enable the ACR or DxO to remove this, and then you start really understanding how much there is in a certain lens.

[jcuknz]

"especially wide angle ones do have a certain amount of drop off, especially wide open. It does drop off as you stop down"

Thankyou Manfred for that