RGB or Adobe RGB: Raw or DNG?

[ScoutR]

Hi Everyone: Up until now I have just been shooting JPG. I plan to start using the RAW format, and will use Adobe Lightroom for PP. In Lightroom I have the option to convert to DNG format. What are the advantages/disadvantages of RAW vs DNG?

Also, on the camera I have the choice between SRGB and Adobe RGB. Right now I just use SRGB. What determines which setting I should use.

I do plan to read up on these things, but as usual, I'm putting the cart before the horse, and want to start shooting using the settings that will give me the best quality, even though I'm not really up to speed on all the different formats.

[Dave Humphries]

Hi ScoutR,

Both questions are right up Colin's street

So I should defer to, and await, his reply.

But that's not my style

The choice of RAW or DNG won't really affect quality, but arguably DNG has other benefits.
I'll be lazy (that is my style) and let Colin recant the reasons (he knows them off by heart) and explains them so much better than I.

I believe the sRGB vs Adobe RGB will also have little effect on quality. As long as the same profile is used throughout for processing AND viewing. However, problems can arise when you use Adobe RGB and publish on the web (e.g. here) and people view the image with a non colour managed browser (like IE - and many others without corrective action) - it will look significantly desaturated. So you usually have to remember to convert and save in sRGB before web publishing.
Thus (I would say - but I may be wrong) it can be safer/easier to stick to sRGB throughout (I do) - unless you are considering a pro career with commercial print publishing, I doubt the benefits of Adobe RGB would be noticed.

Colin, How'd I do with the short answers?

Cheers,

[Colin Southern]

Hi Everyone: Up until now I have just been shooting JPG. I plan to start using the RAW format, and will use Adobe Lightroom for PP. In Lightroom I have the option to convert to DNG format. What are the advantages/disadvantages of RAW vs DNG?

DNG has a number of advantages ...

- Because Adobe products (and many others) understand it, they can write adjustment directly to a DNG file, whereas with camera manufacturers RAW files they have to write these to a companion "sidecar" file, thus doubling the number of files and having a requirement to always keep the two together (there is another options to write changes to an internal database on your PC but it would NEVER get backed up and thus is very risky).

- Because it's an open standard, other manufacturers can open your files if - say in 20 years - Canon stop supporting files shot on the likes of the 300D.

- If you use their free converter then you can move the files from your card reader - convert them to DNG and rename them all in one operation

The downside is that some programs (like DxO optics) won't accept a DNG file as an input file, although with the DNG converter you can embed the original CR2/NEF RAW file if file size isn't an issue.

Also, on the camera I have the choice between SRGB and Adobe RGB. Right now I just use SRGB. What determines which setting I should use.

In short, sRGB will keep you out of trouble, but at the expense of clipping some colours - that you probably won't notice & your monitor won't be able to display anyway - but a printer may be able to print. Adobe RGB captures a wider-range of colours (that you won't be able to display on your monitor) - but MAY be able to print. The "gotcha" is that if you use Adobe RGB then you need to convert to sRGB for anything displayed on a website or the images will look dull and less saturated.

If you're shooting RAW though, it's a moot point as neither colour space applies to the capture; colourspaces are only applied during RAW conversion at which point you're better off saving in an even bigger space like ProPhoto -- SO LONG AS YOU REMEMBER TO CONVERT TO sRGB BEFORE YOU POST TO THE WEB, or ADOBE RGB BEFORE YOU PRINT (and even then, your average photolab still doesn't support Adobe RGB).

The good news is that conversion is a trivial process (Edit, Convert to Profile) - one click with an action.

[Colin Southern]

Hi ScoutR,

Both questions are right up Colin's street

So I should defer to, and await, his reply.

Sorry Dave - I was busy adding a poll to a mini-competition - hit the button - and discovered that you'd beaten me to it!

Colin, How'd I do with the short answers?

Great - how'd I do with the longer ones?

[Dave Humphries]

Great - how'd I do with the longer ones?

You done well - I learnt/re-enforced a little more knowledge (pro-photo).

Cheers,

[ScoutR]

You both done good. I know what to use and the horse is catching up to the cart as far as understanding the differences between the formats and colour options.

Many thanks to you both

[Davey]

I personally think open standard in particular are one of the biggest strong points since one of the primary reasons for shooting raw is long term view because it's more future proof than jpeg so it goes hand in hand. Also compression is a big thing as dng has very good compression rates. I have converted some of my pics but I don't tend to convert to dng lately because of no space issue and due to intergration issues with fuji software and applications.

This isn't just processing software since I usually use adobe products or the things that don't support dng don't always support .raf either, but I do have drivers in windows (and some linux side things to do with dcraw since I run dual boot) that mean the embedded jpeg in my .raf files show as thumbnails. No doubt I could do something similar with dng but I haven't looked into that yet since my current way works for me and is slim and simple method.

Incidentally .rar and a few other compression methods will usually archive your original raws to the same size as .dng so compression wise it's not the only option although arguably easier. Still I like the free converter and open standard but feel it's early days and dng doesn't have broad support yet so IMHO there is no real overwhelming benefit to convert all your photos to .dng although it wouldn't be a mistake there isn't a whole world of benefit yet is all I'm saying (but could be wrong it's just opinion).

Lastly if you never use adobe products and tend to rely on your manufacturers in house software then dng would probably be a mistake. I don't since fuji software especially for my obsolete entry level cam is rubbish being honest but I admit the colour response curves, particularly the blue channel and saturation presets in general tend to be wrong "out the box" in ACR comparred to fuji software(yes I have adobe cam profiles installed and my cam is listed). This point may be minor since it was just a matter of noticing it and adjusting the defaults once but for some, especially with powerful inhouse software or unusual sensors etc, it could be a major point.

All things considered I for one do hope it will be widely adopted and hopefully cameras will natively support it in the near future as one standard could help as long as technical considerations are met.

[eNo]

Colin stated the benefits of DNG very well, but I'll throw a couple of counters just to have the full picture. Though I do believe that having an open standard for RAW files is a good idea, and when one is _truly_ established it would also be a good idea to migrate our RAW files to it,

1) There's no guarantee that this open format will be any more supported than the original RAW, especially when projecting far into the future. Yes, Photoshop/Adobe are well-established and will likely survive for some time, but projecting that their products and standards will extend through a 20-30 year timeframe is a bit chancy.

2) DNG right now is not supported by other applications. Open standard or not, if I can't open it in my editor of choice (PSPX2 in my case) it's of little use to me.

3) Since manufacturer RAW converters (Nikon for instance) do not convert directly to DNG, we are left to depend on products like Adobe ACR that do not bring across all manufacturer settings (for Nikon, Picture control and White Balance come to mind).

4) With disk space the way it is, TIF is a good "portable" alternative, althoughbeit without the capability of saving and reversing modifications.

All that said, the challenge to archive our photos in a format that will be accessible many years down the line is real, and finding a medium and format that lets us preserve our work into the far future a worthy concern. For now, IMO, a complete solution does not exist.

[Colin Southern]

There's no guarantee that this open format will be any more supported than the original RAW, especially when projecting far into the future. Yes, Photoshop/Adobe are well-established and will likely survive for some time, but projecting that their products and standards will extend through a 20-30 year timeframe is a bit chancy.

The joy of an open format is that anyone can write (or have written) a program that can read it because "how it works" is open for all to see.

DNG right now is not supported by other applications. Open standard or not, if I can't open it in my editor of choice (PSPX2 in my case) it's of little use to me.

I believe that support for it has picked up a little, but still a long long way to go.

Since manufacturer RAW converters (Nikon for instance) do not convert directly to DNG, we are left to depend on products like Adobe ACR that do not bring across all manufacturer settings (for Nikon, Picture control and White Balance come to mind).

Um, the idea is mostly for a RAW converter to accept DNG as an input file, not write to it as an output file - perhaps this is what you meant? Whilst it's true that ACR doesn't honour picture styles metadata tags, I also have to say that in my opinion, picture styles are really only useful for "tweaking the recipe" when shooting in-camera "ready-to-eat" JPEGS.

One could argue that they provide a "starting point" for any RAW conversion, but since picture styles are really nothing more than pre-defined changes to the likes of saturation, contrast, sharpness etc all it really amounts to is having to drag a slider (that you'll invariably have to drag anyway) a little further in ACR, and you can apply these tweaks to all images with a few mouse clicks.

With disk space the way it is, TIF is a good "portable" alternative, althoughbeit without the capability of saving and reversing modifications.

Not really - once an image is in a TIFF format then it's already been through RAW conversion, and adjustments become a lot more lossy.

All that said, the challenge to archive our photos in a format that will be accessible many years down the line is real, and finding a medium and format that lets us preserve our work into the far future a worthy concern. For now, IMO, a complete solution does not exist.

Keep in mind too that you can embed the original RAW file in a DNG if you wish (personally I don't bother - I just delete the original RAWs).

[KentDub]

I don't subscribe to the argument that DNG is "the" archival format. Enough photographers out there keep the origional RAW's off of their camera that as software and techonlogy evolve two things will happen:

1. Backwards compatibility will be kept as long as possible, when that stops #2 will happen
2. Some software company will make plugins and/or converters, there will easially be enough demad for this to happen (Don't believe me? Look at how many programs out there can open current RAW formats - it was accomplished by reverse engineering the file, proving it's not that difficult)

Since DNG does not support all manufacturer data that is in the origional RAW format - I find that is enough for me to not convert my library to DNG, regardless of if I really *need* a specific setting or feature. The two items above apply to DNG as well - so don't think I'm trying to make a mass argument againt DNG. I'm suggesting that DNG dosn't really offer archival benifits. How many RAW pictures from your older camera's are you now unable to open because technology evolved? I'm betting zero.

As DNG becomes more widely accepted and matures, it may become an ideal format just so you can open an image in a variety of programs, raw processors, and converters. This is just not the case at this point in time though. Yes you can embed the origional RAW file - I'm usually liberal with disk space, but I'm already eating almost 30mb/image - I don't need to double it for just a different file extension.

TIFF files are secondary to the origional. TIFF is good for saving your retouched/PP'd image as an archival format. I would still hold on to the PSD though, item #1 applies here (most likely #2 as well).

I would also like to point out that although we keep referring to the term "archival", we are really only referencing how programs may or may not be able to open them in the future. The file formats themselves, are not archival formats. RAR, a file compression format, has special archival options (You must explicity turn them on). It generates extra data inside of the file so that is part of the file becomes corrupt, it can be regenerated (Kind of similar to how a RAID5 is able to recover from a completely failed hard drive).

So let's say you go to DNG and burn it onto an archival-grade DVD. The DVD gets scratched and you can't open your files. You can spend $4,000 to go to a data recovery center, but they can't always recover everything, if even at all. Lets take another scenerio. If you take your DNG and compress it inside of an archival RAR file. The disk gets scratched and the file becomes corrupt. WinRAR still opens the file and is able to scan it for it's recovery blocks and, with quite a bit of computation, is able to regenerate the DNG for you.

Just betting on a format that will exist is not enough. The type of media you store it on is very important, and there are various options and recommendations out there. Lots of people will recommend every 5 to 10 years you move the images onto the current storage technology of the time, and that's fine. True archival requires you use a file format that has archival (recovery) properties to it - so if you truely want to archive, don't forget about this part.

There are lots of software developers in the world that always have future-proofing and backwards-compiatiblity in mind. Photographers need to be aware how to store their files for archival, but don't need to obsess over "what-if's" regarding file formats. It's simple: If it's widely used today, it will work tomorrow. If it isn't widely used today - convert the format.

[Colin Southern]

Since DNG does not support all manufacturer data that is in the origional RAW format - I find that is enough for me to not convert my library to DNG, regardless of if I really *need* a specific setting or feature.

They DO support all data - including manufacturers "secret sauce" encrypted fields; the issue is only that programs other than the OEM's don't know what to do with it. The original specification didn't support black-masked pixels, but that's long since been fixed (and they were of extremely limited use anyway).

How many RAW pictures from your older camera's are you now unable to open because technology evolved? I'm betting zero.

Zero that I know of at the moment, but that doesn't mean that it won't be an issue in the future. How much support can you find now for - say - someone's will that is held in a WordStar 3.3 file or perhaps one of the earlier (DOS) editions of WordPerfect?

So let's say you go to DNG and burn it onto an archival-grade DVD. The DVD gets scratched and you can't open your files. You can spend $4,000 to go to a data recovery center, but they can't always recover everything, if even at all. Lets take another scenerio. If you take your DNG and compress it inside of an archival RAR file. The disk gets scratched and the file becomes corrupt. WinRAR still opens the file and is able to scan it for it's recovery blocks and, with quite a bit of computation, is able to regenerate the DNG for you.

Sorry, but I don't get your point with this; having multiple backups at geographically diverse locations is fundamental to any backup strategy.

[KentDub]

Sorry, but I don't get your point with this; having multiple backups at geographically diverse locations is fundamental to any backup strategy.

My point is that backup is not the same as archival - they are similar, yet different beasts. Archival is primary, long-term storage. Backup is secondary, mid-term storage. I completely agree that backups should be geographically diverse.

But let's say you burned all of your disks using the same brand (which is very likely). Let's say the batch of disks you got was bad. All of the disks would have, most likely, the same lifespan before then began to corrupt. You won't know immediatly when they start to corrupt, as they are for long-term storage, and arn't accessed until needed. It is then likely that all of your geographically dervisified backups are all corrupt. If you wrap your files inside of an archival format (such as a RAR file with all archival settings turned on), you will have a much, much, easier time recovering the files.

We have backups as a safeguard against loosing the primary storage. Two situations that benifit from using an archival format are: 1. What protects the backups from corrption? Geographically dispersed backups safegaruds from a lot of major concerns, but is not a total solution. 2. When backups become primary storage (archival), what again is their protection against corrption? Archival file wrappers are the answer, of course incombination with a solid backup plan.

True archival of files requires more work than just a solid backup plan - you can think of it as the next step.

[Colin Southern]

I don't disagree with what you're saying - however although adding redundant (read "archival") information to files gives an additional degree of protection (which CD and DVD format specifications also have by the way), it's not a bullet-proof solution. With a RAID solution you're writing data to seperate media; with your "RAR Archical" solution you still have the data on the one single media (and there's probably an argument that the resultant larger data set increases the chances of any given portion being unreadable due to media failure).

As a case in point, each of my final images (along with associated bulk DNG files) is on a pair of RAID 1 drives - backed up to an additional HDD - and written to 2 DVDs, kept at seperate sites. Additionally, space permitting, previous work is also written to these DVDs (ie "fill the DVD up with other stuff"). And I'll also be adding a tape drive shortly.

I'm not saying that writing "archival" info is a bad thing, but to be honest, I really can't see it adding anything significant to the way I'm already doing it.

[eNo]

The joy of an open format is that anyone can write (or have written) a program that can read it because "how it works" is open for all to see.

I believe that support for it has picked up a little, but still a long long way to go.

Indeed. An open door is only useful if people actually walk through it. That's the problem with free open formats: people must actually use them.

Um, the idea is mostly for a RAW converter to accept DNG as an input file, not write to it as an output file - perhaps this is what you meant? Whilst it's true that ACR doesn't honour picture styles metadata tags, I also have to say that in my opinion, picture styles are really only useful for "tweaking the recipe" when shooting in-camera "ready-to-eat" JPEGS.

Actually I meant that something must convert from native RAW to DNG, and that something should bring across the settings that come along with the RAW. We may quibble about Picture controls, etc., but WB loss is *not* an option for me, and guessing at it in ACR or like RAW-to-whatever converter is not something I plan spending valuable time on.
Case in point: the original scene had challenging mixed lighting, and I spent considerable effort and time (gray card, Live view adjustment, etc.) getting the WB right. If converting to DNG will negate my hard work, no thanks. Personally, even when shooting RAW, I also like to preserve my other "recipe" controls, regardless of whether I'm also recording JPG. This is a preference and not a must-have, but again, for me, any method that throws away what I spent on-scene to fine tune is by default unappealing.

Not really - once an image is in a TIFF format then it's already been through RAW conversion, and adjustments become a lot more lossy.

Yes, but at least most applications can open a 16-bit TIF.

[Colin Southern]

DNG is simply a way of organising the data contained in the manufacturers native format, in a standardized and openly documented structure; no data is lost - white balance tags are included and read by Adobe Camera RAW just fine

Picture styles are also included, but Adobe don't use them regardless of whether the input file is DNG, NEF, CR2, MRW or any of the other million out there. So far only manufacturers packages support picture styles - but they typically don't support DNG, which is really the crux of the issue (Probably to stop Canon users from using Nikon software and vice-versa).

Sorry if this is a bit brief - typing on my iPhone whilst waiting for the family to finish shopping!