Adobe DNG Conversion Discrepancies

[ajax]

Have recently been considering the use of Adobe DNC Converter (ADC) to reformat the raw files from my cameras. I'm pretty sure lots of you make pretty extensive use of Adobe products whereas that is NOT the case for me. Hoping some of you may also use ADC and can explain some findings that I find peculiar.

My understanding, which is clearly lacking, is that the basic idea behind ADC is to provide a standard format for raw files where camera makers use proprietary formats for such files. To me that would mean you still end up with the same data just formatted differently. However, after just a little bit of experimentation that does NOT seem to be the case. Discrepancies found to date include the following:

First, metadata quantity appears to be drastically different. For example, I used ExifTool to extract metadata to a text file. The resulting file for a camera produced raw file contains 543 lines. Doing the same thing for the file created by ADC contains 221 lines. While I wouldn't expect the files to be exactly the same because it does seem to be appropriate for ADC to include items that pertain to what it has done. However, my logic would reason that this would make the file larger rather than dramatically smaller. A somewhat casual comparison of the files finds that the camera produced file contains a pretty sizable chunk of data labeled "Quicktime". That sort of implies, to me, it has something to do with video. While the camera is capable of taking videos this image is a still photograph. Possibly Adobe Digital Negative (DNG) format either does NOT support video or it handles it differently which suggests that could be legitimately omitted but that only accounts for 121 lines. What is of more concern is that something called "Makernotes" is completely absent in the converted (DNG format) file. I'm NO expert on metadata but I think "Makernotes" contain camera specific data that varies (possibly significantly) from one camera maker to another but it very much includes meaningful information about the photograph. In that, the nature of the data is proprietary but I would expect and want it to be retained. The "Makernotes" amount to 217 lines of metadata for the camera produced file.

Therefore, the combination of "Quicktime" and "Makernotes" data accounts for enough data to pretty well explain the discrepancy in the total amount of metadata. As mentioned, even with my limited knowledge, leaving out "Quicktime" makes sense but what might the rationale be for omitting "Makernotes"? Of course this reasoning is the result of my somewhat casual observations. Possibly this has little or nothing to do with the actual logic behind ADC.

Second, it looks like ADC makes its own preview image. This even includes choice of full, medium or none when it comes to preview images. However, as best I can tell image size is the same for both full and medium even though full does take more storage. Anyway, what troubles me is that neither of these were produced by the camera and to my way of seeing things that is what the preview image should be. In the case of my files, which by the way are Canon CR2 & CR3 type, the image size of the preview image is quite a bit bigger but again I'm NOT so much after bigger as I am authentic camera produced.

With that said I do see advantages for using such a standard format even though I do NOT have anything but Canon cameras at present. I'll spare any discussion of that but would like to know what knowledgeable others think about these discrepancies.

[pschlute]

I have been capturing my in-camera images in DNG format for over 15 years now. I thus have no need for any DNG conversion.

I see no need to limit myself to the camera manufacturers proprietary raw format.

[Polar01]

Just wondering what product do you use to post process your images, myself I use Adobe Bridge to process the raw files than open in Adobe Photoshop, I have no need to convert to a DNG file.

Cheers: Allan

[xpatUSA]

There is no one single DNG format and there are a good few variations in how a "DNG" is put together. Some quite major - for example "Linear DNG" can't be opened by everything. So file size is quite variable. If I had Canons I would stick with their stuff.

DNG hasn't taken over the world yet, and TIFF lives on in a few places,

[DanK]

Why bother? Canon is the largest camera manufacturer in the world, for the 19th year in a row. They aren't going anywhere anytime soon. And if they did, with this volume of .CRx files out in the world, there would be a strong incentive to keep supporting them for a long time. And if I live to be 150 and that's no longer true, I can convert to DNG then, assuming that Adobe outlives Canon, which is not a sure thing.

Anyway, what troubles me is that neither of these were produced by the camera and to my way of seeing things that is what the preview image should be.

One more reason not to bother.

I've been shooting mostly with Canon bodies for 14 years and have never converted a Canon raw file to DNG unless there is something specific to be gained. For example, if you want to share a raw file edited in LR or ACR and want it to include the editing history, exporting to DNG will accomplish that. And I end up with DNG files when I use my drone or do an HDR merge in LR. Other than that, I see no benefit to spending the time converting the files.

[ajax]

I have been capturing my in-camera images in DNG format for over 15 years now. I thus have no need for any DNG conversion.

I see no need to limit myself to the camera manufacturers proprietary raw format.

Well I didn't know that was even possible. When you say "in-camera" you do mean your camera is producing raw files in DNG format. Yeh? If so, is that an option you can choose or is the camera maker simply adopting DNG format rather than creating their own format? What kind of camera?

[pschlute]

Well I didn't know that was even possible. When you say "in-camera" you do mean your camera is producing raw files in DNG format. Yeh? If so, is that an option you can choose or is the camera maker simply adopting DNG format rather than creating their own format? What kind of camera?

Hi David

I use Pentax DSLR cameras (yes there are still a few of us around !)

Yes the camera has the option to produce raw files in either the native Pentax format .PEF or .DNG

[ajax]

Thanks to everyone for feedback. Very helpful.

I have to conclude that there is NOT much enthusiasm for using Adobe DNG Converter in this group. It also appears that none of you may even have gotten to the point of recognizing what I'm labeling as discrepancies.

As a novice developing pictures as a hobby for my own use I've been reluctant to pony up the money required to use Adobe products for post processing. Even if it weren't for the money the question becomes what product to invest time and effort learning to use. However, there is also no shortage of what I would call the public license (i.e., freeware) products to choose from. The one I've used the most for the past several years is called Rawtherapee. Unfortunately, it does NOT fully support the files from my new Canon camera. What Canon calls CR3 format. Insofar as all of the other products I either use or have experimented with have NO problem with CR3 format files the Rawtherapee problem is a bit baffling.

As best I can tell the problem is limited to NOT being able to read CR3 metadata. Based on discussion herein I'd say DNG is NOT a good solution. Therefore, investigating methods of handling metadata separately is what I'll be looking at next.

Thanks for your help.

[Manfred M]

One has to remember that DNG is just a "wrapper" for the raw data and is based on the TIFF structure (TIFF is actually owned by Adobe as well). Like Dan, I have found no particular advantage to me and in fact the conversion to DNG takes time, when I could be diving into the images that I shot. I seem to remember a few camera manufacturers; Pentax and Leica come to mind, support it natively.

The other advantage is that Lightroom and Adobe Camera Raw (and I assume other parametric editors) can write to the format and store those edits. That alleviates the need for the catalog or sidecar xml files to store this information, making this format slightly more portable.

The use of DNG is a personal choice. Some love them others do not...

[DanK]

Well I didn't know that was even possible. When you say "in-camera" you do mean your camera is producing raw files in DNG format. Yeh? If so, is that an option you can choose or is the camera maker simply adopting DNG format rather than creating their own format? What kind of camera?

Sorry, I mistyped. I meant in raw format. The only camera I have that stores raw captures in DNG format is my drone. I've stored files in both Canon and Panasonic raw files for many years and have never seen a reason to change that.

[DanK]

Thanks to everyone for feedback. Very helpful.

I have to conclude that there is NOT much enthusiasm for using Adobe DNG Converter in this group. It also appears that none of you may even have gotten to the point of recognizing what I'm labeling as discrepancies.

As a novice developing pictures as a hobby for my own use I've been reluctant to pony up the money required to use Adobe products for post processing. Even if it weren't for the money the question becomes what product to invest time and effort learning to use. However, there is also no shortage of what I would call the public license (i.e., freeware) products to choose from. The one I've used the most for the past several years is called Rawtherapee. Unfortunately, it does NOT fully support the files from my new Canon camera. What Canon calls CR3 format. Insofar as all of the other products I either use or have experimented with have NO problem with CR3 format files the Rawtherapee problem is a bit baffling.

As best I can tell the problem is limited to NOT being able to read CR3 metadata. Based on discussion herein I'd say DNG is NOT a good solution. Therefore, investigating methods of handling metadata separately is what I'll be looking at next.

Thanks for your help.

When manufacturers change their raw formats, generally with a new line of cameras--like Canon's change from CR2 to CR3--people writing software need to update their products to read the new format. This generally happens very quickly with major paid products, as your $$ pays the people who need to write new code. In the case of freeware, this has to wait until someone volunteers to do it. So one of the costs of using freeware is that you may need to use an intermediate step, like converting to DNG, until someone gets around to fixing the problem.

[xpatUSA]

TIt also appears that none of you may even have gotten to the point of recognizing what I'm labeling as discrepancies.

Well, excuuuuuse all of us!!

[xpatUSA]

The one I've used the most for the past several years is called [RawTherapee]. Unfortunately, it does NOT fully support the files from my new Canon camera. What Canon calls CR3 format. Insofar as all of the other products I either use or have experimented with have NO problem with CR3 format files the Rawtherapee problem is a bit baffling.

RawTherapee is still at v5.8 (Feb 2021) so that is probably why. As has been said, apps from outside the pale don't always jump on new raw formats right away.

[pschlute]

It also appears that none of you may even have gotten to the point of recognizing what I'm labeling as discrepancies.

I think it comes down to what you mean by discrepancy.

I have never investigated "lines of data" that you referred to, but do know that you will get a vast difference in file size using different formats. A camera's native raw format can be wildly different from the same image captured in .DNG. When converting either to a TIFF, usually produces a much larger file size often twice as large.

In my experience none of this matters. The quality of the output (in whatever format you require) is the same.

One of the advantages of using .DNG from the outset is that you can mostly process the image in any software. If you use native raw format, you are restricted to using either the manufacturers software, or waiting until your proprietary software is updated, or using a DNG converter

[ajax]

Well, excuuuuuse us!!

Sounds like Ted thinks I was demeaning him in some way. My apologies for that. My real intention was quite the opposite. From the discussion herein I developed the idea that my so-called discrepancies were minutia when compared to the various rationales offered for discouraging the use of DNG format files. Therefore you guys saved yourself the time and effort I invested in making these discoveries. Hats off to you.

In my own defense, I would say that I'd previously learned about Adobe DNG Converter but never thought much of it prior to experiencing a problem that I, erroneously, thought it might fix. However, I'm still with you and do appreciate the help in figuring it out.

It seems this kind of thing "goes with the territory" so to speak when dealing with freeware. Based on other discussions it does look like the products I'm using will eventually catch up and I have found a workaround that adds a bit to my workflow overhead but that is NOT as large a burden on me as it might be on you.

Thanks again!

[xpatUSA]

Originally Posted by ajax It also appears that none of you may even have gotten to the point of recognizing what I'm labeling as discrepancies.

Sounds like Ted thinks I was demeaning him in some way.

Not just me, David: quote "none of you" unquote included all of the responders in this thread up to that magic moment ...