Question about Jpeg in-Camera compression vs RAW

[Nicks Pics]

There are actually a few various things I am wondering about this, but so as not to throw out too many things at once, I will start wit one of them

when your camera is shooting in JPEG, does it only compress images, (take out what it perceives as unnecessary pixels), or does it also do some automated Post processing like boost contrast, saturation etc. I seem to detect that contrast, saturation etc. are usually higher in my JPEG images than an identical RAW, but I don't know if that is because some automated in-camera PP that has been done, or if it is by way of the filtering out the duller pixels, which would concentrate contrast, and color etc. if that makes sense?

If you have any insight about this I would appreciate hearing what you think

Thanks

Nick

[PhotomanJohn]

Nick - You are correct that during the in-camera JPG conversion the camera adjusts a number of things including color balance, saturation, contrast, noise reduction and the like. I don't know what type of camera you have but many DLSR models have settings called picture controls or picture styles that apply different amounts of the various adjustments to give a different look to the JPG files. Many also allow you to adjust the values of the corrections being applied and save them. Time to dig into the manual for your camera.

John

[loosecanon]

but most importantly it turns the 12 or 14 bit raw files which are usually converted into 16 bit files for editing into 8 bit jpeg files, so a large amount of colour information is lost. This of course shows up sometimes as bands in areas of very similar tone such as the sky. It also means if you edit you may see the histogram become spikey.

[pnodrog]

Nick - You are correct that during the in-camera JPG conversion the camera adjusts a number of things including color balance, saturation, contrast, noise reduction and the like. I don't know what type of camera you have but many DLSR models have settings called picture controls or picture styles that apply different amounts of the various adjustments to give a different look to the JPG files. Many also allow you to adjust the values of the corrections being applied and save them. Time to dig into the manual for your camera.

John

Darn. When I read the post I thought here is a question I can answer but then I saw John had already done it.....

[Nicks Pics]

Thanks very much John, I appreciate your response, I have seen those picture adjustments on my new d7100, but I'm just getting started with it. I do hope to spend more time on the manual soon, it's a good book.


Thank you Loosecanon, Very clearly explained, I appreciate your pointing that out to me.

L Paul, Thanks for looking anyways Feel free to drive it in to my head by way of repetition if you wish!

I suppose this brings me to my next question, which is what does the compressed RAW setting do? -I'l look it up now The manual says it records compressed data, as are regular NEF images, but slightly more-so. Therefore, aren't compressed NEF files about the same as a large, un post processed jpeg?

[PhotomanJohn]

Short answers:

The basic raw NEF files contain all of the data from the sensor with some other information included. There is no compression and the files are the largest.

Loss-less compressed raw (NEF) files are smaller and have a relatively small amount of compression which can be reversed by the raw converter so no data is lost.

Compressed raw files are smaller yet and some of the original sensor data cannot be recovered by the raw converter.

JPG files come in various levels of compression with associated smaller file sizes and the data lost during compression can't be recovered.

John

[pnodrog]

Hi Nick my camera gives me two options for compressing RAW (NEF) files. The first option is Lossless which they describe as a reversible algorithm which reduces the file size by about 20-40%. This is how I save most of my images.

The second RAW compression option they just refer to as Compressed which reduces the size by about 35-55% with almost no effect on image quality. This is the option I used on my previous camera when CF cards were smaller and expensive.

The RAW can also be either 12 or 14bit and I always save as 14 bit to keep the dynamic range as high as possible.

[Nicks Pics]

Thanks John and L. Paul
I don't know that my camera is capable of any way to record totally un-compressed RAW files, (or NEF files) It gives me options of "compressed" or "Loss-less compressed) as L Paul indicated about his, Is that the same for your John?

Anyway, my new d7100 seems to be able to shoot about 6 Loss-less compressed RAW shots, and 7-8 compressed RAW shots, though it varies, in burst shooting, and for general purposes this also may be relevant, as RAW files fill a card fast and need a lot of storage space- I'm sure it is mostly up to personal preference or the intent of the particular shot, but do you have any particular opinions about the quality of compressed files vs compressed files?

Thanks

[rpcrowe]

With the relatively low price of memory today, I cannot see any reason for shooting in any but the highest quality (also largest file size) possible...

EXCEPT: an exception may be if you are needing immediate use of the images such as a sports shooter who immediately transmits images to the editor AND/OR if you are shooting with a camera that doesn't allow a burst mode of decent length when shooting in RAW. Happily, my 7D will shoot up to 25 RAW images in burst mode. I don't remembering any need for a longer burst...

[george013]

Nick,
I once made a workflow on how the images are created and could do that only with introducing another image-file: the raster-file where every pixel in the file corresponds with a pixel in the image.
Your RAW-file is a sensor-image, only containing the colors red, green and blue in a certain relation and the camera-settings.
A JPG-file is a compressed raster-file. Pixels of that raster-files are grouped together and given a certain value. How exactly is here unimportant, but be aware that to compress a raster-file, first a raster-file has to be created, totally or partly. This in-camera conversion and compression uses the camera-settings.
This brings up a question you allways should ask yourself when viewing a RAW-file: what do I see. You can't see a RAW-file, only a raster-file. So if you see a raster-file insteed, where does that come from? It can be an embedded JPG made with the camera-settings or it can be a result of an out-camera conversion, with or without the use of the camera-settings.

George

[mrinmoyvk]

I think it depends on the picture control settings you might have done or default settings.
I am talking about different modes like portrait, landscape, standard, neutral.
In Nikon D5200 there are options to tweak brightness/contrast/sharpness etc of each of these and camera will apply the selected setting to every JPG file while saving. Even if I am not customising any of these, manufacturer default settings are applied to JPG file before saving.

It is same as shooting in monochrome with RAW+JPG, JPG will be saved as monochrome due to the settings selected but RAW will have all the colour details.

[george013]

....
It is same as shooting in monochrome with RAW+JPG, JPG will be saved as monochrome due to the settings selected but RAW will have all the colour details.

When you open this NEF in Camera Raw, you will get a coloured image, when you open this NEF in Capture, you will see a monochrome image.

George

[mrinmoyvk]

When you open this NEF in Camera Raw, you will get a coloured image, when you open this NEF in Capture, you will see a monochrome image.

George

Is it? I never tried Capture NX software but I always thought that NEF is direct output of sensor without any processing then why monochrome?

[pnodrog]

Is it? I never tried Capture NX software but I always thought that NEF is direct output of sensor without any processing then why monochrome?

The NEF file basically contains the information as received via the Analog to digital conversion of the sensor information PLUS the metadata giving camera settings, date, time, GPS coordinates if available etc and processing preference.

Capture NX reads the processing preference and uses them where as Camera RAW ignores them. It will depend on conversion software used and options set as to exactly how it is handled. You are correct in that the NEF file contains all the original (bayer) information. The exact structure of the NEF file is propriety information and there is obviously a variation in conversion required from one camera model to another hence having to wait for camera updates for the conversion software.

[mrinmoyvk]

That means it is the preference whether to read the settings or not which depends on the software used for editing.
There must be some way to ignore these settings in Capture NX as Adobe camera Raw does, just for curiosity...
I anyways use Adobe...

[ajohnw]

There are actually a few various things I am wondering about this, but so as not to throw out too many things at once, I will start wit one of them

when your camera is shooting in JPEG, does it only compress images, (take out what it perceives as unnecessary pixels), or does it also do some automated Post processing like boost contrast, saturation etc. I seem to detect that contrast, saturation etc. are usually higher in my JPEG images than an identical RAW, but I don't know if that is because some automated in-camera PP that has been done, or if it is by way of the filtering out the duller pixels, which would concentrate contrast, and color etc. if that makes sense?

If you have any insight about this I would appreciate hearing what you think

Thanks

Nick

Much of that depends on camera settings and on some cameras it will take no notice of noise reduction being turned off. Seems adobe raw always applies some. Then there may be contrast and saturation adjustments or similar. It's almost impossible to determine where in the image processing these are applied. There is a chance for instance that some cameras apply noise filtering to raw data before doing anything else maybe even before applying a colour balance setting. They may then subsequently apply noise reduction to the final jpg as well.

The camera uses raw data which will either have a 10,12 or 14bit colour depth. Some medium format cameras may have 16, possibley even more. Not an area I am interested in. The camera may process this at a higher bit depth, very likely actually. Eventually it then applies a tone curve to the data to generate the jpg and at that point the extra bit depth is lost.

Most cameras these days have a variety of tone curves available such as natural, high key, low key, portrait, landscape etc etc even auto which probably maps the entire raw bit depth into the jpg in some fashion or the other. If you look at camera reviews on Dpreview you will see them. Most modern cameras will map around 9 stops into a jpg in it's "standard" mode. There are likely to be other modes that can get more or less in. They do this by compressing the tonal range some where or the other - much in the same way as people PPing from raw do.

Seeing Nikon raw processing software mentioned it's worth pointing out that they are different. The software takes information that the camera generates other than pure sensor data from the raw file and then generates a camera profile for each and every image. They can differ considerably for each and every shot. The net effect from my limited use is a Nikon like image appears rather quickly with little colour adjustment needed. Different cameras do tend to have a different look. Their cameras can in some cases apply a tone curve to the initial raw data and another later. It seems that this can be included in the raw file as well as other data. I have made use of this aspect using a certain raw developer as it has the facility to use 2 curves in the same way Nikon do. There is at least one person on this forum that can clear this area up as far as Nikon cameras are concerned. I can't but as I have one may do at some point but I expect it only applies to certain cameras.

John
-

[Nicks Pics]

Hi Everyone,
Thanks for your time in answering my questions. Different people seem to have a different point to make, and it has been helpful to hear your thoughts.

That means it is the preference whether to read the settings or not which depends on the software used for editing.
There must be some way to ignore these settings in Capture NX as Adobe camera Raw does, just for curiosity...
I anyways use Adobe...

I don't know if this is what you were talking about here, but in Lightroom, my RAW + Jpeg files were showing up as only RAW files, so I had to set it so that it would read them as individual files.

For most images, the quality of which I would like to manually control, I am not interested in the camera doing in-camera Post Processing for me, but in some cases I might want to use some compression, (To have more frames in rapid shooting, or for other reasons). My next thought is why is a Compressed NEF file larger than a Large "Fine" jpeg if they are compressed at about the same ratio? 35-55 % (the given amount for NEF "Compressed" Files) VS 1 : 4 (the given amount for a "Fine" jpeg compression, (according to the manual)? It seems that these should turn out to be about the same, unless it is significantly affected by the 12 or 14 bit level of the NEF, vs the 8 bit Jpeg. When I compare the two, the NEF is 22.8 mb, and the Fine JPEG is only 8.65 mb. Don't worry about it if this is a lot of nonsense so you don't understand what I'm saying.

Thanks again for your help

[george013]

Is it? I never tried Capture NX software but I always thought that NEF is direct output of sensor without any processing then why monochrome?

Read my former post again.

This brings up a question you allways should ask yourself when viewing a RAW-file: what do I see. You can't see a RAW-file, only a raster-file. So if you see a raster-file instead, where does that come from? It can be an embedded JPG made with the camera-settings or it can be a result of an out-camera conversion, with or without the use of the camera-settings.

George

[xpatUSA]

I never tried Capture NX software but I always thought that NEF is direct output of sensor without any processing then why monochrome?

namaste,

I think confusion is setting in. It is correct that NEF is direct output of sensor, just as you say. It becomes much easier to understand if you buy a program called "RawDigger" where you can see this raw data on-screen either as a composite image or each of the four channels separately. Composite means that no Bayer de-mosaicing is applied. Instead the displayed channels are R, (G1+G2)/2 and B. Additionally, a simple RGB image can be displayed without any of the in-camera stuff being applied. Any of these views can be exported as TIFF files for further examination.

Well worth a look and easily found on the 'net. I bought the research version.

[ajohnw]

I have run NX but as I am on Linux can not save from it. Going on that it will use the Nikon method of converting raw full stop and that's it. No alternatives. The image at high bit depth underneath can then be manipulated as usual.

Actually it does pretty impressive conversions to such an extent it would be the only method I would use initially from raw - if I could.

John
-