The RAW/JPEG Discussion

[darknight]

I only shoot in raw, the main reason i suppose is that you do get that bit extra ability for pp on your photo, and because i have large capacity xcd(was the that the right letters) cards, also the magazines that we regulary upload to only exept raw files.
Also i dont see the point in jpeg apart from the extra you get on a card, as when using, lightroom,photoshop, etc etc you can save a file easily to jpeg for general use.

[Brocken]

If you are like me, and think of the camera's output as consisting of picture elements, pieces of geometry, amorphous shapes, patches of colour, tones, textures, etc which we can assemble or re-assemble or subject to metamorphosis in an interesting way, then there is no question that RAW is the easier to use as a first material in digital image synthesis. The posterization which develops when you start pushing and pulling a jpeg image can be an interesting effect in its own right, but if you only have jpegs as raw material, you'll be stuck with posterization whether you want it or not.

[Colin Southern]

I don't want to see one method as an insurance policy and in that sense as better than the other method. That sounds like a fallacy to me. You need insurance when something goes really wrong, but (again) assuming you have set your camera correctly that should not happen.

Hi Peter,

Let me give you a real-world example from the studio about 3 weeks ago.

Say hello to the lovely Louise; I shot some 300+ frames of Louise -- all with correct exposures as measured with my NZD $1000+ light meter. In post production in most cases I've not needed to make any exposure adjustment -- or in other words "my camera was set correctly for the level of lighting in the studio".

I discovered afterwards that Louise's hair looks much better if I use quite a lot of fill light control to compress the dynamic range of the information captured and thus revealing shadow detail. I shot this with my new Canon 1Dx which has phenomenal dynamic range and thus a LOT of very clean shadow detail that can be revealed if required.

I've taken 2 copies of an original frame - converted the first to a JPEG without any adjustment - and then attempted to reveal shadow detail around the models left ear (ie to camera right). This is what the file would have looked like if I'd shot JPEG and then tried to recover the shot. You'll see that the information just isn't there.



In contrast, this is the information I was able to reveal using the RAW file ...



Two shots - both exposed correctly - no contest as to which one can be used to produce the money shot. The JPEG has simply discarded too much information that was subsequently needed.

Note: Yes, I appreciate that one wouldn't normally want to reveal THIS much information in the shadow areas, but (a) what IS needed isn't far short of what's here and (b) even revealing a lesser degree of shadow detail shows unacceptable noise on the full-resolution original.

Just wanted to make the point that JPEGs can well discard needed information EVEN IN A CORRECTLY EXPOSED SHOT.

[Letrow]

That is interesting Colin.
It does not really surprise me though, as RAW has the possibility to adjust exposure up and down and JPEG does not.

I can spot the difference between the two photos, the second one has clearly recovered more shadow detail. I would argue that for a better comparison the second photo should not be lighter in general, but I do get the point that you are making.

If I would see the two photos separately I don't know what I would think, I'd probably like them both and not think about the difference. If I compare them now the saturation in the first photo looks more pleasing to me, but that can be due to my monitor or personal taste. That dark background looks great against the red and I would probably use that and recover some shadow detail in the hair selectively with the RAW shot.

I can understand that you, if you plan to sell this photo or use it professionally, would probably need that extra flexibility. So, you should use RAW. I don't think there is any doubt about that. My goal is not to get you to shoot JPEG, let that be clear. In fact I don't really care.

So, let's just agree on the fact that you can use RAW for what you do and that I can use JPEG for what I do. And I'll experiment with RAW now and then, to see what difference it makes for me.

Now coming back to the insurance and why I think your argument is a fallacy:
First of all: there is always a cost to insuring yourself.
- If you are pro insurance, you will pay, which is fine.
- If you don't need insurance, it saves you something, but there is a risk associated with that. If the risk is acceptable to you (can I afford to pay for something new or can I reshoot the photo/afford to throw in the recycle bin?), it should be fine as well.
In a real insurance the cost would be money, and here the cost would be time and space (it does not really matter whether that cost is inconsequential to you, as it might be consequential to someone else).

According to you RAW is free insurance, with no downside, which to me seems like an argument that does not belong in an otherwise interesting discussion. Apart from that I agree with what you say.

[bambleweeney]

I too find the insurance argument fallacious. I use RAW because I have a lot more information in the image (as Colin's images nicely demonstrate) It's not so much the fact that you can recover information from a RAW file (which is sometimes useful if your exposure is not quite as expected), it's the fact that the information is actually there in the first place to be recovered. My relatively expensive camera is capable of capturing a certain dynamic range and a certain range of tonalities and to me personally it makes sense not to cripple this range by shooting JPEG, I mean, jeez if I was going to do that I may as well have bought a Canon

[DanK]

it's the fact that the information is actually there in the first place to be recovered. My relatively expensive camera is capable of capturing a certain dynamic range and a certain range of tonalities and to me personally it makes sense not to cripple this range by shooting JPEG

My view exactly, except that my camera is not relatively expensive. I just can't see the value in throwing away information. It's not just dynamic range and proper exposure that is at issue. Any camera-produced jpeg uses a fixed algorithm for all sorts of parameters--saturation, color balance, white balance, contrast, sharpening--that might or might not turn out right. I shot raw+jpeg when I first started shooting raw, but I realized within weeks that I was just deleting all the jpegs, so I started shooting only raw. If the shots in a batch need similar adjustments, it is trivial and extremely fast to sync the editing settings. If they need different adjustments, that takes time, but that is precisely when raw is most valuable.

But, to each her or his own.

[Bryan Conner]

When you shoot jpegs, you are agreeing to be completely satisfied with the processing done by the camera's raw processor. You are agreeing to be satisfied with the opinion and taste of the person who created the in camera raw processor...or you will be satisfied with the modifications you have made to that person's taste in color, sharpness, etc via your picture style presets etc. Sure, you can make a few small changes in post to a jpeg, but you can not do a lot.

When you shoot raw, you can have all of the above if you wish. You can have exactly what the jpeg would have been. PLUS, you can have more. You can apply much larger changes to a raw image if needed, or if wanted, much more than you can to a jpeg.

Raw processing software has had major improvements in the last year or so. Adobe's Process Version 2012 in Lightroom 4 and ACR 7 is a great example. I can revisit raw files that I took 10 years ago and reprocess them into much improved images. Lower noise, better highlight control etc. There is not much I can do to a jpeg even 10 seconds later, much less 10 years later.

I can not see a reason to shoot only jpegs....ok, one: If I do not have any space left on my memory cards and I just have to capture the shot. That is the only reason. But, I probably will never shoot 20,000 plus raws at one time without being able to import them to my hard drive and format at least one card.

[ajohnw]

Really those 2 shots may have given a better idea of the merits of each if it was a camera generated jpg. The post and the shot suggest it isn't. The unusual thing about the jpg is that the dark area has completely zeroed pixels. The only time I have seen that is when the black level is set too high in raw development software so that it's clipped rather than compressed as it usually will be from a camera jpg. If it's compressed it can usually be recovered adequately even from a jpg but some wont even need that. In this case the skin tones are darker in the jpg as well which wont help.

I still don't see this as an argument for not shooting both and using jpg's when they are ok. I suspect given the camera and canon that the camera would have given a decent jpg right out of the box. Given the complexities in camera firmware it would probably even convert on the basis that there are skin tones and dark areas in the shot.

-

[SteveF]

I notice in Colins comparisons the distinct colour difference in the model's clothing and I've noticed this quite a lot when folks have posted raw/jpeg comparisons. Whenever I've tried to push or pull raw files in experiments in the past I've had this effect too.

It does depend on which cameras we're using as to how good the jpeg is from it and how good the raw converter is in matching the shooting parameters from the camera in the first place. Quite often I've found raw files to appear brighter on-screen than the jpeg so have turned down the exposure slightly to find when processed to a jpeg it's too dark.

In high noise situations most cameras I've got do a better job at processing and restraining the noise in the jpeg than Canon's DPP does. When we could use Canon Zoombrowser it did a better job at noise control than DPP currently does now. The only time I found otherwise is when shooting in areas with high EMF emissions such as in the close vicinity of public address systems and other large sound systems when the in-camera jpeg was affected.

Quite a number of the latest compacts do some quite remarkable jpeg processing now and of course all things being equal it should do a better job being designed by the manufacturer than we can. My G1X can perform multi-area white balance in full auto for instance - do the raw programs offer this yet?

In photoshop CS2 I have a wonderful range of controls to alter jpegs but I have to buy CS6 to process my raws from my 1DX? There's noise ninja too we can buy but it's extra expense just for that feature.

The histogram is reported by many on the camera to be one from the onboard jpeg embedded in the raw files, so in that case we're not actually seeing a histogram purely for the raw file so how do we know that's how we want the raw file to be produced? A raw file still has to have a fairly 'correct' exposure surely to produce top class results. I have found, as I started writing at the beginning of this reply, colour balance tends to move around when pushing or pulling raw files.

For me it's not about file space nor learning the raw processing; it's about wether I want to or not. I could buy more space easily and I could go out and buy CS6 and an Mac to process these on. However that's not what I want to do and I would imagine that's the position of a lot who shoot jpegs. The specs Colin S. gives for his processing power is staggering for me with a PC and 2g of RAM! No, when I bought the camera I did not expect to have to go out and buy loads of extra stuff.

You'll see I haven't lectured on about getting it right in camera as I find that to a be slightly arrogant way of approaching the discussion. Sure we should try and perfect our tecniques but sometimes it's not always in our control to do so.

[ajohnw]

I find that on a Pen that the camera jpg makes a much easier job at it's max iso of 1600. In fact dpreview pointed out that it's the best they have seen. Sarcastically it does seem to need to be on Panasonic sensors. Given a scene with no extreme dark areas I usually select iAuto in which defaults to 1600iso in my case when I need it and let the camera get on with it.

I don't think people realise just how much effort camera manufacturers usually put into generating decent jpg's straight from the camera. To do that as the internal colour space and the pc screen / jpg space are different all sorts of strange things go on whether it's done long hand or by the camera. Personally I believe in letting the camera do as much as it can especially on white balance. Some say cloudy so set this etc but I find that the camera Pen, 2 canons and a number of compacts over the years do a better job of that aspect and save work later. Typically a jpg just needs curves to uncompress blacks a little and of course sharpening when the size of the actual shot is reduced.

Not an area I have looked at and may have wrong but on either or both of the e-pl1 and e-p3 raw files can be edited in the camera. One or both of them have camera curve controls. The latter may just apply to jpg's not sure. Quote

In-camera Editing, RAW Data Edit
Newer Olympus cameras support editing certain aspects of RAW files right in the camera. This feature is typically found in the camera’s “Edit” menu. Raw editing options include image quality, White Balance, Sharpness, Contrast, Saturation and Art Filters. Please check your camera manual for available RAW Edit functions.

I'm too busy on other aspects to check this out yet but seem to remember variable high and low light compression which is what raw development is manly about.

If some one wants to work on raw files without spending cash there are several programs about. Rawtherapee for one. It's fairly light weight as far as pc needs are concerned. Even has it's own forum and there are a number of tutorials about. It will take care of chromatic aberration and fringing and lots of other things and has camera icc profiles available. Really there is no need to buy adobe products and free software doesn't mean bad software but it's wise to stick with open source. Other freebies usually for windows often contain things a user doesn't really want. For straight raw conversion and just camera curve modification plus noise reduction Ufraw is another one. Often used with the Gimp. Many people may need the auto exposure button to get close. A result of a development process designed to get the most out of all shots what ever is in them. Read professionals. It will often result in dark shots that need the exposure level changing. Playing with the 2 camera curves it offers is none destructive and probably the best way to learn. Again there are lots if tutorials on the web including using it along side the Gimp. The Gimp can be difficult to get into just like CSx. There are a couple of others but not sure how they are on windows DigiKam and ShowPhoto. More on http://www.digikam.org/drupal/about/overview . These 2 are along on the lines of Lightroom. I've only recently played around to see what they will do. The auto levels is often impressive with default settings. The other packages can be easily found on the web.

PC upgrades. There have been versions of windows on machines with 4gig of ram even though the software can't make use of more than 2. That has changed and an upgrade from 2 to 4 gig can make a big difference even in normal use. Tends to be a cheap option these days as well.

Rawtherapee etc will also work on jpg's and has the basic techniques that are usually needed built in.

-

[Colin Southern]

OOps!

My apologies - in making the JPEG version above, I didn't notice that my camera RAW defaults still had the black clipping point set to 5 (set as a personal preference), so I've redone the image with it set to zero below. As you can see, more shadow detail is revealed, but (and this is far more obvious when looking at 100% crops), it's extremely low-quality detail. Absolutely no comparison with the RAW equivalent.

[Colin Southern]

It does not really surprise me though, as RAW has the possibility to adjust exposure up and down and JPEG does not.

JPEG does offer the ability to alter exposure - due to the "safety margin" built in, but not to anywhere near the same degree.

I can spot the difference between the two photos, the second one has clearly recovered more shadow detail. I would argue that for a better comparison the second photo should not be lighter in general

There is no difference between the exposure of the two photos.

If I would see the two photos separately I don't know what I would think, I'd probably like them both and not think about the difference. If I compare them now the saturation in the first photo looks more pleasing to me, but that can be due to my monitor or personal taste. That dark background looks great against the red and I would probably use that and recover some shadow detail in the hair selectively with the RAW shot.

The processed shot doesn't have detail revealed to this level (neither the hair or the background) -- I just used it as an example of how a JPEG capture discards information that may be needed due to reasons other than incorrect exposure.

My goal is not to get you to shoot JPEG, let that be clear. In fact I don't really care.

And mine isn't to try to get you to shoot RAW -- I'm just trying to demonstrate a scenario where information may be required that's not adequately contained in a JPEG. Or put another way - if I'd conducted that shoot shooting only JPEG - with all images correctly exposed - I'd have had vastly inferior images.

So, let's just agree on the fact that you can use RAW for what you do and that I can use JPEG for what I do. And I'll experiment with RAW now and then, to see what difference it makes for me.

For sure - as I say, I'm not trying to convince anyone to change - I'm just trying to educate them as to how shooting JPEG can bite them in the bum if they subsequently need to recover shadow detail like this.

Now coming back to the insurance and why I think your argument is a fallacy:
First of all: there is always a cost to insuring yourself.
- If you are pro insurance, you will pay, which is fine.
- If you don't need insurance, it saves you something, but there is a risk associated with that. If the risk is acceptable to you (can I afford to pay for something new or can I reshoot the photo/afford to throw in the recycle bin?), it should be fine as well.
In a real insurance the cost would be money, and here the cost would be time and space (it does not really matter whether that cost is inconsequential to you, as it might be consequential to someone else).

According to you RAW is free insurance, with no downside, which to me seems like an argument that does not belong in an otherwise interesting discussion. Apart from that I agree with what you say.

I'm afraid that I'm not following your logic here. The only "downside" to shooting RAW that I know of is an extra 1 or 2 mouse clicks to convert the images to the same standard as they would have popped out of the camera. The problem with having / not having insurance is that you don't know when you're going to need it; I'm not planning on having a car accident today, but I'm still going to wear my seatbelt as insurance. I wouldn't have thought that I'd need the insurance of RAW captures in a studio shoot (because with my 1Ds3 camera revealing that kind of detail would have been too noisy even in a RAW shot), but with the 1Dx I'm finding that the detail is much cleaner and can now be used -- getting better photos -- detail that wouldn't have been possible from a JPEG capture. Very relevant to the discussion IMO.

[Colin Southern]

It's not so much the fact that you can recover information from a RAW file (which is sometimes useful if your exposure is not quite as expected),

My point is that one can need to reveal detail from areas that a JPEG discards EVEN THOUGH THE EXPOSURE WAS SPOT ON.

[Colin Southern]

Really those 2 shots may have given a better idea of the merits of each if it was a camera generated jpg. The post and the shot suggest it isn't. The unusual thing about the jpg is that the dark area has completely zeroed pixels. The only time I have seen that is when the black level is set too high in raw development software so that it's clipped rather than compressed as it usually will be from a camera jpg. If it's compressed it can usually be recovered adequately even from a jpg but some wont even need that. In this case the skin tones are darker in the jpg as well which wont help.

Please see my updated JPEG image above.

Traditionally I use a black clipping point of 5 in RAW processing because any detail below that was too noisy to be of any use -- but I probably need to revisit that now I've replaced the 1Ds3 with the 1Dx which has approx 5 stops better DR.

Yes, it would have been nice to compare with an actual in-camera JPEG, but (a) I don't have one, and (b) that would in turn be influenced by any picture styles set in the camera at the time, whereas a conversion with all controls set to zero should be the next best thing.

The differences in skin tones are due only to the difference effect of the fill light control as far as I know.

I still don't see this as an argument for not shooting both and using jpg's when they are ok. I suspect given the camera and canon that the camera would have given a decent jpg right out of the box. Given the complexities in camera firmware it would probably even convert on the basis that there are skin tones and dark areas in the shot.

I'm not trying to argue that folks shouldn't shoot JPEG - only trying to demonstrate that JPEG discards information that might be needed later despite a correct exposure. And no, the camera won't change the processing on the basis of skin tones (that would be a consistency nightmare) - and any differences in "dark areas" will only affect the initial choice of exposure (assuming an automatic exposure mode, which isn't applicable in a studio anyway), unless additional modes are selected in-camera.

[Colin Southern]

The histogram is reported by many on the camera to be one from the onboard jpeg embedded in the raw files, so in that case we're not actually seeing a histogram purely for the raw file

Hi Steve,

A histogram for a RAW file is meaningless to humans because it's based on a linear capture and thus doesn't have gamma correction applied to simulate the response of the human eye. It doesn't look anything like a typical gamma-corrected histogram that we're used to seeing.

[pnodrog]

Now coming back to the insurance and why I think your argument is a fallacy:
First of all: there is always a cost to insuring yourself.
- If you are pro insurance, you will pay, which is fine.
- If you don't need insurance, it saves you something, but there is a risk associated with that. If the risk is acceptable to you (can I afford to pay for something new or can I reshoot the photo/afford to throw in the recycle bin?), it should be fine as well.
In a real insurance the cost would be money, and here the cost would be time and space (it does not really matter whether that cost is inconsequential to you, as it might be consequential to someone else).

According to you RAW is free insurance, with no downside, which to me seems like an argument that does not belong in an otherwise interesting discussion. Apart from that I agree with what you say.

Peter the main benefit is not for something you can reshoot. It is for opportunities that require you to react instantly with your camera almost regardless of the settings. It seldom happens but when it does the success of grabbing a usable photograph may depend on the ability of the raw file. As camera sensors move from 12-14 bits to 16-18 bits the difference in the ability to recover from incorrect exposure will become even more extreme. I thought JPEGs where restricted to RGB values of 8 bits per colour but a report I read stated JPEG files could manage up to an effective 11 bits of dynamic range so they are a little better than I gave them credit for. BUT unless the JPEG standard is upgraded it will fall further behind sensor technology in the next few years.

Compared to the overall costs in pursuing our interest in photography (including transport) the additional investment in memory is totally insignificant. To say the insurance (exposure latitude and colour correction) does not belong in an otherwise interesting discussion is baffling to me.

[SteveF]

Hi Steve,

A histogram for a RAW file is meaningless to humans because it's based on a linear capture and thus doesn't have gamma correction applied to simulate the response of the human eye. It doesn't look anything like a typical gamma-corrected histogram that we're used to seeing.

Hi Colin,

Thanks for that. Quite honestly I find these discussions entertaining, humerous and informative. We should never give up on the fact that whilst we might defend our chosen format, there's always a different way which, one day, might benefit us. Jpegs still rule, of course, as we know raw users still can't get it right 'in-camera' ...

[Colin Southern]

As camera sensors move from 12-14 bits to 16-18 bits the difference in the ability to recover from incorrect exposure will become even more extreme.

Hi Paul,

I think you might be confusing Dynamic Range with A/D resolution. Dynamic range is "the height of the staircase", A/D resolution is "the number of steps between the bottom and the top of it".

[Colin Southern]

Hi Colin,

Thanks for that. Quite honestly I find these discussions entertaining, humerous and informative. We should never give up on the fact that whilst we might defend our chosen format, there's always a different way which, one day, might benefit us. Jpegs still rule, of course, as we know raw users still can't get it right 'in-camera' ...

Hi Steve,

To be honest, I really don't mind what folks use - what I do like to do though is "counter the myths" when I come across them -- like the myth that it "takes ages to convert RAWs to JPEG" when it's as easy as "select all" and "drop onto export preset" in Bridge. Personally, I find JPGs to be like frozen pre-cooked meals: Easy, but they never taste as good as ones I make myself!

[Mark von Kanel]

I just dont get the discussion... if i shoot jpeg, it should be right 99% of the time, but if i want to do any pp on the image i may well be hamstrung....

if i shoot raw it will be right 100% of the time with no hamstringing to the pp and if i want a jpeg i just push a few buttons on the PC and away i go

if i shoot raw and jpeg then ive got even more images taking up more space on my HDD and it slows mycam down due to buffering....

what was the argument for bothering with jpegs again....