Evolution of a Newbie - HDR software, technique, etc

[John T]

Stage 1: Disappointment

This summer I made a goal for my spare time to learn a bit about HDR. I have enjoyed and admired many of the HDR and tone mapped images on the web. To start the process I decided to read the following two books: 1)“A World in HDR” by Trey Ratcliff; 2)“Practical HDR: A Complete Guide to Creating High Dynamic Range Images with Your Digital SLR” by David Nightingale.

From these two books I gleaned the general standard practices of the trade and headed off in late May for my first trip of the summer to Europe. Here my wife and I visited Italy (Roma, Capri, and Pompeii). Geared with some new images from these sites I started my first post image processing steps with Photomatix Pro and HDR Tools, the two standard software packages recommended in these books as well as on other sites on the web. I used the standard methods for image capture based on Digital SLR using tripod and bracketed at 0, 2, -2 EV.

To be frank, I was disappointed with the quality of many of the processed images. See the following URL for a compilation of pictures taken on this trip. All of these images are HDR tonemapped.

http://gallery.me.com/jtmcdevitt#100261

Many of these images seemed to pale in comparison to the shots I took on the same trip that were not done with the HDR approach. Some images I took on the same trip without the HDR method are collected at the following URL:

http://gallery.me.com/jtmcdevitt#100228

In these two photo albums you will note that I have encoded information about what software was used in the file name.

From this experience I made the following observations. I share these in the hope that I can help others along their pathway of discovery and perhaps also I can glean insight from those in this space that have taken similar steps ahead of me. Please see my questions at the end of the message.

Observations from Disappointment Phase:
1) The HDR images lost much of the color punch that was in the original images.
2) The HDR images had a new source of noise, especially in the skies that was lacking in the original images.
3) Photomatix Pro and HDR Tools produced similar final images, although there were differences in each of the packages. There is no clear winner for all cases. Photomatix Pro seems to have an edge for hyper real images and HDR Tools seems to have a slight edge for photorealistic cases.


Stage 2: Enlightenment Phase

Being a stubborn Irishmen and not willing to give up easily, I embarked on a second trip to Europe this summer and this time navigated to Turkey where I attended a Global Health conference. During my spare cycles, I grabbed some new images of the coast of Turkey as well as in the Istanbul area. At this time, I introduced a couple of new variables into the mix. The first being I opened the door for the new HDR utility of Photoshop (CS5 version) and also started to use the Topaz 4 Adjust and Topaz Denoise utilities.

These two new additions immediately made a change in the quality of my images. I have collected images from this trip in the following site:

http://gallery.me.com/jtmcdevitt#100254

Here are some of my observations from CS5 and Topaz tools:
1) The HDR tool in Photoshop CS5 was more forgiving in terms of the management of the colors. I found it easier to manage the colors and create photorealistic images as the final product.
2) Topaz 4 Adjust is a very useful tool that I began to use in all post HDR image processing steps. Even if the HDR tonemapped left the image a bit dull, the Topaz could be used to pop color back into the image. This completely changed my process flow allowing me to relax the initial image processing goals at the HDR tonemapping stage.
3) After using the Topaz Adjust and the HDR tone mapping, there is often image noise introduced into the picture. This is nicely managed with Topaz Denoise, which serves as a second Photoshop plugin.
4) The Topaz Tools were so effective (no I do not have a financial interest in this company) that I began to use these same tools for popping color into single images. This has allowed me to create and manage local contrast in single images to secure HDR-like effects that I find very pleasing and quick to execute. There are some good examples of this in the image sequence from the Harem in the Topacki palace in Istanbul (i.e. colorful rooms decorated with beautiful tiles).

The CS5 version of Photoshop and Topaz utilities have produced at least in my hands a more efficient process flow for HDR and HDR-like image processing that takes the following typical sequence:

1) export images as RAW files to desktop (AEB: 2, 0, -2)
2) import images to Photoshop
3) combine to HDR 32 bit image
4) make adjustments to sliders to get a decent, but not perfect image
5) complete tone-mapping in Photoshop
6) initiate Topaz 4 Adjust
7) select preset that gets closest to desired effect
8) fine tune sliders for near final effect
9) accept parameters and return control to Photoshop
10) execute auto-color feature of Photoshop
11) use levels to adjust final contrast
12) initiate Topaz Denoise to eliminate sources of noise
13) save optimized image in final form


Perhaps these steps may be useful to others getting started in the area. For both the new persons in the HDR as well as for the more experienced persons in this area I seek your feedback on the following issues:

1) For those of you that are not “hardwired” to one HDR software preference, what have been your experiences with Photomatix Pro, FDR Tools and Photoshop CS5? Have others observed similar things to what I describe here?
2) General comments and suggestions on my process flow would be appreciated.
3) General comments and suggestions on how I might further improve the quality of the images both in the image capture and the post processing would be appreciated.

[Colin Southern]

Hi John,

As someone who does more than their fair share of HDR photography, I think that a LOT of people end up being unable to see the forrest because of all the trees; or rather they're so intent on creating an HDR image that they don't pause before hand to ask "do I need HDR to capture this scene" and if so "what techniques will work best".

Some things to keep in mind ...

- Most reflective scenes (ie scenes without backlighting) only have a dynamic range of around 4 stops, and yet our cameras typically capture around 11 or 12 stops of dynamic range. Often though people don't realise this because most monitors will only display around 6 stops of dynamic range ... so people don't realise that they still have a LOT more detail captured (assuming RAW capture here) than they realise. Nine times out of ten people don't need to be bracketing exposures and then combining (or perhaps "torturing" is a better word) in the likes of Photomatix; all they need to be doing is revealing detail using the fill light slider in Adobe Camera RAW, and possibly augmenting the exposure with a GND filter (usually only needed when shooting into the light). Done this way you don't end up with movement between exposures - and you don't have to spend time trying to correct all the "tortures" that the likes of Photomatix apply to most images (grossly over-saturated & over-sharpened, and with bad colour casts), and thus what pops out the other end is a photorealistic capture rather than the hidiously over-processed mush that - unfortunately - many are now (mis) interpreting as "the HDR" look.

So done with ACR and GND filters you end up with something like this ...



Rather than something like this (my attempt at a "parody" of Photomatix) ...

[Dave Humphries]

Hi John,

I was fortunate enough to visit the same locations as a teenager with my parents on a coach trip through Italy (only about a day in each place ), and it was a loooong time ago now, but I have the colour of the sea etched into my memory. I don't see that colour in almost any of the HDR album shots, although I suppose it is possible it has changed in 35 years. I do see it more often in the non HDR album though.

I assume you still have the RAW files, even from the first trip, so why not take one image, say one from Capri harbour, and process that by traditional ACR+CS5 methods as Colin suggests and again using your new workflow, then decide which is more accurate?

2. General comments and suggestions on my process flow would be appreciated

In all processing methods, don't overlook the basics like WB, straightening horizons, dealing with CA (chromatic aberation) and other lens defects, plus ensuring the output image does (as a rule) have a good dynamic range that spans the histogram - I see one or two of each of these problems in most of the shots, but more often in the HDR ones. The old 'trees and forest' thing, I do it myself, but not with HDR as (confessing) I don't do it.

3. General comments and suggestions on how I might further improve the quality of the images both in the image capture and the post processing would be appreciated.

In respect of at the capture stage, many of the 'indoor looking at windows' shots from Turkey seem to exhibit excessive flare, I wonder if you had a dirty filter/front element that day, or maybe it got 'enhanced' by the HDR workflow.

Hope that helps (and doesn't appear too critical),

[Colin Southern]

I assume you still have the RAW files, even from the first trip, so why not take one image, say one from Capri harbour, and process that by traditional ACR+CS5 methods as Colin suggests and again using your new workflow, then decide which is more accurate?

Also, if you'd like to send me one of the RAW captures that Dave mentioned, I'll see what I can extract out of it for you (one where the exposure stops just short of blown would be perfect).

[John T]

Dave,

I appreciate your comments too. I do not consider them to be critical. My goal here is to learn from the experts like yourself en route to improving my technique.

Your comments are very specific and thus give me something to try. I will do the experiment that you suggest here.

I agree with you on the misaligned colors from the HDR processed images. I will begin at the next step to go back to the original RAWs that I have kept and reprocess them both with ACR+CS5 as well as with Topaz. The former is new to me so this will expand my horizons. At this stage I am looking for efficient work flow so I will keep this in mind as I explore new options. I do not want to spend an hour in photoshop fixing each image. I would prefer to limit the time to 10 minutes or less, except perhaps for very special images.

On the issue of the flares at the windows, you make a good observation. I chose that look on purpose due to an artistic decision that I suppose we could each have an opinion about. In my case, I like the look of some flare as it represents what I see in many of these cases and provides some punch to the experience. I note that this effect is easily removed by increasing the strength of the HDR sliders. For the cases you specify, I took the image from a single exposure and did a couple of things. One was that I used only Topaz Adjust and Topaz Denoise and worked in a single layer in Photoshop CS5. In another similar case (file names includes layer), I did a similar thing but, added a second layer to mix back with the original image.

This brings up an interesting issue, some of these "artistic effects". I would consider to be a personal choice of the artist, but some experts may consider them to be "Rookie Mistakes". At this stage, I appreciate some input at identifying these issues so that in the future I can decide to fix them or leave them for impact.

Thanks again for your specific comments.

John

[carregwen]

John

I just looked at your CV via your profile. I am very impressed!

Your original post is very detailed and well thought-out, and you raise some interesting points.

I use Dynamic Photo HDR for my HDR processing, but I firmly believe that HDR (and that includes tone-mapping) is best reserved for specific types of subjects. I find general landscapes and people shots rarely work in HDR mode. Why? Because to me they always look as if the photographer has taken a shot, which may be technically very good and well-composed, and has tried to make it look 'different' in an attempt to take it away from being a shot that the viewer might consider 'pretty' and well executed, but at the end of the day is just a pretty well-executed shot. IOW, HDR might be used by some as a means of introducing artistic artifice without really addressing the main question - 'what really makes a good shot?'

I find shots that work best in HDR are unusual scenes where the use of HDR is not too apparent, as the nature of the scene lends itself more to the HDR treatment than a 'standard' shot might. I find that dereliction and industrial scenes work well for HDR for this reason - the textural look of HDR fits in with the subject matter. Here's an example of one of mine - I'm only posting it as an example.

Have you tried other subjects as a treatment for HDR?

[Dave Humphries]

Hi John,

I am so glad it was useful.

However, there is one more 'error' that needs correcting - I do not consider myself an expert - and niether should you, I'm just an average guy with opinions, not only that, sometimes I get things wrong, or like all of us, jump to a wrong conclusion (what flare?)

I don't have CS5 (only Elements) and your use of layers already exceeds my experience - the great thing about CiC is we all learn off each other, as we take this journey into improving our photography together.

The mis-aligned colours I saw was more likely CA than HDR induced, as it was on a white post against a dark background and showed red one side of the post and green the other. Also, I think it was in the non-HDR album; in fact it is this one, check out the left hand side.

Also, I don't think I said welcome either, which was very remiss of me

Welcome to the CiC forums from ...

[John T]

Colin, Dave,

Thanks for your suggestions.

I have taken one of the images from the harbor region on the North side of Capri and I have processed this using ACR utility of Photoshop CS5. After making some basic adjustments here with this RAW file import interface, I then went the next step and used the GND filter. The end result is here posted:



The original image without these steps is posted below:





The new image I also included a change in the white balance and this led to warmer hints in the mountains and trees that I think looks more like the original scene.

Note that I also tried my new flow (see above section, this post) including the Topaz Adjust step and found the ACR and GND combination in CS5 to be preferable, at least in this case. However, when all three elements are combined there is additional interest generated in the clouds (now shown).

Thanks again for the good suggestion.

John

[John T]

Rob,

Thanks for your kind message about the CV. In my real job, I use imaging chips derived from the digital photography industry to develop affordable healthcare devices. My role here at the CIC is to learn from others to improve upon my hobby.

I checked out your web site and found high quality images there. Thanks for the suggestion about choice of subjects.

I like the industrial look in HDR. I agree this makes good subjects. Also, ancient cities seem to do well. Let me know what you think about the following subjects for HDR.

First, here is an image of the Galata tower and Galata bridge in Istanbul.



Second, I have posted here an image of the ancient tombs at Myra, Turkey:



Third, here is an image of the Myra tombs from a different perspective. Here I wanted to push the limits a bit and make sure I had a large dynamic range to cover. After the HDR and tone-mapping steps, the image was still blown out with little color. I popped the colors with Topaz Adjust.




Feedback on these images is welcomed.

[carregwen]

Rob,
Thanks for your kind message about the CV. In my real job, I use imaging chips derived from the digital photography industry to develop affordable healthcare devices. My role here at the CIC is to learn from others to improve upon my hobby....I like the industrial look in HDR. I agree this makes good subjects. Also, ancient cities seem to do well. Let me know what you think about the following subjects for HDR.

Yes, I agree that old buildings also work well, and for the same reason - the distressed look of older building lends itself to HDR. I prefer the shot of the Galata tower.

[John T]

Colin and Dave,

I wanted to send a second message to explore with a bit more depth your suggestions.

I have reprocessed images from my Capri trip as per your suggestion, this time with focus on the low dynamic range cases. My goal here was to try to capture the beautiful colors of the Capri sea.

I have used the ACR and GND combination in CS5 as we discussed before. The new images are now posted at the following URL:

https://www.me.com/gallery/#100269

I think we are making progress.

Thanks again for the great suggestion and the encouragement.

John

[Colin Southern]

Colin and Dave,

I wanted to send a second message to explore with a bit more depth your suggestions.

I have reprocessed images from my Capri trip as per your suggestion, this time with focus on the low dynamic range cases. My goal here was to try to capture the beautiful colors of the Capri sea.

I have used the ACR and GND combination in CS5 as we discussed before. The new images are now posted at the following URL:

https://www.me.com/gallery/#100269

I think we are making progress.

Thanks again for the great suggestion and the encouragement.

John

Hi John,

Unfortunately, it appears that one needs a login with MobileMe to be able to view your image (and I don't have one).

Did you know that you can host images here for use in the forums? (just click on "Community" and then "My Albums", and follow your nose from there).

PS: By the way, with regards to GND filters, I was meaning the physical ones that I use at the time of capture, not digital ones in Photoshop -- wasn't sure if that was clear or not.

If you're interested, Singh-Ray make the best GND (and many other kinds of) filters -- also, I've written a few articles for their Blog (some regarding GND filter use), which you may (or may not!) find interesting.

[John T]

Colin,

Thanks for the followup.

I will explore the physical GND and read with interest your article on the same subject.

In the mean time, I did make changes with photos I already collected using the virtual tool in Photoshop. Here I Googled GND and Photoshop and found an excellent tutorial. See below link:

http://pshero.com/photoshop-tutorial...density-filter

I use this digital tool with nice results and the process takes only a couple of minutes. The same utility can be applied a couple of time from different directions and can be set at various levels of intensity so the effect can be optimized. This may not be exactly what you intended, but the suggestion did open a door for me. Thanks.

Regarding the photo access, I have just changed the settings on my online albums. You should now have access to view and download the original RAW images.

Check out the following link first:

https://www.me.com/gallery/#100276

This folder has the RAW images from Turkey (and London). These are collected most typically as 0, -2, +2 EV. Cannon Raw, tripod mounted, ASA 100, and AV11.

If you have any suggestions about how to better process these images with or without HDR I would appreciate it.

Thanks again, John

[Colin Southern]

Colin,

Thanks for the followup.

I will explore the physical GND and read with interest your article on the same subject.

Hi John,

No worries

There's two basic approaches to "GND" in Photoshop:

- The first one is along the lines of the article you referenced, which by the way is "handy", but it's also obsolete, since Adobe have now introduced the same functionality into ACR in the form of both the adjustment brush and Graduated Filter tools. The big issue with this approach is that it doesn't compress the dynamic range of the scene at the time of capture with a real GND filter does. As I mentioned above (and I'm sure you already new) modern sensors can capture in the region of 11 to 12 stops of DR anyway, so revealing shadow detail ususally isn't a big issue - but - if it is a very high DR scene then some of the shadow detail may be quite noisy - so using a typical 3-Stop GND filter essentially buys you 3 stops less noise.

- The second one is a variation on the theme where you simply stack 2 images one above the other in Photoshop (same scene, different exposures) and paint a layer mask to transition between the two (and thus revealing the idea exposure portions of each image).

I'll have a look at your RAW files a bit later in the day ... time to take #2 daughter off to school, and head off to work for me

PS: If you haven't got a copy already, you REALLY need to purchase this.

PPS: Still no luck accessing your mobileme files - same security challenge as before. Might be easiest to just use www.sendthisfile.com (free account) (send it to yourself and then copy/paste the download link to me here in a PM.)

[arith]

I like the third John T photo because it has potential. You could clone out artifacts but more I was thinking maybe taking the sky separately or using a GND filter, and maybe a lens with a round diaphragm. Since there isn't going to be much in the way of fine detail, hows about a very small aperture.

Well an interesting trick I haven't had a chance to try yet is to take two scenes of only a few degrees rotation apart, then use photoshop scene cleaner to remove flare. Like I said I haven't done it except in an image with too many problems to show, however it did remove the flare.

[Colin Southern]

Hi John,

I've had a crack at processing your RAW files for you - for the first two I just used the darkest exposure, and for the last I (think) I used the middle exposure. I'll talk to you more about these, but first of all, I'm interested to hear what you think of what I've done.

PS: I've kept them all quite large, you you might light to click on them to display them at 100%

[John T]

Colin,

Wow. What a difference this approach has made.

I really like the Myra image the way you have processed it. I did not think the single exposure had enough details in both the shadows and the blown out sky to create this image.

Now that I know the general pathway, I will attempt to create such an image myself.

More to follow soon. John

[Colin Southern]

Hi John,

The Myra image in particular was "interesting" - to be honest, it was more difficult than I thought; normally fill light alone pretty much does most of the job, but in this case it revealed a bit of noise, but also what I believe was some subtle but relatively widespead veiling flare (in addition to the other flare which is also present) - which made sharpening VERY touchy, and I ended up doing a fair bit of hand dodging & burning.

Probably the best advice I can give with any of these is to work on the shooting technique first -- shooting into full and direct sunlight is going to present all sorts of image issues due to the extreme (under-statement) contrast. I'd suggest avoiding it like the plague on all but very wide angle shots where it takes up only a small portion of the shot.

The other thing I'd be doing is sticking to a lower ISO (where you'll have more dynamic range available in the capture) - stopping down more to get more into the sweet spot of the lens - and DEFINATELY shoot from a tripod with a remote release (or at the very least, use the built-in timer).

Not sure what lens you've used, but I have seen sharper - but that may have had a lot to do with the veiling flare and aperture. From what I've seen so far though, no apparent need for any HDR techniques or special programs - everything I did in these shots was just using CS5 (and I didn't spend a whole lot of time on them).

Hope this helps

[Boondocksaint]

A lot of fantastic info in this thread. I have bookmarked it for future reference.

[Shadowman]

Pushed a few limits here. I was supposed to shoot with high ISO but forgot to reset.
Nikon D60
f/4.8 aperture
3sec
ISO 100
75mm through window
Processed with FDRTools for HDR then post processed with PSE8.