Night Photography Reducing Noise

[xpatUSA]

Ted, re your image: I suspect you got that effect because of the specifics of the camera. While I don't do many exposures as long as this example, I do have exposures frequently that are in the range of your example when I do urban night photography, and I generally get no appreciable noise and almost never use any noise reduction.

Yes, the Foveon does get hot and it was about 90F when I did that test - A series of shots at 800 ISO with little time to cool down between shots. And ordinary cameras will always be better.

On that same day, I took a shot with that camera at 1600 ISO and about 1/60 sec IIRC and the image was reasonably good - so time was a major factor in the opposite direction to your suggestion.

Noise due to temperature does exist and all sensors do get warm.

https://www.sciencedirect.com/topics.../thermal-noise
.

[DanK]

However, noise due to temperature does exist and all sensors do get warm.

yes, but that isn't quite the question in this thread. There are two questions here:

1. is noise from temperature when taking long exposures with modern cameras appreciable? and

2. Would breaking a long exposure into shorter ones with a lower EV, with very short breaks in between them and then summing them, be appreciably better?

3. Would breaking a long exposure into shorter ones with the same EV, again with very short breaks in between them, and then averaging them be appreciably better?

My experience is that with my current equipment, the answer to #1 is no with exposures up to 10 minutes, assuming that one doesn't expose to the left. I haven't tried exposures longer than that with this camera (Canon 5D Mark III).

I don't know the answers to #2 and #3. I'm skeptical, but it's an empirical question. I have a hard time imagining that the very short breaks would have much effect on temperature by the end of the series, and if that's right, the key would be the impact of averaging or summing across images. To the extent that the noise is random, it should be an improvement, but I have doubts that it would be appreciable. If I'm wrong, I may start using multiple exposures.

[xpatUSA]

yes, but that isn't quite the question in this thread. There are two questions here:

1. is noise from temperature when taking long exposures with modern cameras appreciable? and

2. Would breaking a long exposure into shorter ones with a lower EV, with very short breaks in between them and then summing them, be appreciably better?

3. Would breaking a long exposure into shorter ones with the same EV, again with very short breaks in between them, and then averaging them be appreciably better?

My experience is that with my current equipment, the answer to #1 is no with exposures up to 10 minutes, assuming that one doesn't expose to the left. I haven't tried exposures longer than that with this camera (Canon 5D Mark III).

I don't know the answers to #2 and #3. I'm skeptical, but it's an empirical question. I have a hard time imagining that the very short breaks would have much effect on temperature by the end of the series, and if that's right, the key would be the impact of averaging or summing across images. To the extent that the noise is random, it should be an improvement, but I have doubts that it would be appreciable. If I'm wrong, I may start using multiple exposures.

I'm confused. I was responding to your "simply lengthening exposure as a way of avoiding random noise" - which I took to mean that a longer exposure avoids random noise, implying that a short exposure does not.

Perhaps your understanding of the meaning "random noise" is different than mine. May I ask how lengthening an exposure avoids random noise?

[DanK]

I'm confused. I was responding to your "simply lengthening exposure as a way of avoiding random noise" - which I took to mean that a longer exposure avoids random noise, implying that a short exposure does not.

Perhaps your understanding of the meaning "random noise" is different than mine. May I ask how lengthening an exposure avoids random noise?

No, I was just sloppy in my language. I meant lengthening exposure to avoid underexposure and hence noise, since one of the issues was summing underexposed images, or images with a lot of deep shadow.

the more I think about this, the less clear it becomes to me. Clearly, shooting 5 identical images and averaging them will reduce random noise, whatever its cause. That's just math. the more interesting case is when one exposes to the left and averages images, rather than exposing longer and darkening in post, or exposing to the left and summing. At this point, all that is clear to me is that as a practical matter, long exposures with the equipment I have, if exposed well, aren't noisy.

[xpatUSA]

No, I was just sloppy in my language. I meant lengthening exposure to avoid underexposure and hence noise, since one of the issues was summing underexposed images, or images with a lot of deep shadow.

Thanks, now I get it. Certainly please disregard my posted examples - the exposure was the same in each, i.e. as the shutter time lengthened, so did the f-number thereby keeping N^2/t constant.

[Manfred M]

Dan - the underlying physics of what causes sensor noise has not changed. What the modern technology has done is to mitigate some of those weaknesses.

When I compare the output of my images taken with the D800 versus the D810, the older camera, which had best of class performance for that generation of cameras is bested by the newer generation. When I compare my images taken with the D810 (which was again a very clean camera), it is not as clean as the D850 that superseded it. Grahame is shooting the D800, if I recall correctly.

I recall that Nikon made an even better camera for astrophotography; the D810A.

https://www.nikonusa.com/en/Nikon-Pr...ras/D810A.html

[Stagecoach]

A recap and update .............................

Firstly, with regard to random noise caused by sensor heat/time operational I'm ignoring completely due to two reasons, I have never encountered this on either of my cameras D300/800 (both old tech) with exposures as long as 5 minutes. In addition, with all the tests I'm doing the multiple exposures being produced to provide stacks for summing in different ways are increasing sensor time in operation 'coming close to' what would be a single long exposure at base ISO for which it's a given is going to give us minimum noise without any manipulation.

As for my testing, which to recap is to investigate a 'claim' that stacking in camera using the exposure 'additive' method had an advantage I'm still investigating this to see if I can find what advantage, if any at all, it has compared to other options.

But at present I'm concentrating on why I can produce a good 'additive' exposure stack in camera using 'multi exposure' but can not replicate this in PS yet. Whereas, I can replicate the same results of in camera exposure 'mean' stacking in PS as well.

Here's examples of what I've found and searching for clues has failed so far.

No 1


No 2


There's an answer somewhere

[DanK]

I don't know the answer to your question, but I will be interested if you can sort it out.

As for this:

As for my testing, which to recap is to investigate a 'claim' that stacking in camera using the exposure 'additive' method had an advantage I'm still investigating this to see if I can find what advantage, if any at all, it has compared to other options.

Just thinking about the math, this seems like a complicated question. Summing, after all, is like just averaging without the division. So it would seem that the additive method has one advantage over a single exposure: to the extent that the noise is truly random, summing should attenuate it, although it wouldn't eliminate it. On the other hand, the base for the additive method, if I understand right, is lower exposures, and that should decrease the S/N ratio.

[Stagecoach]

I don't know the answer to your question, but I will be interested if you can sort it out.

I'm still at a total loss investigating this one Dan. I can not believe that a simple process that can be undertaken on my two older technology Nikons can not be replicated exactly in PS.

That process being, in camera multi-exposure mode, 'adding' together 6 exposures taken at 10s, ISO 200, f/7.1 and producing an identical result as you would have got with 1 exposure taken at 60s, ISO200, f7.1 or 8s, ISO1600, f/7.1. (disregarding noise of course)

As for this:

Just thinking about the math, this seems like a complicated question. Summing, after all, is like just averaging without the division. So it would seem that the additive method has one advantage over a single exposure: to the extent that the noise is truly random, summing should attenuate it, although it wouldn't eliminate it. On the other hand, the base for the additive method, if I understand right, is lower exposures, and that should decrease the S/N ratio.

Until such time as I can undertake the 'adding' process manually in PS so it's equivalent to the in-camera results obtained I can't progress any comparisons.

[xpatUSA]

I'm still at a total loss investigating this one Dan. I can not believe that a simple process that can be undertaken on my two older technology Nikons can not be replicated exactly in PS.
Until such time as I can undertake the 'adding' process manually in PS so it's equivalent to the in-camera results obtained I can't progress any comparisons.

Pardon my naïvety, Grahame. Would that not be like blending 6 layers in Additive mode, assuming that PS can do that, like the GIMP?

[Stagecoach]

Pardon my naïvety, Grahame. Would that not be like blending 6 layers in Additive mode, assuming that PS can do that, like the GIMP?

Yes you would expect it to be Ted.

But as I have explained and demonstrated in post 27 the results I was easily able to achieve using the PS Stack 'Summation' (additive) mode were not 'clean' results as achieved in camera using the 'additive' mode with multi exposure.

It may be that there is an alternative method that can be used in PS but considerable searching has not found one.

Have you trailed the method in Gimp using Raw files and can you confirm the results are as clean as a single shot? By clean I mean no mush such as odd patches of a strange colour, as opposed to simply noise.

But, I think I'm on to something and 'possibly' making progress

[xpatUSA]

Yes you would expect it to be Ted.

But as I have explained and demonstrated in post 27 the results I was easily able to achieve using the PS Stack 'Summation' (additive) mode were not 'clean' results as achieved in camera using the 'additive' mode with multi exposure.

Thanks, didn't think it would be that simple. I must have missed comment 1d in #27 or failed to realize it's significance.

It may be that there is an alternative method that can be used in PS but considerable searching has not found one.

Can't help there, sorry.

Have you [tried] the method in [the GIMP] using Raw files and can you confirm the results are as clean as a single shot? By clean I mean no mush such as odd patches of a strange colour, as opposed to simply noise.

No, I've never tried that kind of photography and I don't have any raw plug-ins for the GIMP anyhoo. My rare attempts at stacking have been to do with "HDR" or focus. ImageJ/Fiji has lots of stacking functions which I've never investigated or tried - I seem to recall 'average' is in there ...

But, I think I'm on to something and 'possibly' making progress

Aha! Please keep us posted ...

[Stagecoach]

An update .................

As mentioned in previous posts my investigation into 'noise differences when using different processes' was put on hold whilst I tried to fathom out why I was getting a difference in image quality between a number of frames that were stacked 'in camera' in the 'additive' mode compared to a number of frames that were stacked 'in PS' in the 'additive' mode.

To recap, the difference in image quality was that those stacked in PS showed some areas of coloured mush, generally green and magenta predominately visible in the sky that were not evident in the camera stacked equivalents. This problem was evident to varying degrees whatever combination of frame 'darkness' or number off.

A further recap being that the 'additive' mode I am referring to is where a number of 'darker' exposures are summed together in a way that the final image is 'brighter' than the individual ones.

So, my findings to date ......................................

Investigating one of the Raw files of a set in ACR I noted an area of clipped blacks and decided to change the assigned profile from Standard to Neutral knowing this would give me a less contrasty image removing the clipping and it also lightened the sky slightly. I ran a set adjusted to Neutral profile through the the process and there was a distinct improvement with far less mush in the sky. I then experimented further with WB and found I could improve things further.

The conclusion I'm coming to is that when the additive process takes place 'in camera' with the Raw frame it is likely done without any 'profile' being assigned to it at that time. When the additive process is undertaken in PS with a Raw frame a 'profile' has been assigned to it. Whilst the 'Camera Neutral' profile is giving far improved results it may be that 'Adobe Neutral' is different so I'll have a look at this too.

This may also be supported by the fact that 'profiles' are not so important or the affect less noticeable when images are stacked using the 'Mean/Average' process in PS.

Here's last nights progress;

No 1 - ******* Correction this image taken at ISO200 not ISO1600 as labeled *********


No 2


No 3


No 4 - This frame being brighter than the scene viewed by eye


Observation;

Stacking using the 'Summation' mode in PS using just two frames gave a 'brighter' end result than that with 6 frames at equivalent exposures stacked in camera.

Investigations continue ..........................................

[revi]

Wouldn't the big difference between the two be that PS works on demosaiced images, where in-camera the raw sensor data are used?
The input profile is, afaik, applied after demosicing. Another difference might be the point if application of raw black and white points:
in-camera, that can be done either before or after the stacking, in PS it seems necessarily done before the stacking (if stacking is done
after demosaicing). And I've noticed that demosaicing tends to "smear" hot pixels.

[Stagecoach]

Wouldn't the big difference between the two be that PS works on demosaiced images, where in-camera the raw sensor data are used?
The input profile is, afaik, applied after demosicing. Another difference might be the point if application of raw black and white points:
in-camera, that can be done either before or after the stacking, in PS it seems necessarily done before the stacking (if stacking is done
after demosaicing). And I've noticed that demosaicing tends to "smear" hot pixels.

You may well be right Remco.

Further experimentation has now shown that changing to 'Adobe Neutral' profile in ACR for the images I am stacking in PS 'Summation' mode is giving even better results almost equivalent (and good enough for now) to what I get out of the camera with respect to final stacked image IQ.

But, I'm still not understanding why I'm needing less images at exposure 'x' using PS 'Summation' stacking than the number of same exposure 'x' ones used in camera 'additive' stacking for the same output image 'brightness'. What the processes are doing can't be the same

[DanK]

But, I'm still not understanding why I'm needing less images at exposure 'x' using PS 'Summation' stacking than the number of same exposure 'x' ones used in camera 'additive' stacking for the same output image 'brightness'. What the processes are doing can't be the same

Maybe gamma encoding?

[xpatUSA]

<> But, I'm still not understanding why I'm needing less images at exposure 'x' using PS 'Summation' stacking than the number of same exposure 'x' ones used in camera 'additive' stacking for the same output image 'brightness'. What the processes are doing can't be the same

Clearly not the same, Grahame, having observed 2 vs 6 in your #33. But, unless you can hack into and understand Nikon's and PS's code, you'll never know why. Similarities abound in the genre. There's a long thread going on elsewhere about a difference in reds between embedded JPEGs in raw files and JPEGs converted from those same raw files by the proprietary converter. In that same converter, there's even a difference between monochrome Mode and monochrome WB.

Still, 2 vs 6 is a bit much - I shudder to think what that might be in equivalent EV ...

[Stagecoach]

Maybe gamma encoding?

Far too many "maybes" at present Dan, I need answers before either my hard drive gets full of trial images or gets thrown out the window

[Stagecoach]

Still, 2 vs 6 is a bit much - I shudder to think what that might be in equivalent EV ...

And this PS 'Summation' mode gets stranger ...............................

Feeding in 2 files at exposure 'x' gives a certain brightness of the output file.

Feeding in 3 files at the same exposure 'x' gives an even brighter output file than with 2.

Feeding in 4 files at the same exposure 'x' gives an even brighter output file than with 3. And so on..............

The above increase in output brightness you would expect, although the reason for the ratios are unknown, but then it becomes truly weird;

Feeding in 3 files, 2 at exposure 'x' and 1 at a much lower (darker) exposure the output is the same as just feeding in 2 at exposure 'x'.

A very confused Grahame

[xpatUSA]

And this PS 'Summation' mode gets stranger ...............................

Feeding in 2 files at exposure 'x' gives a certain brightness of the output file.

Feeding in 3 files at the same exposure 'x' gives an even brighter output file than with 2.

Feeding in 4 files at the same exposure 'x' gives an even brighter output file than with 3. And so on..............

The above increase in output brightness you would expect, although the reason for the ratios are unknown, but then it becomes truly weird;

Feeding in 3 files, 2 at exposure 'x' and 1 at a much lower (darker) exposure the output is the same as just feeding in 2 at exposure 'x'.

A very confused Grahame

Limits kicking in? Perhaps on a per-layer basis, not just a+b+c+ ... ?

Adobe imposing black and white levels that were not in the original?

The GIMP manual gives actual formulae for blending modes, does Adobe?

Not much help, best I can offer at a glance.

Remember, the Elephant in the Room knows best ...