It's something worth knowing how to do, but it's a red herring here. There is no reason why you should need noise reduction in this case. First, your camera should be able to produce very clean images at ISO levels above 100. When I do candids of kids, I routinely shoot at 400 with no NR at all, and that's with a camera (Canon 5d IV) with a sensor older than yours. It is important, however, that you expose well. If you underexpose, as you did in this image, the negative effects of raising ISO will be apparent at lower ISO levels than if you expose to the right.Quote:
Honestly I've never done any noise reduction with my images. This might be the right time to learn how to do that.
Second, noise becomes less apparent as the size of the displayed image decreases. You're not printing, so you are viewing the image at a very low resolution.
My cat Ruby
Printable View
-
24th March 2021, 12:00 PMDanKMy cat Ruby
-
24th March 2021, 05:27 PMxpatUSARe: My cat RubyBoth "resized" and "resampled" will be understood here, Leo. Resampled is technically more correct.Quote:
Originally Posted by lunaticitizen
I tried a procedure similar to yours above and it gave me different SNRs. That was because of uneven lighting and a slight vignette. Hence my previous post #18.
I think your comparison images -the 8 MP ones - will look the the same to your eyes at 100% zoom on your screen, by which I mean that one pixel of image data is one on-screen pixel.Quote:
So my question was, will image A and image B look the same if viewed on the same monitor? By 'the same' I mean will they have the same SNR?
They will probably look the same to your eyes but the SNR could be different due to different content in the crop from the original - unless both the exposure and lighting are perfectly uniform. We could say that in your method - the greater the uniformity, the closer the SNRs. In my similar test the SNRs were significantly different.
OK, that's good.Quote:
The original problem was that the DoF of the subject in my image is too thin.
You advised me to take a few steps back, make another exposure, and crop the image around the subject. I understand that the resulting image will then have greater DoF than my original one.
SNR is equal to the mean signal level of a area divided by it's standard deviation (sd) from mean. So if the mean (average) of the levels is say 77 and the sd is 4 then the SNR is 77/4 equals 19.25. Usually SNR is expressed in dB like in audio technology, e.g. SNR equals 20 x log10(77/4) equals 25.7 dB.Quote:
If you tell me how to calculate the SNR of a JPG file, I'll definitely try that. I didn't have a grey card so I ordered one on Amazon.
To get the mean and the sd of an area you need an app that can select an area in a displayed image and show the area's histogram plus the mean and sd of the levels. The GIMP can do this. Others too, no doubt ... Anybody?
The formula itself applies to any image not just it's noise content, hence my comment about uniformity. It is also OK for the comparison of any image area but the more the variation of the brightness in the area the more different the "SNR" of each area could be - unless the target is uniform and the exposure is expressly designed to show just noise more or less. Example - shooting in the dark with the lens cap on.
Hope this helps rather than hinders your understanding ... -
25th March 2021, 05:01 PMxpatUSARe: My cat RubyThe more I think about it, the more I think that it was incorrect to mention SNR in the first place (# 11) as if it were a key measure of "clean-ness" between two different images, cropped or not, resampled or not.Quote:
Originally Posted by xpatUSA
-
26th March 2021, 10:10 AMlunaticitizenRe: My cat Ruby
OK I got my grey card so let me try the experiment maybe tomorrow. I'll download GIMP too.
-
26th March 2021, 10:15 AMlunaticitizenRe: My cat RubyHmm so what would be the right measure?Quote:
Originally Posted by xpatUSA
DXOMARK lists several measures for their camera testing protocol. I only understand SNR and DR. -
26th March 2021, 11:01 AMlunaticitizenRe: My cat RubyHey Dan, is the image really underexposed that badly?Quote:
Originally Posted by DanK
Here is the raw histogram.
https://LeoCDNendpoint01.azureedge.n...-9568x6376.png
So I guess if I had wanted to do ETTR, I could have given it at least 2 stops more exposure. :eek: But it was impossible in this case wasn't it? Lower f-stop would've resulted in even thinner DoF; lower shutter speed might've resulted in a blurred image.
Using the Auto Tone button in Lightroom, it chose to lighten the image by 0.56 stop.
https://LeoCDNendpoint01.azureedge.n...m-autotone.PNG
Doesn't this mean that Lightroom thinks that the image is underexposed only by that amount? -
26th March 2021, 12:50 PMDanKRe: My cat Ruby
Leo,
I don't recognize the format of the histogram you posted. Here is the histogram I obtained by opening your posted original in Photoshop:
https://photos.smugmug.com/photos/i-.../i-G5bgjXX.jpg
I was making only two points. First, your camera should be capable of clean images at least two, probably 3 stops above base ISO without noise reduction, provided that you don't have to boost shadows much (and possibly even if you do). Second, in the case of an image that doesn't span the entire dynamic range of the sensor--like this image--you can minimize the effects of using a higher ISO by exposing to the right.
You added another issue: are you better off raising ISO and exposing to the right or exposing less and amplifying the image in software? The answer to that depends on the camera. If I understand correctly, in the case of cameras that are misleadingly labeled "ISO-less", the answer is that it doesn't matter. In the case of other cameras, including mine, you are better off raising ISO and exposing to the right, at least at relatively low ISOs. I believe this is a function in part of where in the electronics the amplification is done, but perhaps someone who knows more than I about this can clarify. -
26th March 2021, 12:58 PMxpatUSARe: My cat RubyOK, no problem.Quote:
Originally Posted by lunaticitizen
Measuring the SNR of any scene is OK so long as we understand what the formula measures. Although it's title suggests that noise is measured, the camera is unable to distinguish between "noise" and the variation in levels of a normal scene. Only when we present a scene that itself has no variation in luminance (i.e. a gray card) can we think of the result as comparing only the noise at different framings of a gray card in a scene where the lighting of the gray card does not change and neither does the camera settings (same exposure of the sensor).Quote:
OK I got my grey card so let me try the experiment maybe tomorrow. I'll download GIMP too.
I still assume that by "clean" you are referring to the amount of noise and that you believe that a fully-framed subject has less noise in it than the same subject shot smaller in the frame.
I would suggest:
Shoot the card just less that fully framed - in manual shutter and aperture at 100 ISO and spot metering - adjusted so that exposure deviation is zero. This should give a converted image that is about 120/255 assuming that sRGB , not Adobe, is selected in the camera. The raw histogram should show 18% raw levels or thereabouts - i.e. about 20% of what you get when the sensor is almost saturated.
Then step back without adjusting the camera other than re-focusing the lens and shoot again.
In RawDigger, you can select an area and read the mean and standard deviation of the selection at the top of the window, as you probably know. That would be better for this test.
Select the whole gray card in your first raw image. Determine the SNR of a green channel.
Select only the gray card in the second raw image. Determine the SNR of the same green channel.
I'll do the same here and report results.
If you like, post the results here for comment. -
26th March 2021, 01:23 PMxpatUSARe: My cat RubyDan, it is a raw histogram obtained with RawDigger which I use a lot. The histogram is set to show linear counts versus log levels. You may not be used to that. If we knew the saturation level of the camera we could then examine the actual (sensor) exposure of the shot as opposed to the brightness of the converted image.Quote:
Originally Posted by DanK
-
26th March 2021, 01:30 PMDanKRe: My cat RubyThanks. That certainly explains it. Well, the math anyway. But which is the more useful for present purposes?Quote:
Originally Posted by xpatUSA
-
26th March 2021, 01:42 PMlunaticitizenRe: My cat RubyOops I did a test but my protocol is different than what you suggested above. :(Quote:
Originally Posted by xpatUSA
- I shoot the grey card fully framed at 1/125 s, f/4, ISO 5000. I tried to set the ISO at ISO 100 but the viewfinder went dark and I didn't know how to disable this 'exposure preview' mode on the camera.
- I resampled the original image to 1188x792 pixels and measured its SNR.
- I center-cropped the original image to 1188x792 pixels and measured its SNR.
I understand since this is not a perfectly uniform target, image (2) is different than image (3). I wonder if the difference is negligible?
Here are the result.
First, the raw file.
https://LeoCDNendpoint01.azureedge.n...1/DSC04274.ARW
The original file exported to JPG with white balance corrected.
https://LeoCDNendpoint01.azureedge.n...-9504x6336.jpg
The resampled image. The SNR is 20*log(121.40/7.61) = 24.1 dB
https://LeoCDNendpoint01.azureedge.n...d-1188x792.jpg
The cropped image. The SNR is 20*log(130.99/7.63) = 24.7 dB
https://LeoCDNendpoint01.azureedge.n...d-1188x792.jpg
The resampled image is definitely cleaner than the cropped one but their SNRs are similar. :confused:
Btw I determined the mean values and standard deviation values using the histogram in Photoshop as Dan showed above. :p
I'll try your protocol tomorrow. -
26th March 2021, 01:48 PMlunaticitizenRe: My cat RubyUnderstood, Dan. Thank you.Quote:
Originally Posted by DanK
-
26th March 2021, 02:04 PMxpatUSARe: My cat RubyFully framed: 11.07dBQuote:
Originally Posted by xpatUSA
Stepped back: 11.37dB
RawDigger green channel. -
26th March 2021, 02:18 PMxpatUSARe: My cat RubyFor my purposes, I prefer to exclude as much of anything aft of the sensor data as possible. Therefore the raw histogram is telling me more what the basic camera did excluding WB, conversion, ISO tricks, etc.Quote:
Originally Posted by DanK
In Leo's case, he maybe prefers to compare final images - so I'll repeat my test with the same shots but converted images.
later. -
26th March 2021, 02:37 PMxpatUSARe: My cat RubyFully-framed: 17.361dB.Quote:
Originally Posted by xpatUSA
Stepped back: 17.252dB.
Images converted to best quality JPEG sRGB with the proprietary converter and compared in the GIMP (can't open JPEGs in RawDigger).
My results versus Leo's show that Leo is not currently comparing apples to apples. My results also show to my satisfaction that Leo's apparent belief that a fully-framed subject has less noise in it than the same subject shot smaller in the frame is false. -
26th March 2021, 03:13 PMxpatUSARe: My cat RubyThe GIMP disagrees significantly with Photoshop:Quote:
Originally Posted by lunaticitizen
I get 23.7dB for the smooth image and 22.4dB for the crop. Not saying that Photoshop is wrong, just different - although the value of the sd for the noisy image doesn't look right to me. I'll see what ImageJ says ...
... Image J gives 24.42dB and 21.99dB respectively. Of the three sets of values, I would believe ImageJ the most and Photoshop the least.
When you get the GIMP, perhaps have another look at your two above images. -
27th March 2021, 12:08 AMlunaticitizenRe: My cat RubyOK this is the second test. I made the measurement on the raw files using RawDigger.Quote:
Originally Posted by lunaticitizen
I've no idea why the SNR of DSC4278 is lower than that of DSC4279. Maybe I did something wrong.
Overall scene.
https://LeoCDNendpoint01.azureedge.n...DSC04281-2.jpg
Scene 1 (DSC4278 raw file)
https://LeoCDNendpoint01.azureedge.n...1/DSC04278.jpg
Scene 2 (DSC4279 raw file)
https://LeoCDNendpoint01.azureedge.n...1/DSC04279.jpg
SNR measurement result
https://LeoCDNendpoint01.azureedge.n...er01/excel.PNG -
27th March 2021, 12:38 AMlunaticitizenRe: My cat RubyWell, I trust your numbers. So I redid the calculation using another tool called Pillow, a Python library usually used for machine learning.Quote:
Originally Posted by xpatUSA
The smooth image
The noisy imageCode:> python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 22:45:29) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import asarray
>>> from PIL import Image
>>> import math
>>>
>>> image = Image.open('resampled-1188x792.jpg')
>>> pixels = asarray(image)
>>> pixels = pixels.astype('float32')
>>> mean, std = pixels.mean(), pixels.std()
>>>
>>> print('Mean: %.1f\r\nStandard Deviation: %.1f\r\nSNR: %.1f dB\r\n' % (mean, std, 20*math.log10(mean/std)))
Mean: 121.3
Standard Deviation: 7.9
SNR: 23.7 dB
Code:> python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 22:45:29) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import asarray
>>> from PIL import Image
>>> import math
>>>
>>> image = Image.open('centercropped-1188x792.jpg')
>>> pixels = asarray(image)
>>> pixels = pixels.astype('float32')
>>> mean, std = pixels.mean(), pixels.std()
>>>
>>> print('Mean: %.1f\r\nStandard Deviation: %.1f\r\nSNR: %.1f dB\r\n' % (mean, std, 20*math.log10(mean/std)))
Mean: 130.9
Standard Deviation: 10.9
SNR: 21.6 dB
-
27th March 2021, 02:36 AMxpatUSARe: My cat Ruby
In your second test, perhaps the "stepping back" was a little extreme. The close image contains 271K pixels in my selection and the far image only about 359 pixels, again in my selection. In the case of the cat, I was thinking about half a meter for close up and maybe two meters for stepped back. A distance ratio of 4 therefore a pixel count ratio of 16, certainly not 755.
Pixel count ratio equals distance ratio squared ...
GIMP says 34dB for close and 12dB for far - quite inconclusive because of that large ratio. Your results are nicely presented though - well done! -
27th March 2021, 02:56 PMlunaticitizenRe: My cat Ruby
Thanks. This is the third test using synthetic images. Here is the methodology:
- I created a synthetic, 4K-sized uniform JPEG image with Pillow library. (image A)
- I introduced Gaussian noise and Poisson noise to the above image, which simulates read noise and photon shot noise, respectively . (image B)
- I downsampled image B to 1920x1080 pixels for viewing on an FHD monitor. (image C)
- I cropped image B to 960x540 pixels. (image D)
- I upsampled image D to 1920x1080 pixels for viewing on an FHD monitor. (image E)
- I measured the mean, standard deviation, and SNR of all the images above.
I found that the SNR of the downsampled image (image C) is higher than that of the cropped-then-upsampled one (image E). The downsampled image is also cleaner when viewed on my 24-inch FHD monitor.
Here are the downsampled and upsampled images (image C and image E, respectively):
Image C
https://LeoCDNendpoint01.azureedge.n...ownsampled.jpg
Image E
https://leostorage02.blob.core.windo..._upsampled.jpg
Here are the measurement results:
And finally here is the code.Code:
Image A
Mean: 118.0
StdDev: 0.0
SNR: inf
Image B
Mean: 118.0
StdDev: 10.87
SNR: 20.72
Image C
Mean: 117.5
StdDev: 4.87
SNR: 27.65
Image D
Mean: 117.49
StdDev: 10.88
SNR: 20.67
Image E
Mean: 117.49
StdDev: 9.73
SNR: 21.64
Code:from PIL import Image
from numpy import asarray
import numpy as np
import math
import cv2
""" Add photon shot noise (poisson noise) and read noise (gaussian noise)
to the image
"""
def noisy(noise_typ,image):
if noise_typ == "gauss":
row,col,ch= image.shape
mean = 0
var = 0.1
sigma = var**0.5
gauss = np.random.normal(mean,sigma,(row,col,ch))
gauss = gauss.reshape(row,col,ch)
noisy = image + gauss
return noisy
elif noise_typ == "poisson":
vals = len(np.unique(image))
vals = 2 ** np.ceil(np.log2(vals))
noisy = np.random.poisson(image * vals) / float(vals)
return noisy
""" Calculate image's mean, standard deviation, and SNR """
def mean_std_snr(img):
img_int = asarray(img)
img_float32 = img_int.astype('float32')
img_mean, img_std = img_float32.mean(), img_float32.std()
img_snr = 20*math.log10(img_mean/img_std)
return({'mean': img_mean, 'std': img_std, 'snr': img_snr})
files = [{'name': 'mgrey3840x2160_noiseless.jpg'},
{'name': 'mgrey3840x2160_poisson_gauss.jpg'},
{'name': 'mgrey1920x1080_poisson_gauss_downsampled.jpg'},
{'name': 'mgrey960x540_poisson_gauss_cropped.jpg'},
{'name': 'mgrey1920x1080_poisson_gauss_upsampled.jpg'}]
# Generate a 4K-size JPEG image
img0 = Image.new('RGB', (3840, 2160), color = (118, 118, 118))
img0_pix_float32 = asarray(img0).astype('float32')
files[0].update(mean_std_snr(img0))
img0.save(files[0]['name'], format='JPEG', quality=100)
# Add photon shot noise and read noise to the generated image
# and calculate the mean, stdev and SNR
img1_pix_poisson = noisy('poisson', img0_pix_float32)
img1_pix_poisson_gauss = noisy('gauss', img1_pix_poisson)
files[1].update(mean_std_snr(img1_pix_poisson_gauss))
img1_pix_int = img1_pix_poisson_gauss.astype('uint8')
img1 = Image.fromarray(img1_pix_int)
img1.save(files[1]['name'], format='JPEG', quality=100)
# Downsample image to FHD-size image for viewing on monitor
# and calculate the mean, stdev and SNR
img2 = img1.resize((1920, 1080), resample=Image.LANCZOS)
files[2].update(mean_std_snr(img2))
img2.save(files[2]['name'], format='JPEG', quality=100)
# Crop the original, noise-added image to 960x540 pixels
img3 = img1.crop((1440, 810, 2400, 1350))
files[3].update(mean_std_snr(img3))
img3.save(files[3]['name'], format='JPEG', quality=100)
# Upsample the cropped image to FHD-size for viewing on monitor
# and calculate the mean, stdev and SNR
img4 = img3.resize((1920, 1080), resample=Image.LANCZOS)
files[4].update(mean_std_snr(img4))
img4.save(files[4]['name'], format='JPEG', quality=100)
# Print the result
for file in files:
print(f'Filename: {file["name"]}\r\n' +
f'Mean: {str(round(file["mean"], 2))}\r\n' +
f'StdDev: {str(round(file["std"], 2))}\r\n' +
f'SNR: {str(round(file["snr"], 2))}\r\n\r\n')