Downsampling large resolution sensors to achieve superior prints

[Fredbare]

My first post on a site that appears to be informative and from which I've learnt a lot already.
If this question has been posed before (a search seems to indicate otherwise) please forgive me.
There's been a lot made of downsampling to improve image quality in print - as least as regards comparing different sensors with different resolutions - now that camera resolutions have been improved.
Although I can see that downsampling will improve SNR for a given sensor by 'averaging pixels' I don't seem to be able to grasp how such measurements such as dynamic and tonal range in addition to colour sensitivity can increase when an image is downsized as they are intrinsic properties of the photosites themselves at any given ISO.
A well known site implies that they are improved on downsampling and provides mathematical extrapolations with no derivations to the formulae utilised.
Perhaps those on this site can provide ellucidation why this might be the case (i.e. the 'improvement' part)?
An informed article on downsizing covering these aspects might also not go amiss on the site.

Thanks!

[Colin Southern]

Hi Fred,

I think it's one of those "more theory than practice" type areas (ie probably doesn't produce anything that's decernable with real-world photos).

In theory though, if one lowers the noise floor by averaging then one increases the dynamic range (with DR being the difference between sensor saturation and the noise floor) Too early in the day for me to start thinking about colour sensitivity and tonal range though!

[Fredbare]

Hi Colin,

Thanks for taking the time to answer my query.
My problem remains in that the noise floor of a photosite is an invariant (within manufaturing tolerance), hence the DR will stay the same given a specific 'well size'. Downsizing will not change this attribute of a photosite as we're looking at a single value output, not seperate noise and signal values. Thus I would guess that the DR will stay the same on downsizing, not increase.
If it's otherwise, then I'd like to know why for personal edification if nothing else - and to see if old sports injuries are to blame :-)

Ultimately I agree that DR is a moot point as printers can't render the full gambit anyway (or so I understand).

[Colin Southern]

Ultimately I agree that DR is a moot point as printers can't render the full gambit anyway (or so I understand).

Hi Fred,

Not sure if you're meaning gamut -- which is seperate to DR.

Most cameras capture a DR of around 12 stops - which is generally more than we need (monitors only display about the top 6 unless we start compressing the DR -- which we typically do a little, but not 6 stops worth). Most paper can only reproduce 4 stops of DR.

Gamut (range of colours) is different again though; most printers do reasonably well compared to what the camera captures, and monitors are catching up slowly -- but most folks can't see the difference anyway.

With regards to the original questions - you could be right - I really don't know enough about it. I certainly don't think/rely on down-sampling to change anything apart from the number of pixels, personally!

I'll see if Sean would like to chip in on this ...

[herbert]

Well it depends on how you measure the noise floor. Here's some information on the signal to noise ratio:

http://en.wikipedia.org/wiki/Signal_..._%28imaging%29

Basically the signal-to-noise ratio (SNR) is measured as the average of all the pixels divided by the standard deviation of all the pixels.

To calculate the noise floor takes another assumption on what is an acceptable signal to noise ratio. In the case of DxO Mark they use a SNR of 1. So I assume they just rank all the pixels by value and then iteratively remove the brightest pixels until the SNR falls to 1. The average level of the remaining pixels is the noise floor.

What downsizing does is increase the dynamic range when measured from the peak intensity value to the noise floor. So we can interpret this as meaning that the noise floor has reduced more than the peak value after downsizing. This means that the reduction in size has reduced the standard deviation of the dark pixels more than the average value of the dark pixels. That is you have to take away more bright pixels from the set before the SNR reaches 1.

I don't think that there is anything sinister going on in the DxO Mark scores. It just needs to be understood that the down sampling favours large pixel images since they can squash noise when they are down sampled. Anyone who has resized a noisy image for the web can attest to this. Downsizing does get rid of noise a lot more than it reduces the intensity range of the image.

The gain in down sampling may be the same as that commonly attested to the gain when using a larger sensor. Larger sensors capture more light and so given the same electronics they should have better low light performance than a small sensor camera. The gain is stated as the increase in sensor area. In stops it is log2 of the increase in area. For example a 1.6 crop camera is 1.6*1.6 = 2.56 times smaller than a full 35mm sensor. This is log2(2.56) = 1.36 stops difference.

I think the down sampling will give the same kind of benefit. But the exact equation to calculate the benefit I do not know although it could be estimated from sample images with various downsizing examples.

As an aside note that the flip side of the crop camera verses full frame camera is that the depth of field for the same aperture is smaller by the same crop ratio. So f2.8 on a crop is like a f4.5 on a full frame camera (i.e. shooting at f2.8 on a full frame camera will have too shallow a depth of field for equivalence). Since the depth of field is smaller you have to use a higher ISO and thus lose the dynamic range advantage that you had with a full frame camera.

This makes comparing different sensors harder than you think. You need to account for:

- Sensor size
- Sensor technology (e.g. quantum efficiency, electronic noise)
- Number of megapixels

It is probably only fair to compare cameras with the same sensor size, or similar number of megapixels. Then you will probably find that the newest/most expensive technology is best.

The Nikon D800 is certainly an amazing sensor. But anyone with a Nikon/Sony/Pentax with the predecessor generation Sony Exmor sensor for crop cameras will know that.

The bottom line is that it doesn't matter most of the time in the real world anyway. Just expose the image correctly and you will still get a good image. People who shoot slide film with 5 stops of dynamic range will tell you that.

Sure I'd love to be able to push the 1 in 100 images I have a bit in the shadows to see some detail. However most of the time I can live with my 10 stops of dynamic range from my camera.

Alex

[Colin Southern]

Well it depends on how you measure the noise floor. Here's some information on the signal to noise ratio:

I love posts like this Alex - especially the whooooosh sound they make as they fly right over my head!

[revi]

I think I have to disagree on some details with Alex though.
For me, the noise floor would be the level at which the signal has the same intensity as the noise for a S/N ratio of 1
(or: the average value of the pixels is the same as the standard deviation of the noise, µ=σ). No need to remove the
brightest pixels.

And what the downsizing does:
Assume you average over a zone of constant colour. Then in the absence of noise, all pixels will have the same
value. In that case, averaging gains you exactly nothing (except a bit of heat from your CPU doing the
calculations).
But, in reality there's noise. Meaning that the pixels will differ from the ideal value by some amount, some will
be higher, some lower. The average value (over an infinite region) will still be the noise-free value.
Averaging now means that the differences caused by noise will (partially) cancel one another, so your final pixel will
be closer to the 'true' value than the extreme pixels in your original image. So in the final image, all pixels will be
closer to their 'true' values...
This is exactly the same principle as noise reduction through image averaging, where you take several/many
identical shots and then average those. It is possible to show that this lowers the average noise level linear with
the square root of the number of images used (i.e. you get half the noise with 4 images, a quarter with 16 images
and so on, in theory...).

As we reduce the amplitude of the noise (the standard deviation), our noise floor goes down automatically, as it
is defined as level at which signal and noise have the same amplitude.
Note that in the image, the signal has not changed, we can just see details in the shadow parts that were hidden
in the noise before (easy to show by doing the inverse: add a good deal of noise to an image with delicate shadow
detail, you'll agree that the signal has not been changed by adding the noise...)

And yes, a larger number of pixels in the original image will mean you average more pixels for a given output size,
so the noise will be reduced more, but you need a lot more pixels to really see the difference. Say you downsize
from 3000 to 750 pixels (short side). That means that 1 pixel in the output represents 4×4=16 pixels in the original,
so we reduce the noise to 1/4th of the original. Halving that again, would require 8×8=64 pixels in the original for
each final pixel, or 6000 pixels. Assuming 3:2 aspect ratios throughout, that would mean we have to go from a
3000×4500 sensor (14Mp or so) to a 6000×9000 sensor (54Mp...). So, yes larger sensors have an advantage, but
it's not all that enormous, and going down fast with increasing pixel numbers...

So I guess I agree with Alex on his final conclusion:
Interesting mental exercises, but not making much difference in the real world (except perhaps in understanding why
certain things work and others don't).

Remco

[herbert]

I think I have to disagree on some details with Alex though.
For me, the noise floor would be the level at which the signal has the same intensity as the noise for a S/N ratio of 1
(or: the average value of the pixels is the same as the standard deviation of the noise, µ=σ). No need to remove the
brightest pixels.

Hi Remco,

I was proposing a solution for measuring the noise floor from a real-world image that has pixels of many different values. So to find out where the SNR is 1 would require subtracting the brightest pixels until the ratio drops to this level. However one can assume that the noise floor is constant for the specific camera at a given ISO and so you do not need to calculate it for every image. You can do it once using a controlled environment.

In practice if you really wanted to find the noise floor you would use a light box with a grey filter to provide a flat signal across the image (or at least a sample where you will measure). This is what DxO Mark do to measure the sensitivity of the sensor. I believe DP Review use a step wedge on a light box. This allow comparison of highlights and shadows in the same frame.

In this case you do not have to subtract bright pixels since you have an area of constant intensity. The intensity of your light box can be varied with filters and the SNR computed for each. Where the SNR reaches 1 would be considered the noise floor for the sensor.

I hope that makes it a bit more clear.

Thanks for the extra comments about the down sampling benefits. To reiterate your statements one can see that if you use a flat signal image then down sampling infinitely will converge to an average of the signal with zero standard deviation. However a single pixel with no noise is not a very useful image. However stopping part way will reduce the noise and so the noise floor. This will increase the apparent dynamic range of the image, although at the cost of resolution.

Alex

[marstar]

Not sure if this applies because a lot of what you all are talking about is still a bit over my head, so forgive me if it doesnt.

http://nikonrumors.com/2012/04/10/ni...px/#more-37722

I just came across this earlier.

[herbert]

I love posts like this Alex - especially the whooooosh sound they make as they fly right over my head!

Is that because your head is behind a camera?

Anyway I thought I would try and clarify my thoughts on the subject by writing a post. I'm scoring it as a partial success.

Alex

[herbert]

Not sure if this applies because a lot of what you all are talking about is still a bit over my head, so forgive me if it doesnt.

http://nikonrumors.com/2012/04/10/ni...px/#more-37722

I just came across this earlier.

Interesting comparison although the wonderful dynamic range of the D800 is for low ISOs and this is a test of higher ISOs. At ISO 1600 it's a wash but after that the D3s wins, even after down sampling the D800 image. Even though they matched ISO, WB, and exposure I say that on the D3s although the highlights are comparable to the D800 the blacks look a bit blocked up. The D3s seems more/unnaturally saturated.

One problem with a test like this is that the lighting is quite good. One situation where you would want to use ISO 25000 is when it is very dark, e.g. moon light dark. Here the noise may be much worse due to the longer exposure times and increased photon shot noise of low light. Still it is impressive performance from both cameras.

The main outcome of this test is it makes me want a D3s and D800 to play with to see what they can do in the real world. And I am a Canon shooter.

Alex

[Colin Southern]

The main outcome of this test is it makes me want a D3s and D800 to play with to see what they can do in the real world. And I am a Canon shooter.

You'll be wanting a 1Dx then

[herbert]

You'll be wanting a 1Dx then

Well obviously I want that too. Unfortunately I may have to get in the queue behind a few thousand Olympics press photographers (Canon is the official sponsor) and you Colin. I am sorry you've had to wait so long. Let's hope the delays are to make the camera better and not just to make the camera which wasn't ready when it was announced.

Alex

[Fredbare]

Thanks everyone for some excellent answers!
Ultimately I think most will agree that in the final analysis it doesn't make an enormous difference to the perception of the final image.
Would it also be true to say that:
1. Downsampling will also increase the noise frequency/improve noise texture?
2. Increasing the number of pixels per unit area will increase the sensor/chip temperature and hence the thermal noise?

One further question though is that all noise will have a 'positive' value and thus averaging the seperate noise 'generators' (photosites) should result in an average noise that will reach a mean value that is not zero?
e.g. a rectified sinosoidal wave form will result in an RMS value.
Or maybe it's another 'senior moment' I'm having ;-)

Unfortunately since the age of 12 I've had this thirst for knowledge. Sometimes it's a curse.
Hope you'll bear with me.

As there's a lot more information presented than I can get my head around during a working week I'll hopefuly find the time to learn more over the weekend.
Once again, thanks to everyone.

[herbert]

1. Downsampling will also increase the noise frequency/improve noise texture?
2. Increasing the number of pixels per unit area will increase the sensor/chip temperature and hence the thermal noise?

One further question though is that all noise will have a 'positive' value and thus averaging the seperate noise 'generators' (photosites) should result in an average noise that will reach a mean value that is not zero?

If you down sample then you are throwing away high frequency information, i.e. information that only exists between adjacent pixels will be merged. However you then end up with a resampled image where the old medium frequency information is now high frequency. Hopefully the noise is mainly in the high frequency range and so is lost more than anything important.

Increasing pixels per unit area classically makes thing worse for the reasons you state, i.e. there is more going on on the chip so it works harder and noise is higher.

The rest of this information is from memory but I will try and find the source material later. So take it under advisement.

I believe readout noise can be negative with respect to the true value. For this reason Canon sets the black point for their raw files above zero. I think it is usually around 1024 but on the new 5D it may be 2048. Nikon on the other hand truncate negative noise and have a zero black point. This effectively reduces the noise at low levels. They also do some non-linear response curve mapping on their raw data. So the data is not actually raw.

I will try and find the extensive paper I read about on this topic and post up some quotes. It is very interesting. I remember being persuaded that Nikon had a good system going for much more usable images but one that makes the camera appear better than it is. But better images is what we photographers are about.

However their manipulation of zero black points and low level readout is not good for accurate astrophotography. You may have read that Nikons are not used for astrophotography due to their noise reduction on raw data. I don't think it is noise reduction. However it is a bit of manipulation. I also do not have the numbers of Nikon shooters for astro work so others can correct me if they like.

Nikon may not have published this, or if they did they did it very quietly a while ago. The sensor workings may have been reverse engineered. There are a lot of people in the world and some of them get up to all kinds of investigations.

Maybe someone else can add more information while I search for my source.

Alex

[Fredbare]

Hi Alex,

Thank you for the response.
I'd never thought of reducing the noise floor simply by ignoring lower values.
I'd heard that manufacturers were manipulating data before raw storage - certainly appears logical from what you have written

People on the forum have been very helpful and I've certainly learnt a lot from them.
Hopefully others will find the information in this thread of interest as well.

If you come up with any additional information it would be highly appreciated.

Regards,

[herbert]

Here is the comprehensive reference on Noise:

Noise, Dynamic Range and Bit Depth in Digital SLRs

It was written in 2008 but it is still very relevant.

The section I was referring to with regard to the Nikon manipulation of raw data is here:

theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html

Read the section titled 'An aside on "lossy" NEF compression'.

The section discusses recording the light at high and low points in the intensity range of the image, e.g. 0-4095 for 12-bit files. The article states that the photon shot noise at high levels is more than the difference between levels, i.e. the randomness of light varies more than the step difference in brightness recorded in the image file. For this reason you do not need to record the exact value. You can compress the upper values together and there will be no difference in the image to the visible eye. However in the low levels you need to record all the information. So the low levels are not compressed. This means that you can record a greater amount of information in the same space by compressing part of the image data. This is interesting but does not constitute noise reduction. It is simply a neat way to save space in the image file.

Zero clipping:

http://theory.uchicago.edu/~ejm/pix/.../noise-p2.html

Read the top section titled 'Read noise, Shot noise'. In this section it states that read noise can be negative. It then states that Canon applies an offset to the data allowing a full capture of the noise histogram where as Nikon do not and so the noise histogram is clipped. This makes it harder to analyse and remove the noise from Nikon cameras.

Noise Reduction on raw data:

http://theory.uchicago.edu/~ejm/pix/.../noise-p4.html

This analysis shows that Nikon applies noise reduction to raw data in images taken over 1/4 of a second. This cannot be turned off in the camera. It must be this that is a problem for astro photography since they will consistently use longer exposure times than 1/4 second and will not like their files having noise reduction applied.

Hope that helps with your question on noise only having a positive value.

Regards,

Alex

[Fredbare]

Excellent Alex,

Thank you for the details
Weekend here now so I can spend some time going through the data.

You've been more than helpful.

Regards,