Comparing two cameras of different MP - best method?

[xpatUSA]

I wish to compare two cameras on the basis of detail/micro-contrast. Normally, I wheel them up to a combined slant-edge/Siemens-star target - frame it about the same and compare edge response using an MTF program.

Less precisely, I could of course shoot a Real World Scene and eyeball the difference.

Lately I wonder if "equal framing" is valid when the MPs are greatly different.

A good example is the comparison between my Sigma SD14 and my SD1M. Different sensor sizes and different MP:

SD14 is 2640px wide and sensor is 20.7mm wide. SD1M is 4704px wide and sensor is 23.5mm wide.

I see two reasonable ways to normalize test images from these two cameras for comparison of detail/micro-contrast:

Shoot so as frame an object to be the same pixel size on the sensor by zooming with the lens or the feet. For my cameras and 1600px object size, the SD14 would frame the object at 61% of the width and the SD1M at 34%, ignoring viewfinder differences, if any.

Shoot from the same distance with the same lens focal length and re-sample either or both images to make the object the same pixel size.

Which of these two ways is more valid and why? (or vice-versa: less valid and why?).

I don't see equal framing as being valid for different crop cameras - what if I were comparing an m4/3 with a so-called full-frame?

Should it be though that "camera equivalence" is a factor, here is one of the better references for that:

http://www.falklumo.com/lumolabs/art...nce/index.html
.

[DanK]

It seems to me that the way to compare is to start with the image one would want and therefore use setups that would give the same image. That would entail using different focal lengths to get the same framing and perspective.

[ajohnw]

Same resolution in respect to the sensor size with detail that would be resolved if the higher pixel count achieves anything. Tricky though as going on the increasing absence of anti aliasing filters the lenses are taking care of that aspect. Not sure what foveon do in this respect. I suspect they don't have to do anything.

To be meaningful I think you need the same level of detail on both sensors but at some pixel count lens performance will make comparison impossible and some detail needs to be more than either can resolve. Standard test resolution test targets can do this sort of thing but can't account for lens effects. That's what they are usually used to measure. To get finer resolution it's just a case of increasing the distance to them. The MTF50 levels can be judged by looking at contrast.

I would need to spend some time reading the link and dinner calls. My initial impression is that lens quality in terms of lp/mm needs to increase with reductions in sensor size. It generally does on many m 4/3 lenses. Nikon 1 goes even higher at times but doesn't relate to the decrease in sensor size really. M 4/3 more or less does in relation to crop but in principle can't match full frame. Optically speaking smaller lenses can produce higher quality images more easily than larger ones. The net effect of that is that if any optical lens type system is simply scaled up it's level of performance will drop off - pretty rapidly too. If scaled down it will improve. Should have added at the same F ratio.

John
-

[xpatUSA]

It seems to me that the way to compare is to start with the image one would want and therefore use setups that would give the same image. That would entail using different focal lengths to get the same framing and perspective.

Thanks Dan,

If I understand correctly, you are using the entire scene instead of my "an object" and going for equal framing (different pixel sizes) and equal distance (equal perspective)?

How do you compare the different sized images for detail/micro-contrast ?

[xpatUSA]

Same resolution in respect to the sensor size with detail that would be resolved if the higher pixel count achieves anything. Tricky though as going on the increasing absence of anti aliasing filters the lenses are taking care of that aspect.

Thanks, John. Does that mean neither method in the OP is valid for comparing detail/micro-contrast?

Not sure what foveon do in this respect. I suspect they don't have to do anything.

Yes, the Foveon has never had AA 'blur' filters. They started at 9.12um pixel pitch and are currently down to 4.32um.

To be meaningful I think you need the same level of detail on both sensors but at some pixel count lens performance will make comparison impossible and some detail needs to be more than either can resolve.

"Level of detail" being sampling rate e.g. sensor pixels/mm?

Good point about lenses. Between the two cameras in the OP, I calculate a total of 0.64 MTF and 0.75 MTF respectively at 40 lp/mm for an average lens at f/5.6 and 40 lp/mm is well below Nyquist for either camera.

[edit] forgot to mention that I would use the same lens when testing my two cameras mentioned in the OP [/edit]

Standard test resolution test targets can do this sort of thing but can't account for lens effects. That's what they are usually used to measure. To get finer resolution it's just a case of increasing the distance to them. The MTF50 levels can be judged by looking at contrast.

Yes, I use QuickMTF Light with the ISO method selected. Good for comparison work.

I would need to spend some time reading the link and dinner calls. My initial impression is that lens quality in terms of lp/mm needs to increase with reductions in sensor size. It generally does on many m 4/3 lenses. Nikon 1 goes even higher at times but doesn't relate to the decrease in sensor size really. M 4/3 more or less does in relation to crop but in principle can't match full frame. Optically speaking smaller lenses can produce higher quality images more easily than larger ones. The net effect of that is that if any optical lens type system is simply scaled up it's level of performance will drop off - pretty rapidly too. If scaled down it will improve. Should have added at the same F ratio.

John
-

Interesting, thanks!

[DanK]

Thanks Dan,

If I understand correctly, you are using the entire scene instead of my "an object" and going for equal framing (different pixel sizes) and equal distance (equal perspective)?

How do you compare the different sized images for detail/micro-contrast ?

My thought was that the most useful comparison is how well you can produce one specific image that you want. To do that requires the same framing and perspective, as well as the same output, e.g., a print at the same size, or a computer image at the same physical size. I wasn't actually commenting on which variables I would examine once I had that in hand. But if you don't hold these other things constant, I don't see the utility of a comparison.

[xpatUSA]

My thought was that the most useful comparison is how well you can produce one specific image that you want. To do that requires the same framing and perspective, as well as the same output, e.g., a print at the same size, or a computer image at the same physical size.

Thanks, Dan, now I understand the POV. So re-sampling is inevitably involved - unless the cameras being compared have exactly the same MP and aspect ratio and no funny stuff like binning, sRAW, etc.

I wasn't actually commenting on which variables I would examine once I had that in hand. But if you don't hold these other things constant, I don't see the utility of a comparison.

For my testing that's where it can get a bit fuzzy because the choice of the variable used to assess detail/micro-contrast does come into play. I once sold a perfectly good m4/3 because it's apparent edge response was worse than a 1.7 crop camera that I already owned, in spite of it being 12MP versus 3.4MP.

I had compared MTFs at a given cycles per pixel instead of a given lp/mm. Duh.

[ajohnw]

I was thinking in terms of image detail on the sensor Ted. Not a pleasant area. Imagine 3 lines say white black white projected onto the sensor and each is the width of a pixel. In principle it should resolve it if they align. Offset it by 1/2 pixel and that is no longer the case. However if it's slanted at some angle it may in places. Go back to aligned on a sensor with larger pixels and again it can't resolve it. There has to be some contrast effects but at some angle it will also resolve it but should for the same exposure capture less light, contrast again.

I was trying to think of some way of taking the lens out of the equation. There may be holes in what I have gone through but I'm sure you will know what I mean. There have been some utterance on photography through microscope in this general area. There as the image is scaled to suit the sensor only megapixels count. Lower magnifications need more even though they have lower na.

You can't take the lens out of it really on a camera if it's going to be meaningful in use. I think I would use test targets at a distance where the finest detail will project at pixel size or smaller on the sensor. Best lens at it's best aperture same distance. Contrast should should then tell you what you want to know but focusing will be critical. If you change framing the scaling of the image on the sensor will change. You could use same framing as well to account for the changes that will be seen in actual use due to different sensor sizes. I would have thought that same distance would be adequate though and any fall off due to increased width noticeable. I haven't used any test charts for years. Once a UK magazine used to give them away regularly to try and improve manufacturers consistency more recently I've used USAF types, several on a board.

John
-

[xpatUSA]

I was thinking in terms of image detail on the sensor Ted. Not a pleasant area. Imagine 3 lines say white black white projected onto the sensor and each is the width of a pixel. In principle it should resolve it if they align. Offset it by 1/2 pixel and that is no longer the case. However if it's slanted at some angle it may in places. Go back to aligned on a sensor with larger pixels and again it can't resolve it. There has to be some contrast effects but at some angle it will also resolve it but should for the same exposure capture less light, contrast again.

The textbook example for early sensor resolution measurement, usually followed by the much easier slant-edge technique.

I was trying to think of some way of taking the lens out of the equation. There may be holes in what I have gone through but I'm sure you will know what I mean. There have been some utterance on photography through microscope in this general area. There, as the image is scaled to suit the sensor, only megapixels count. Lower magnifications need more even though they have lower na.

Yes, Nikon's publications on microscopy are highly informative in that regard.

You can't take the lens out of it really on a camera if it's going to be meaningful in use.

What I can do is use the same lens for Sigma camera tests. That lens's properties do not change between cameras for a given lp/mm at the sensor. How that affects "meaningful" I don't know. But the sensor MTF does vary with pixel pitch for a given lp/mm at the sensor. The greater the pitch for a given lp/mm, the lower the MTF, as I'm sure that you know.

I think I would use test targets at a distance where the finest detail will project at pixel size or smaller on the sensor. Best lens at it's best aperture same distance. Contrast should should then tell you what you want to know but focusing will be critical. If you change framing the scaling of the image on the sensor will change. You could use same framing as well to account for the changes that will be seen in actual use due to different sensor sizes. I would have thought that same distance would be adequate though and any fall off due to increased width noticeable. I haven't used any test charts for years. Once a UK magazine used to give them away regularly to try and improve manufacturers consistency more recently I've used USAF types, several on a board.

John
-

BTW, have you decided if one of the two methods posited in the OP is more valid than the other, apropos of the original question?

[ajohnw]

I don't think either are really valid Ted so suggested something else. I don't see any need to resample and feel that fixed distance for both is best.

The view with the wider sensor will be wider and maybe the lens isn't as good at this slightly wider angle. In terms of better photo's that matters. It's pretty easy to print resolution targets and for comparison they don't need to be highly black and white. Zooming or anything like that will alter the lens characteristics. Resampling to the smaller size needn't show anything and doing that the other way would be unfair on the smaller sensor.

John
-

[xpatUSA]

I don't think either are really valid Ted so suggested something else. I don't see any need to resample and feel that fixed distance for both is best.

Thanks for giving this some more thought, John. So, with no re-sampling and a constant distance my images will come out on my screen with the one about twice as wide as the other. Prompts the same question that I asked of Dan: how would you view those two images for comparison of detail/micro-contrast?

Probably OK for a good slant-edge target and the same ROI in pixels, I'm beginning to think.

The view with the wider sensor will be wider and maybe the lens isn't as good at this slightly wider angle. In terms of better photo's that matters. It's pretty easy to print resolution targets and for comparison they don't need to be highly black and white. Zooming or anything like that will alter the lens characteristics.

Usually I use the DG (FF) prime 50mm f/2.8 Macro which seem quite good at the flat field thing and is preety sharp into the bargain. As to angle, the cameras are close-ish 1.7 crop vs. 1.5 crop.

Resampling to the smaller size needn't show anything [?] and doing that the other way would be unfair on the smaller sensor.

Got to agree with that. There's a member of DPR and DPI, that compares Sigmas by upsizing images to 39MP (double size or even greater for earlier models) and by post-processing to boot! Another gent on the same forum prints everything at 8x10" as a basis for comparison. And so it goes, on and on and on . . .

[ajohnw]

I'd just view the results on the screen Ted. As you know contrast is resolution and where the term MTF50 comes from - contrast reduced by 50% in the fashion it's calculated. Rayliegh's limit too, contrast circa 7 1/2% if I remember correctly. Ok for stars which are more or less black and white.

Then maybe look at what that means when shots are reduced. It depends what some one wants to prove really and cameras with that sort of pixel count can produce very large images at circa 300 dpi or be generous and look at 600dpi results.

John
-