Seeking opinions on comparing two lenses on same camera

[xpatUSA]

Having a bit of an equipment shoot-out in the front yard:



Which is the fairer comparison for IQ between images from a 50mm lens and a 70mm lens:

Shoot at 5 paces for the 50mm and 7 paces for the 70mm?

Shoot at 5 paces for each one and re-sample the 70mm image downward?

Shoot at 7 paces for each one and re-sample the 50mm image upward?

Would appreciate your choice with supporting reasons!

[Shadowman]

Assuming you are trying to determine the quality of each lens on its own merits I would choose either the second or third choice and include the criteria listed on this site, specifically the SQF rating. I wouldn't necessarily re-sample the image but look for differences from the center to the edge for sharpness and color quality.

http://www.imatest.com/docs/sqf/

[xpatUSA]

Assuming you are trying to determine the quality of each lens on its own merits I would choose either the second or third choice and include the criteria listed on this site, specifically the SQF rating. I wouldn't necessarily re-sample the image but look for differences from the center to the edge for sharpness and color quality.

http://www.imatest.com/docs/sqf/

Thanks John, a very interesting reference which I have read before with some difficulty!

I'll be using visual appearance for center-to-edge - plus MTF and Edge Response plots from QuickMTF, and HSV color accuracy from the barely visible Macbeth card (the latter being more applicable to an upcoming camera comparison).

I tend to agree with not re-sampling - it just introduces another factor into the comparison.

[Scott Stephen]

Or you could let The Digital Picture website do it for you. See http://www.the-digital-picture.com/R...ple-Crops.aspx

[xpatUSA]

Or you could let The Digital Picture website do it for you. See http://www.the-digital-picture.com/R...ple-Crops.aspx

Thanks for the link, Scott.

These people do good reviews too, see http://www.slrgear.com/reviews/index.php

Having said that, I was really looking to test my lenses on my cameras in my front yard, as shown in the OP.

[xpatUSA]

Which is the fairer comparison for IQ between images from a 50mm lens and a 70mm lens:

a) Shoot at 5 paces for the 50mm and 7 paces for the 70mm?
b) Shoot at 5 paces for each one and re-sample the 70mm image downward?
c) Shoot at 7 paces for each one and re-sample the 50mm image upward?

I should have said "lens quality using images" - would have made the intent more clear!

After a day of thinking about the choices. I am beginning to favor a) which is really the same as equal framing as practiced by those who shoot test charts. If one considers the extremes of comparing a 10mm with a 1000mm each at the same distance like b) or c) that approach soon shows up practical and theoretical difficulties.

So a preliminary test reveals the 70mm as slightly "better" vs. the 18-50mm kit lens @ 50mm:

Sharpness (Edge rise 10-90% in pixels): 2.15 vs. 2.35 (lower is better)
Contrast (LPH at MTF50) 711 vs. 673 (image height is 1512px)
Micro-contrast (MTF at Nyquist) 7% vs. 8% (Sigma camera, no AA filter)

The images were soft due to an absolute minimum of conversion processing and due to no post-processing at all. For this reason, I'll not post the images themselves - so as not to excite adverse comment on images properties unrelated to the purpose of the test ;-). Plus, I want to redo the test to get the framing more equal and under more constant lighting i.e. not windy and not with the earth's rotation inducing tree shadows in the scene . . must get up earlier ;-)

The tripod is absolute cr@p which doesn't help - as a noob some years back, I was fooled by the "National Geographic" brand name, duh . .

[Brev00]

Very interesting. The results seem within the range of what one might expect but, for a genuine scientific analysis you are going to have to find a decent tripod and eliminate the other variables mentioned (as best you can). Another thing to consider, perhaps, is the design of the lens. The 70 might be sharper closer and lose resolution with distance. I have not tested this myself but it is something to look for in a macro. A kit lens might have a different sweet spot in terms of distance. In fact, it might be best at the midrange distance tested. So, for your purposes, shooting watches fairly close range, I might include a test closer to a shared minimum distance. I would expect the 70 macro to clean up with that test. Or, do some tests of each set up from a variety of internally consistent distances (minimum, moderate, distant) and see how distance effects resolution. In other words, examine the results of each set up and then compare along a few different lines instead of doing one perfect test.

[xpatUSA]

Yes, good advice - especially with a "sample of one" being statically insignificant.

These tests are leading up a shoot-out between the Sigma and the Panasonic which will no doubt reveal other questions ;-). At least I will be able to compare like-with-like, more or less: 14-42mm (m4/3 20.3 x 13.7mm) vs. 18-50mm (Foveon 20.7 x 13.8mm).

[Dave Humphries]

Hi Ted,

If you're DIY testing, I'd suggest shooting the same framing of test chart and/or scenes with both lenses, but also at a variety of apertures. I suspect the differences between wide open and sweet spot on the same lens are going to be bigger than the differences between lenses.

I try not to obsess about thing such as this unless there's a definite purpose to it; e.g. to make a purchase choice or prove something is faulty. Perhaps because I don't have the time, what with a full time job and so much TV to watch. I don't know your situation, but if I were say, retired and with more time to kill, I might feel differently

If you have both lenses already, can't you just use both for a year and see how your gut reacts to the images produced?
Effective PP sharpening is likely to over-ride the differences, although you may develop and intuition that one lens, or the other, requires a tad more sharpening to achieve the desired finished result.

Cheers,

[xpatUSA]

Hi Ted,

If you're DIY testing, I'd suggest shooting the same framing of test chart and/or scenes with both lenses, but also at a variety of apertures. I suspect the differences between wide open and sweet spot on the same lens are going to be bigger than the differences between lenses.

Good advice and I'm glad you agree with my thought re: equal framing. Yes, I'm retired and am between construction projects and no watches to fix ;-).

If you have both lenses already, can't you just use both for a year and see how your gut reacts to the images produced?

I've had them for over a year (over two for the 70mm), but have recently sold the Nikon D50 and was looking to use the Sigma SD10's 18-50mm kit lens for the few outside shots that I make. Since I've got some analytic software (ImageJ, QuickMTF) it was a good excuse to play with the software.

Effective PP sharpening is likely to over-ride the differences, although you may develop and intuition that one lens, or the other, requires a tad more sharpening to achieve the desired finished result.

Can't agree in this case, sorry. For lens comparisons I want to compare the lenses (at the image plane as far as possible), not my skill at development especially with these two lenses and their different applications. Also, in this case, my sampling at the image plane is with big fat 9.12um sensels, so testing at various apertures may not show up much. By which I mean that what might look like horrendous CA on a 24MP camera, for example, might be barely noticeable on my 3MP Sigma. By the same token, diffraction is hard to spot until about f/16. There's some implicit support for that view from these lens testers (second paragraph).

Might use the pickup truck as a quadrapod for the next tests ;-)

[andrewaxford]

I had a Nikon d 70-300 zoom lens and wanted to get better quality. To decide if I could justify a new lens i took my camera to jesops( now gone unfortunately ) and they let me take a series of hand held shots in the street. I then compared the images IN CAMERA and could easily see for myself the massive difference in quality. Andrew

[xpatUSA]

I had a Nikon d 70-300 zoom lens and wanted to get better quality. To decide if I could justify a new lens i took my camera to jesops( now gone unfortunately ) and they let me take a series of hand held shots in the street. I then compared the images IN CAMERA and could easily see for myself the massive difference in quality. Andrew

Thanks, Andrew,

If you've ever seen a Sigma SD10's rear LCD, you would understand why IN CAMERA wouldn't work for me!