Lenses tests

Shootings

It was during my many exchanges of emails with my friend Victor that the idea came to try a series of lens tests. Victor often asks my opinion on such or such lens seen on eBay which I have one, and I needed an objective way to compare.
Till then, I went outside and did a series of the same view with different lenses, but we looked for a more scientific method...
First method: test using patterns. I had done tests like this at the school of photography in 1980 with very fine grain film, I remembered that to copy the method, using digital photography. This test allows to judge the "sharpness" of a lens, but it is clearly insufficient for a full trial, particularly in terms of color rendering. It will require supplement later.

First challenge: obtaining a good quality test chart. After much research on the internet I found this one, reschrt3.gif, on http://members.cox.net/lenstestr1/lenstest.htm. The author points out that printed, it must measure 4.5"x5.5 ", that's what I get, black frame included, by printing at 100% from XnView (50% from PSP 7) on my Epson EPL-5800L laser printer, and moreover the result is perfect!
This chart is a copy of the one designed for the U.S. Air Force in 1951, and very often used for testing lenses since. The few companies who market it sell it very expensive, luckily kind surfers publish perfectly usable versions.

Watch out! the image here against is a facsimile, you have to print the original file!

I bought a sheet of hardboard 1m by 1,50 m, I installed it along the wall of the living room (the largest room in the house) The center is 1.50 m above the ground, for practical reasons: being over the sofa! the length of the room does not allow to use the entire surface of the plate, the charts are arranged in a smaller rectangle, 1,30x0,87m. Lighting is provided by two times two 25w 5400°K Sumikon bulbs purchased from Pearl, mounted on my (silver) umbrella tripods bought in China through eBay. Depending on the lens focal length (50 to 58mm), the camera will be about 3.50 m - 4 m (I put the sights in the four corners of the viewfinder,which cuts a little, my sights will not be exactly in the corners of the image.

The shooting will be done using my EOS 400D, 10.1 megapixels, APS-C sensor. Why not with the EOS 5D and it's "full frame" sensor? Some of the lenses I want to test do not fit on it (Volna 1), and my "client" uses a APS-C sensor camera body (like most users of digital SLRs). The 5D has a 12.8 megapixel resolution, brought to the surface of an APS-C, it gives about 8 Mpixels, therefore, equal distance to the chart, the 400D has a higher resolution.

The camera, without its memory card, is connected via its USB to the computer. through the EOS Utility software I control the exposurespeed and the shooting, the images are directly written on the hard disk. They are saved as Raw, the estimation of theresults will be done directly on the images displayed to 200% in the Canon DPP Software, with the ability to change the exposure to make reading easier, without changing the original file. The lamps are said as giving a color temperature of 5400°K, the images will be displayed at this color temperature in the software.

Reading

Rating:
The targets of each series are numbered from 1 to 6, the largest series (here) called -2, then -1 and so the following...
So I chosed to mark starting from 1 to 6 for the largest series and continuing 7, 8, 9 etc. in order of size (it would give a maximum of 24 points for this chart, but in reality the 400D's sensor in these conditions allows at the most up to 15-16 (being very "kind").
So I view every shot at 200% in DPP, eventually altering the exposure to make readings more comfortable (because of vignetting at full aperture, it is not uncommon to have more than one aperture step difference between the center and edges). By default, Canon EOS cameras have "sharpness" set to 2, it is set to zero in my bodies. And I try to transform what I see in numerical results. For the corners, I choose the best one.
This is where subjectivity comes into play: at their best aperture, most lenses have a higher resolution than the sensor of 400D can distinguish, and if I want to have three vertical and three horizontal perfect lines, all my lenses will have a maximum score of 12 and will display an almost identical chart. Now I want to compare! So I rated "kindly", trying to rate, however, the same way for each.

Note: these tests are not "scientific" resolution tests, they just give the resolution of one lens mounted on Thierry's EOS 400D and judged by Thierry. That's all! Beside, I chosed not to make any calculation which would give a resolution in "lines / mm," I only give points. They allow me to compare my lenses, and therefore if you have one of those will allow you to make a little opinion. Another role: they allow to know at which aperture(s) a lens gives it's best.

First observations: on equivalent tests carried out with film, the interpretation of results was sometimes distorted by the difference in contrast between lenses: the "Russian" lenses give often less contrast than the "Japanese" ones (which sometimes gave the impression that the Japanese were better, but objectively it was not true) Here the difference is "leveled" by the digital sensor, the higher contrast results for some "Japanese" by interferences producing "leaking" lines, which could let me give lower marks. I had to do my readings again, considering it, my marks tended to reach a ceiling of 12!

A hitherto little annoying aberration has appeared with digital sensors: it is the coma, that appears when approaching the edges, due to the inclination of the light rays, the images appear to "bleed" outwards (that's why, in theory, users of digital cameras should use optics specifically designed for digital).
Here the top left corner compared to the center.
The coma disappears on all lenses tested here towards f:4 - 5.6. It is not even sure that it is embarrassing, it may be involved in the famous "bokeh" appreciated by so many Internet users of old optics. I who have the habit of using my lenses at f:8 or 11, I do not even know it. Wiewing my ratings, you will have to choose between avoiding it and playing with it...

I had also planned to use this test to estimate the color rendition. That's why I've put this color chart on my "wall". The result is not convincing enough, I'll have to find something else. As an example here, from left to right: Industar-50, Industar-61 L/Z, SMC Takumar 1.4/50. The I-50 gives pretty cold images, I-61 gives a beautiful color saturation, the Takumar gives very yellow. The difference between the two Industar is not clearly visible, while the Takumar's dominant appears well, but note that it will be "erased" by the automatic white balancing, the images here are all set to 5400°K.

Examples

I logged my results into a (well known!) spreadsheet, which allowed me to create graphics, much more convenient to exploit my work.

End