Using SFR Part 1

Current Documentation

Setting up and photographing SFR test charts

Imatest SFR measures spatial frequency response, also known as the Modulation Transfer Function (MTF), of digital cameras using a slanted-edge (light-to-dark) target, as described in Sharpness: What is it and how is it measured? Although it doesn’t provide the as strong a visual indication of MTF as wedge or sinusoidal chats (such as Log Frequency), it provides a more accurate quantitative measurement and uses space much more efficiently.


Old ISO 12233:2000 test chart (left) with Imatest SFR cropping indicated by the red rectangle New ISO 12233:2014 test chart (right– supported with a high degree of automation in eSFR ISO)

Imatest SFR also measures Chromatic aberration and noise. A standardized sharpening algorithm can be used to compare cameras with different amounts of sharpening (not recommended for camera development or lens testing).

These instructions also apply to the Rescharts Slanted-edge SFR module, which is performs identical calculations with a more interactive interface.

The SFRplus and eSFR ISO modules performs identical slanted-edge calculations using special test charts that enable automatic region selection and offer numerous advantages. If you are just starting MTF testing and if the available geometries fit your needs, we recommend one of these two chart/module combinations.

Origins of Imatest SFR The algorithms for calculating MTF/SFR were adapted from a Matlab program, sfrmat, written by Peter Burns (

) to implement the ISO 12233 standard. Imatest SFR incorporates numerous improvements, including improved edge detection, better handling of lens distortion, and far more detailed output. A description can be found here. The original Matlab code is available from peter burns website. In comparing sfrmat 2.0 results with Imatest, keep the following in mind: If an OECF (tonal response curve) file is not entered into sfrmat, it assumes that there is no tonal response curve, i.e., gamma = 1. In Imatest, the default gamma is 0.5, which is typical of digital cameras. To obtain good agreement with sfrmat, you must set gamma to 1.

A horizontal slanted-edge for measuring vertical MTF is shown below; a vertical slanted-edge for measuring horizontal MTF is shown on the right. These edges were cropped from an image of the old ISO 12233:2000 test chart, shown above. A crop area is indicated by a red rectangle.

Cropped slanted-edge image for measuring vertical MTF

Imatest Master can analyze edges of nearly any angle. For best results, avoid exact vertical, horizontal, and 45° edges.

Cropped slanted-edge image for measuring horizontal MTF

Obtaining, assembling, and photographing the target

Summary

You can run Imatest SFR with any clean, sharp, straight black-to-white or dark-to-light edge. The solid areas need to be smooth and uniform: dark on one side and light on the other. For best results contrast should be no more than 10:1; 4:1 is recommended in the ISO 12233:2014 standard.

Obtain a test chart.

Several alternatives are available for obtaining test charts.

Purchase a chart from the Imatest Store. SFRplus or eSFR ISO charts are strongly recommended. The ISO 12233 (QA-72) or QA-77 will do the job, but are not recommended (and no longer a part of the ISO 12233 standard). These (pre-2012) ISO 12233:2000 charts have very high contrast (>=40:1), which often results in reduced accuracy, and they have a great deal of wasted real estate: they lack usable edges in regions of great interest such as the four corners.
- For microscopic work, we recommend the SFRplus chrome-on-glass chart. Precision Ronchi Ruling from Edmund Optics, which come in several varieties, may be suitable for some applications even though they have higher than optimum contrast and edges on only one direction (two directions, i.e., a square or checkerboard pattern would be nicer). Choose a a lp/mm value that results in at least 40 image pixels per line pair (50-100 is even better). The ruling must be tilted by about 5 degrees when photographed.
Create and print a test image from files generated by the Imatest Test Charts module. A variety of configurations, contrast ratios, and highlight colors are available. Contrasts between 10: and 2:1 are recommended. (4:1 contrast is specified in the ISO 12233:2014 standard.) Several Structured vector graphics (SVG) charts are available, including SFRplus and eSFR ISO, which have numerous advantages over the ISO 12233:2000 chart.

Nonlinear signal processing and chart contrast Although Imatest SFR is relatively insensitive to chart contrast (MTF is normalized to 100% at low spatial frequencies), measured SFR is often affected by chart contrast due to nonlinear signal processing in cameras, i.e., processing that depends on the contents of neighboring pixels, and hence may vary throughout an image. Nonlinear processing is almost universal in digital cameras (though you can avoid it by using RAW images with dcraw). It improves pictorial quality but complicates measurements. It takes two primary forms.

Sharpening, applied in the proximity of contrasty features like edges. Boosts response at high spatial frequencies.
Noise reduction, applied in the absence of contrasty features. Attenuates response at high spatial frequencies, i.e., removes fine, low contrast detail (texture), which is interpreted as noise. Many cameras reduce sharpening and increase noise reduction at high ISO speeds.

Different patterns respond differently to signal processing. Slanted-Edge versus Siemens Star compared two chart configurations of special interest because they are included in the ISO 12233:2014 standard.

Signal processing algorithms are proprietary; they are a part of a manufacturer’s “secret sauce” for producing pleasing images. Though they vary a great deal, some generalizations can be made.

Most cameras do NOT apply noise reduction and sharpening uniformly throughout an image.
Contrasty edges tend to have better (more extended) MTF than low contrast edges.

For this reason it can be useful to photograph both a relatively contrasty edge (though not so high that it causes clipping) as well as a relatively low contrast edge. Edge contrast can be selected with Imatest Test Charts. An estimate of chart contrast derived from the average light and dark pixel levels (away from the transition) and gamma is displayed in several places in SFR and Rescharts Slanted-edge SFR. (Estimated chart contrast = (avg. pixel level of light area/avg. pixel level of dark area)^(1/gamma) ).

Nonlinearities are analyzed in depth in the Log F-Contrast module.

Print the charts

Properly tilted test chart

The use of guide marks for tilting the chart

Print the test image with a high quality inkjet photo printer on semigloss, luster paper, or high quality matte paper. Alternatively, send it to a lab to be printed. Dry Creek Photo has an excellent listing. If you are printing small charts, print several to measure center and edge sharpness. Be sure the edges look clean and sharp to your eyes; examine them with a good magnifier or loupe. Chart quality is described in detail in Chart quality and distance, below.

The bitmap charts are intended to be tilted approximately 5.7 degrees (anywhere between 2 and 7 degrees is OK) when they are photographed. They are tilted 5.7 degrees when the tick marks, located near the edges, are vertically or horizontally aligned with the center. This is illustrated by the red lines on the right. Imatest Master can analyze edges of nearly any angle; exact horizontal, vertical, and 45° edges should be avoided for best results.

Bitmap charts are printed straight and physically tilted because the edges print sharper that way. If they were printed at an angle, the printer dot pattern could cause some jaggedness. A 5.71° angle (tan^-1(0.1)) is an offset of one part in 10. SVG charts are less affected by tilting.

Assemble the charts into a target to be photographed.

The following target is described in detail in Building a low-cost Test Lab. It can be used to measure lens performance near the center, part-way out, and near all four corners. The Log F-Contrast and Star charts to the left and right of center are not necessary for lens testing; they provide information about signal processing. Many other chart arrangements and added charts (the X-Rite Colorchecker, for example) are possible. Since 2009 we have used the SFRplus chartfor most of our lens testing.

Target for SFR, Log F-Contrast, and Star, 30x40 inches
Sharpness target (30×40 inch) for cameras up to 13 megapixels

The printed charts are mounted on 32x40 or 40×60 inch sheets of 1/2 inch thick black foam board with spray adhesive (such as 3M^TM 77 or Photo Mount) or double-sided tape (such as 3M^TM Positionable Mounting Tape 568, which comes in 16 and 24-inch widths). If a ColorChecker is substituted, Velcro is recommended so it can be removed for dark storage. 1/2 inch foam board stays flatter than thinner boards. Black board results in less flare light than white board. (Flare light is light that bounces between lens elements and off the inside of the lens barrel, reducing image contrast.)

An image of a horizontal or vertical edge on an LCD monitor (desktop or laptop) may also be used as a target if the monitor does not flicker (many do). The camera should be tilted with respect to the monitor. The disadvantage of this technique is that you have only one edge to work with; you can’t easily create a map of lens performance. Imatest Screen Patterns can display suitable LCD test images. You can set contrast, which should be in the range of 4:1 to 10:1 to minimize clipping. To minimize artifacts (Moire, etc.) the camera must be far enough from the monitor so the sensor pixel “frequency” (1/(2*pixel spacing) at the image sensor) is at least 30% above the Nyquist frequency of the LCD screen. A rule of thumb is that the LCD screen image should take up no more than 1/3 to 1/4 the image width (1/9 to 1/16 the area).

Photograph the target.

If possible, frame the chart so there are usable edges near the center, part-way out, and near the corners. Take care that the chart is properly aligned. A number of useful alignment techniques and tricks are presented in Building a low-cost Test Lab.

Lighting

The chart below summarizes lighting considerations. The goal is even, glare-free illumination. Lighting angles between 30 and 45 degrees are ideal in most cases. At least two lights (one on each side) is recommended; four or six is better. Avoid lighting behind the camera, which can cause glare. Check for glare and lighting uniformity before you expose. A description of recommended lighting setups can be found in The Imatest Test Lab. (We formerly recommended 4700K (near-daylight) SoLux quartz-halogen lamps, but we now recommend LED lamps, which are available with adjustable intensity and color temperature.) The BK Precision 615 Light meter (Lux meter) is an outstanding low-cost instrument (about $100 USD) for measuring the intensity and uniformity of illumination.

Distance

Distance and field of view

The camera must be far enough from the chart so you are measuring the sharpness of your camera and lens, not the chart. But remember,

It’s the field of view, not the chart-to-camera distance, that counts.

A rough rule of thumb: For an inkjet-printed chart the field of view should be at least

22 inches (60 cm) for a 6-megapixel camera; 35 inches (90 cm) for a 16 megapixel camera.

Details below. For high-quality photographically-printed chart you can get somewhat closer.

A letter-sized (8.5x11 inch) chart printed on Permium Luster paper on the Epson 2200 (a high quality pigment-based inkjet photo printer) was analyzed for MTF using the 6.3 megapixel Canon EOS-10D. There was no change when the image field was greater than 22 inches (56 cm) wide— twice the length of the chart. Performance fell off slowly for smaller fields.

Choose a camera-to-target distance that gives at least this image field width. The actual distance depends on the sensor size and the focal length of the lens. The minimum image field is illustrated on the right.

Commercial photographically-printed charts are often have better chart MTF than inkjet printers can achieve, and hence may be used at shorter distances— microscopic in some cases (chrome-on-glass charts).

Cameras with more pixels, and hence higher potential resolution, should should have a larger image field width.

Distance/field width guidelines for high quality inkjet charts
(You can get closer with photographically-printed charts.)

The camera-to-target distance is not critical as long as it is greater than a reasonable minimum.


Sensor sizes
Desig- nation	Diagonal mm.	Width mm.	Height mm.
Height 1/3.6″	5.0	4.0	3.0
1/3.2″	5.68	4.54	3.42
1/3″	6.0	4.8	3.6
1/2.7″	6.59	5.27	3.96
1/2″	8.0	6.4	4.8
1/1.8″	8.93	7.18	5.32
2/3″	11.0	8.8	6.6
1″	16.0	12.8	9.6
4/3″	22.5	18.0	13.5
35mm	44.3	24.0	36.0

Image field width (in inches) = 8.8 × sqrt(megapixels)
Image field width (in cm) = 22 × sqrt(megapixels) — or —There should be no more than 140 sensor pixels per inch of target or 55 sensor pixels per centimeter of the target. — or —The distance to the target should be at least 40X the focal length of the lens for 6-10 megapixel digital SLRs. (25X is the absolute minimun for 6 megapixel DSLRs; 40X leaves some margin.) For compact digital cameras, which have much smaller sensors, the distance should be at least 100X the actual focal length: the field of view is about the same as an SLR with comparable pixel count. The recommended distance is described in more detail in Chart quality and distance, below.

There is some confusion about lens focal lengths in compact digital cameras. They are often given as the “35mm-equivalent,” which many photographers can relate to viewing angle. 35-105mm or 28-140mm are typical “35mm-equivalent” numbers, but they are not the true lens focal length, which is often omitted from the specs. What is given is the sensor size in 1/n inches, a confusing designation based on the outside diameter of long-obsolete vidicon tubes. It The table on the right relates the 1/ndesignation to the diagonal dimension of the sensor. True focal length = “35mm-equivalent” × (diagonal mm.) / 44.3

Exposure

Proper exposure is important for accurate Imatest SFR results. Neither the black nor the white regions of the chart should clip— have substantial areas that reach pixel levels 0 or 255. The best way to ensure proper exposure is to use the histogram in your digital camera. Blacks (the peaks on the left) should be above the minimum and whites (the peak(s) on the right) should be below the maximum.

Low contrast test charts (with contrast ratios of 10:1 or lower) are helpful for avoiding clipping. The revised ISO 12233 chart (due out in 2012) has a contrast ratio of 4:1.

The above image (taken from the Canon File Viewer Utility) is close to a perfect exposure.

Tips on photographing the chart

Distance doesn’t matter as long as the target far enough from the camera so sharpness is limited by the camera and lens, not by the target.

The target should be evenly lit and free of glare.

White balance should be approximately neutral.

Use a sturdy tripod and a cable release. If possible, use the mirror lock. You can use Imatest SFR to find the difference made by a good tripod or mirror lock— to sharpen your technique, literally ( pun intended ).

Be sure to expose the image so detail is maintained in both light and dark areas. Neither should be blocked (clipped). Use your camera’s histogram. If more than 0.5% of the pixels are at levels 0 or 255, Imatest SFR will assume that clipping has taken place and issue a warning message. This has no effect on the calculations— it’s just a warning that accuracy may be compromised. Newer, lower contrast charts have fewer problems with clipping.

Be sure the camera is correctly focused on the chart.

Place slanted-edge images near the corners of the field as well as near the center. (Better: Use SFRplus.)

Save the image as a RAW file or maximum quality JPEG. If you are using a RAW converter, convert to JPEG (maximum quality), TIFF, or PNG. The file name should be descriptive and should indicate the parameters you are testing. The Imatest View/Rename Files module makes this easy. Use dashes and underscores ( – and _ ), but avoid spaces. (Spaces work with Imatest, but they can be troublesome in DOS command lines and web pages.) An example would be Canon_EOS10D_70-200f4L_100mm_f8_ctr.jpg.

You are now ready to run Imatest SFR.

Chart quality and distance

Three printed charts were scanned with the Epson 3200 flatbed scanner at 1200 dpi— a much higher scan density than normal for reflective documents (150–400 dpi). At a typical monitor resolution of 96 dpi, the images are enlarged 12.5x. The charts were tilted around 5.7 degrees (a 1.1 inch offset on an 11 inch letter-sized print) for analysis with Imatest SFR.

Epson 2200
Premium Glossy
1440 dpi
High contrast

Epson 2200
Premium Glossy
1440 dpi
Reduced contrast

HP LaserJet 5L
bond paper
Reduced contrast

The high contrast chart appears to be the sharpest. The reduced contrast chart is still very good: its dot pattern (always a concern) is not a problem. The laser print is clearly unsuitable.

Results of running Imatest SFR on the high contrast printed chart are shown below. The SFR input dialog box was set to display cycles/millimeter for 1200 pixels/inch (scan density). The key result is MTF50 (50% contrast spatial frequency) = 5.15 cycles/millimeter (131 cycles/inch). Results in LW/PH and with standardized sharpening (indicated by (corr.); dashed red lines) are not applicable.

As expected, the reduced contrast chart, which doesn’t appear as sharp, has a lower MTF50: 4.31 cycles/mm (109 cycles/inch). It would benefit from sharpening in the image editor.

The chart MTF50 affects the minimum recommended camera-to-target distance. For example, the Canon EOS-10D (and 300D) has a pixel spacing of 7.4 microns. Its Nyquist frequency is 1000/(2*7.4) = 68 cycles/mm. When the target is 40x the lens focal length, so the magnification at the sensor is approximately 1/40 (recommended above), the MTF50 imaged on the sensor for the reduced contrast chart is 4.31*40 = 172 cycles/mm, which is 2.5 times the Nyquist frequency. The imaged target contrast is approximately 0.9 in the critical region around 0.6*Nyquist (0.3 cycles/pixel). This is adequate, though a little more distance) (e.g., 50x focal length) wouldn’t hurt.

A print of Stephen H. Westin’s printable vector-graphics version of the ISO 12233chart, made on the Epson 2200 at 1440 dpi on Premium Luster paper and scanned by the Epson 3200 at 1200 dpi, has been analyzed to compare slanted and straight edges. The comparison is of interest because Stephen’s chart uses vector graphics, hence the slanted edge is as fine as the printer can make; it is not limited by image pixels. Straight and slanted edges where compared with the print scanned straight and tilted.

Document straight		Document tilted
Slanted edge	Straight edge	Slanted edge	Straight edge

The straight edge is clearly smoother than the slanted edge, though the sharpness of the individual edges (horizontal scan lines) is about the same. This MTF results are surprising: MTF50 for the slanted edge scanned straight is 4.13 cycles/mm, not very different from MTF50 for the straight edge scanned tilted, 4.24 cycles/mm. The difference is more pronounced at lower MTFs (higher spatial frequencies). MTF20 for the slanted edge scanned straight is 8.6 cycles/mm while MTF20 for the straight edge scanned tilted is 10.6 cycles/mm. The straight edge, tilted, clearly looks nicer, although the numbers suggest it would make little difference, I recommend a 20% greater distance if the slanted edge is used.

The results in this box are for charts printed on high quality inkjet printers. Commercially-available charts (printed photographically or by precision techniques) are often printed at higher resolution, and hence may be used at shorter distances— microscopic in some cases.

Documentation – Current v24.2