Using eSFR ISO Part 1

This document introduces eSFR ISO and explains how to obtain and photograph the chart. Part 2 shows how to run eSFR ISO interactively inside Rescharts and how to save settings for automated runs. Part 3 and Part 4 illustrates the results for slanted-edge and other measurements, respectively.

Click on the thumbnails below (or the image itself, repeatedly) to explore the many capabilities of eSFR ISO.
Note the description below the image. Results are described in more detail in Part 3.

eSFR ISO setup window. Layout is similar to SFRplus setup. Settings on the lower-left select auto detection of the grayscale step chart, color patches, and wedges. Edge and MTF plot for a single region. Any region can be selected. Large view of edge with Edge and MTF measurements-- for visual comparison. Available in Edge & MTF and Image & Geometry displays. 3D plot. Has somewhat less spatial detail than SFRplus. You can select any of a great many results (including secondary readouts). Lateral Chromatic aberration (affected by the lens and demosaicing) is most significant near the edges of the image. CPIQ Acutance or Subjective Quality Factor: a perceptual measurement that includes viewing conditions and the human visual system. Tonal response: Log pixel level vs. log exposure. Gamma is the average slope in the highlight to dark gray areas. The lower curve can contain local contrast or white balance. Multi-Region display: can become cluttered if too many regions are selected. Options are available to simplify the display. Display the image and selected regions and show a number of geometric results, including distortion, tilt, shifts, convergence angles, and field of view. Checking Crop to ROI lets you compare edge appearance to Edge and MTF results. Edge roughness, which can be decomposed into periodic (aliasing-related) and aperiodic (noise-related) components. Reference colors on the upper-left; input colors on the lower-right of each patch. Probe can be used to examine each color patch in detail for color differences (Delta-E ab, 94, and 2000). Color differences on the a*b* plane of CIELAB color space. Noise measurement from grayscale patches, with tonal response on top. All noise measurements from Multicharts and Multitest are available, including chroma noise, scene-referenced noise, sensor (raw) noise, and ISO 15739 visual noise. Wedge MTF and Onset of Aliasing (where the number of detected bars drops). Wedges are automatically detected when the Wedge checkbox in the Setup window is checked.

eSFR ISO test charts

Imatest E-SFR ISO 2023 test chart comes in a single variant: Extended. Additional variations will be available in the future. All variations are compliant to the ISO 12233:2023 standard. 

Imatest E-SFR ISO 2017 test charts come in three variants: Standard (minimal), Enhanced and Extended. All conform to the ISO 12233:2017 standard. The standard E-SFR ISO test chart, which has only features called for in the ISO standard, is shown below.   Click on any of these charts for larger versions.

Standard (minimal) E-SFR ISO chart: 3:2 aspect ratio, no added features
Enhanced E-SFR ISO test chart: 3:2 aspect ratio, 6 added squares on sides,
16 added color patches, four pairs of added wedge patterns. Extended E-SFR ISO test chart: 16:9 aspect ratio, 6 added squares on sides,
16 added color patches, four pairs of added wedge patterns.

Enhanced E-SFR ISO test chart: 3:2 aspect ratio, shown with extra wedges
(4 low frequency, 16 high frequency total) and pre-distortion for use with fisheye lenses.
A great many options are available.

Sharpness is derived from light/dark slanted edges at the boundaries of the squares, as described in Sharpness: What is it and how is it measured? eSFR ISO can handle a wide range of camera aspect ratios; the left and right sides of the chart may extend beyond the frame or be well inside it.

ISO 12233:2017 Compliance:  The Imatest standard (minimal) E-SFR chart is similar to the chart illustrated in the ISO 12233:2017 standard document, but with a few minor differences that maintain ISO-compliance. (We recommend the Enhancedor Extended charts.)

• There are four registration (fiduciary) marks instead of three. An additional black dot, just above the lower-left registration mark, is used to detect orientation. This change was made because the projective transformation used for automatic region detection requires four points. Annex C, Table C.1, I of the standard states, “Asymmetric features for automatic target detection. These features should be placed within the 4:3 active area aspect ratio, but may be placed for convenience as dictated by use cases.”
   
• The focus-aid patterns in the center of the squares are different from the patterns illustrated in the ISO standard document (which are not described in the standard): they are square (which allows more room for the slanted-edge regions used for measuring sharpness), and they have been designed to minimize interference with automatic region detection. They have no effect on measurement results.
   
• A 20-patch OECF gray scale pattern is used. Annex C states, “The test chart also includes grey scale patches which are used to determine the OECF. While the example chart in Figure C1 uses 16 OECF test patches; a 20 patch version as described in ISO 14524 may also be substituted.”

The Enhanced and Extendedcharts contain valuable features (additional squares, wedges, and color patches) in regions of the chart that are not specified by the ISO standard. These features do not violate the standard in any way.

Obtaining the chart

Summary

High resolution photographic eSFR ISO chart

The three Edge SFR charts– Standard (minimal), Enhanced, and Extended– can be purchased from the Imatest store in a variety of media: inkjet (reflective and transmissive), LVT film (transmissive), and photographic paper (reflective; much higher resolution than inkjet charts). (For Chrome-on-Glass charts, SFRplus is often preferred because Chrome-on-Glass can't support a grayscale.) Inkjet charts can be printed on widebody inkjet printers, but you must have fine materials, skill, and a knowledge of color management. We strongly recommend that you purchase a chart.

Inkjet and photographic charts should be mounted on 32x40 or 40 x60 inch sheets of 1/2 inch (12.5 mm) thick foam board with a spray adhesive (such as 3M^TM 77 or Photo Mount) or double-sided tape (such as 3M^TM #568 Positionable Mounting Adhesive). 1/2 inch foam board stays flatter than standard 1/4 or 3/8 inch foam board. PVC board is also very promising: it may be more durable and it comes in a variety of sizes.

Photographing the chart.

Framing requirements:   Ideal cropping The cropping on the right strictly conforms to the ISO standard, but is not required for practical measurements. With this cropping the calibration of the wedges will be correct (LW/PH is the printed number X 100). These numbers are not used for Imatest analysis.
Close Cropping: The registration marks and the left and right (center row) squares should be within the image. Use SFR if you can't obtain this.	Far cropping: the chart should occupy at least half the image height. This cropping is not recommended because regions near the image boundaries are not analyzed and edges and other features may be too small to provide optimum (consistent) results with low resolution images.

• The boundaries of the chart may extend beyond the image (as shown below) or be well within the image. The software is designed to accommodate a wide variety of framing and aspect ratios. Edges near the left and right boundaries will be properly located or omitted if necessary. If the sides of the chart are inside the image, you should try to minimize interfering patterns that could be mistaken for registration marks. Chart surroundings inside the image should be light to middle gray if possible. If there are problems due to the surroundings, you can crop the image in the eSFR ISO setup window.
• If possible, avoid patterns that could be mistaken for registration marks outside the chart area. In practice eSFR ISO is surprisingly robust in such situations, but it's best to keep surroundings simple, and if possible, close to neutral gray (15-25% reflectance).

Well-framed eSFR ISO image
(though not quite perfect— a little closer would have been better)
It's OK (and quite commonplace) for the sides of the chart to run off the image.

• The chart may be aligned using techniques shown in Building a Low-Cost Test Lab. eSFR ISO tolerates moderate misalignment: a tilt of ~2 degrees, some perspective (keystone) distortion, etc., but every effort should be made to align the chart properly. Moderate barrel or pincushion distortion (<8% SMIA) is also well tolerated, but a pre-distorted chart should be used with extreme fisheye lenses (with high barrel distortion).
• The image can be cropped (starting with) to remove interfering features near the edges, using the Crop borders button in the eSFR ISO setup window.

If exposure compensation is available, increasing exposure by +1 f-stop often produces a better exposure, especially on the grayscale stepchart pattern.

Lighting

The chart below summarizes lighting considerations. The goal is even, glare-free illumination. Lighting angles between 30 and 45 degrees are ideal in most cases. At least two lights (one on each side) is recommended; four or six is better. Avoid lighting behind the camera, which can cause glare. Check for glare and lighting uniformity before you expose. A description of recommended lighting setups can be found in The Imatest Test Lab. (We formerly recommended 4700K (near-daylight) SoLux quartz-halogen lamps, but we now recommend LED lamps, which are available with adjustable intensity and color temperature.) The BK Precision 615 Light meter (Lux meter) is an outstanding low-cost instrument (about $100 USD) for measuring the intensity and uniformity of illumination.

Simplified lighting diagram

Distance

A letter-sized (8.5x11 inch) chart printed on Premium Luster paper on the Epson 2200 (a high quality pigment-based inkjet photo printer) was analyzed for MTF using the 6.3 megapixel Canon EOS-10D. There was no change when the image field was at least 22 inches (56 cm) wide— twice the length of the chart. Performance fell off slowly for smaller fields.

Choose a camera-to-target distance that gives at least this image field width. The actual distance depends on the sensor size and the focal length of the lens. The minimum image field is illustrated on the right.

Cameras with more pixels, and hence higher potential resolution, should have a larger image field width, hence printed chart width.

Exposure

The image on the right (taken from the Canon File Viewer Utility) is close to a perfect exposure. Some exposure compensation, typically around +0.7 or +1 f-stop, is often helpful.

Good exposure is important for accurate eSFR ISO results. Neither the black nor the white regions of the chart should clip— have substantial areas that reach pixel levels 0 or 255. The best way to ensure proper exposure is to use the histogram in your digital camera. Blacks (the peaks on the left) should be above the minimum and whites (the peak(s) on the right) should be below the maximum.

Links

How to Read MTF Curves by H. H. Nasse of Carl Zeiss. Excellent, thorough introduction. 33 pages long; requires patience. Has a lot of detail on the MTF curves similar to the Lens-style MTF curve in SFRplus. Even more detail in Part II. Their (optical) MTF Tester K8 is of some interest.

Understanding MTF from Luminous Landscape.com has a much shorter introduction.

Understanding image sharpness and MTF A multi-part series by the author of Imatest, mostly written prior to Imatest's founding. Moderately technical.

Slanted-Edge MTF for Digital Camera and Scanner Analysis by Peter D. Burns (2000) (alternate source). An excellent introduction to the ISO 12233 slanted-edge measurement. Closely related: Diagnostics for Digital Capture using MTF by Don Williams and Peter D. Burns (2001), Applying and Extending ISO/TC42 Digital Camera Resolution Standards to Mobile Imaging Products by Don Williams and Peter D. Burns (2007) (Contains an image of an early version of the low-contrast slanted-edge test chart proposed for the revised ISO 12233 standard.)

Bob Atkins has an excellent introduction to MTF and SQF. SQF (subjective quality factor) is a measure of perceived print sharpness that incorporates the contrast sensitivity function (CSF) of the human eye. It will be added to Imatest Master in late October 2006.

Optikos makes instruments for measuring lens MTF. Their 64 page PDF document, How to Measure MTF and other Properties of Lenses, is of particular interest.