Using SFRreg, Part 2: Running SFRreg

Running SFRreg (Interactive and Auto mode settings)

Setup window – ROI numbering – More settings – SFRreg target detection
Auto mode settings – Warnings – Summary

Imatest SFRreg performs highly automated measurements of sharpness (expressed as Spatial Frequency Response (SFR), also known as Modulation Transfer Function (MTF)) and Lateral Chromatic Aberration from images that contain registration mark patterns (circles that contain two light and two dark quadrants). This document shows how to run SFRreg in Rescharts and how to save settings for automated runs. Part 1 introduced SFRreg and explained how to obtain and photograph the chart. Part 3 illustrates the results.

Open Imatest by double-clicking the Imatest icon on

• the Desktop,
• the Windows Start menu,
• the Imatest folder (typically C:Program files\Imatest\vm.n\Edition (where m.n is the version, e.g., 4.2, and Edition can be Master, Image Sensor, etc.) in English language installations).

SFRreg operates in two modes: interactive/setup and automatic.

To initiate an interactive SFRreg run (in Rescharts), press  Rescharts then 10. SFRreg or press the Interactive/Postprocessors dropdown menu then SFRreg setup. This opens a dialog box for reading an SFRreg file; pressing Rescharts is more general; it lets you open any Rescharts module. Either of these buttons allows you to analyze an SFRreg image, examine detailed results interactively, and save settings for the highly automated SFRreg auto runs (or the even more automated Imatest IT EXE or DLL versions). SFRreg should be run at least once interactively prior to the first SFRreg auto run. SFRreg settings can also be updated by pressing Settings, SFRreg Auto settings from the Imatest main window.

Rescharts SFRreg

Selecting file(s)

The portion of the Rescharts window used for opening files is shown on the right. You can open a file by clicking on Read image file if the correct chart type is displayed, or by selecting a Chart type. One or more files may be selected, as shown below. If you select multiple files, they can be combined (averaged), and you’ll be given the option of saving the combined file.

If the folder contains meaningless camera-generated file names such as IMG_3734.jpg, IMG_3735.jpg, etc., you can change them to meaningful names that include focal length, aperture, etc., with the View/Rename Files utility, which takes advantage of EXIF data stored in each file.

The folder saved from the previous run appears in the Look in: box on the top. You are free to change it. The file name from the previous run is displayed at the bottom. You can open a single file by simply double-clicking on it. You can select multiple files for combined runs (for interactive or SFRreg auto runs) or for batch runs (for SFRreg auto-only) by the usual Windows techniques: control-click to add a file; shift-click to select a block of files. Then click Open. Three image files are highlighted. Large files can take several seconds to load.

File selection

SFRreg setup window

When the file (or files) have been opened, the SFRreg setup window, shown below, appears. This window allows you to select groups of regions (ROIs; shown as violet rectangles) for analysis. It also lets you select the size of the regions, whether to analyze vertical or horizontal edges, and much more. Pressing More settings on the left opens the SFRreg More settings, which allows you to select additional settings that affect the calculations and display. Pressing Auto mode settings lets you select the output figures and files (for SFRreg auto mode runs).

SFRreg setup window; all regions in each registration mark selected for analysis

After you’ve finished making settings, click OK to save settings and continue with the run. You can Click Save settings to save the settings without continuing.

Chart ROI (Region of Interest) numbering

Starting with Imatest 2020.1, ROI numbering should be much more consistent. Prior to 2020.1, ROIs were numbered from inside to outside, regardless of location. That could cause inconsistencies with images similar to the one shown below, which has four registration marks near the corners. Because the corner ROIs all have similar radii (with respect to the image center), small changes in image framing could change the ROI order.

The new ordering algorithm is shown below.

Chart numbering zones

All vertical edges precede all horizontal edges. Registration marks in the center region (which is 20% of the center-to-corner distance) come first if any are present, followed by zones 2-9.

More settings window

The SFRreg More settings window, shown below, opens when

• More settings is pressed in the SFRreg setup or Rescharts windows.
• Settings, SFRreg settings is pressed in the Imatest main window, then More settings is pressed in the SFRreg setup window.

The settings are read from the imatest-v2.ini file, and saved when OK is pressed. Settings are similar to settings in the SFR input dialog box and also the SFRplus and eSFR ISO More settings windows with several settings removed.

SFRreg More settings window

This window is divided into sections: Title and Help on top, then Display options, Settings, Optional parameters, and finally, OK or Cancel at the bottom.

Title defaults to the input file name. You may leave it unchanged, replace it, or add descriptive information for the camera, lens, converter settings, etc.— as you please.

Help opens a browser window containing a web page describing the module. The browser window sometimes opens behind other windows; you may need to check if it doesn’t pop right up.

Zone weights Weights of the center, part-way, and corner zones. Used for calculating weighted means of key results, displayed in the Multi-ROI and 3D plots. The defaults of 1 (center), 0.75 (part-war), and 0.5 (corners) are for typical pictorial photography; corners should probably be given more weight for technical photography.

Reset restores the settings in SFRreg More Settings to their default values.

ISO standard SFR If this checkbox is checked, SFR calculations are performed according to the ISO 12233 standard, and the y-axis is labeled SFR (MTF) (ISO standard). This method is slightly less accurate than the normal Imatest calculation, which incorporates a number of refinements, including a better edge detection algorithm and a second-order polynomial fit to the average edge for a more accurate estimate of SFR in the presence of lens distortion. This box is normally left unchecked; it should only be used for comparing normal Imatest calculations to the ISO standard. The difference is typically very small.

Settings for SFRreg Target Detection (4.5+)

Imatest’s automatic detection routines work hard to detect SFRreg targets in an image regardless of size, lighting, distortion, and their placement confusing, busy scenes. However, many real-world test setups will contain scene content that could be mistaken for registration mark targets. This is especially true when the targets are low contrast and the scene is high contrast (containing light and/or dark areas more extreme than the light/dark areas of the SFRreg mark). The following options can help guide SFRreg detection in the case that auto-detection selects too few targets or mis-identifies scene content as a test chart.

Number of SFRreg targets: Setting the number of SFRreg targets in your scene will speed up detection, and will help find the correct number of targets if the auto-detection amount is incorrect. In general, you should set this if possible. (A value of 0 corresponds to “Use auto-determination of the number of SFRreg targets.”)

The following options should be left at their default values if auto-detection is working on your images, as (in general) they will increase runtime.

Minimum SFRreg mark diameter: Set this value, in pixels, to guide the routine to disregard confusing scene content smaller than your SFRreg targets if it is having difficulty doing so by default. 

Maximum SFRreg mark diameter: Set this value, in pixels, to guide the routine to disregard confusing scene content larger than your SFRreg targets if it is having difficulty doing so by default. 

Contrast Robustness: SFRreg targets that have dynamic range (light area/dark area contrast) significantly smaller than the dynamic range of the rest of the scene will be more difficult to detect. Select from various levels of robustness to low-contrast charts with this menu, at the cost of increased runtime.

Rotation Robustness: SFRreg targets with a -15 to +15 degree rotation are favored by the detection algorithm (in accordance with idea slanted edge rotation for SFR analysis). If targets rotated by a greater amount are not detected, you may use this option to set various levels of robustness to different rotations, at the cost of increased runtime.

When entries are complete, click OK to return control to the SFRreg setup window or to save settings and return to the Imatest main window. When all entries are complete, click either Save settings, OK, or Cancel. Save settings saves the settings for use in automated SFRreg runs, which can be initiated from the SFRreg Auto button in the main Imatest window. OK saves the settings then calculates results for interactive viewing. A sequence of Calculating… boxes appear to let you know how calculations are proceeding. When calculations are complete, results are displayed interactively in the Rescharts window, as shown below.

Auto mode settings window

Auto mode settings affect plots and output files for SFRreg Auto runs as well as Imatest IT/EXE and DLL.It does not directly affect interactive (Rescharts) SFRreg runs. The SFRreg Auto mode settings window, shown below, opens when

• Auto mode settings is pressed in the SFRreg setup window.
• More settings is pressed in the Rescharts window, then Switch to SFRreg Auto mode setup is pressed in the More settings window.
• Settings, SFRreg Auto settings is pressed in the Imatest main window.

SFRreg auto mode settings

The upper box allows you to select single and multi region figures to be plotted and saved in SFRreg auto output. For the most part they are self-explanatory. Note that all plotted figures are saved.

Saved figures, CSV, XML, and JSON files are given names that consist of a root file name (which defaults to the image file name) with a suffix added. Examples:

Canon_17-40_24_f8_C1_1409_YR7_cpp.png
Canon_17-40_24_f8_C1_1409_YR7_MTF.csv

When 3D plot is checked, SFRreg auto plots the last 3D plot displayed in Rescharts unless the More 3D plots button has been pressed and one or more plots has been selected. More 3D plots will be displayed in pink in this case. This allows several 3D plots to be displayed and saved by SFRreg auto.

Close figures after save should be checked if a large number of figures are to be displayed. It prevents a buildup of figures, which can slow processing.

Figures can be saved as PNG or FIG files. PNG files (a losslessly-compressed image file format) are the default— they require the least storage. Matlab FIG files allow the data to be manipulated– Figures can be resized, zoomed, or rotated (3D figures-only), but FIG files should rarely be used because they can be huge. PNG files are preferred if no additional manipulation is required.

A CSV summary file is saved for all runs. An XML file is saved if Save XML results is checked.

You can select either Save CSV files for individual ROIs or Save summary CSV file only (the summary file is always saved).

Save folder determines where results are stored. It can be set either to subfolder Results of the image folder or to a folder of your choice. Subfolder Results is recommended because it is easy to find if the image folder is known.

Warnings

A Clipping warning is issued if more than 0.5% of the pixels are clipped (saturated), i.e., if dark pixels reach level 0 or light pixels reach the maximum level (255 for bit depth = 8). This warning is emphasized if over 5% of the pixels are clipped. Clipping reduces the accuracy of SFR results. It makes measured sharpness better than reality.
The percentage of clipped pixels is not a reliable index of the severity of clipping or of MTF measurement error. For example, it is possible to just barely clip a large portion of the image with little loss of accuracy. The plot on the right illustrates strong clipping, indicated by the sharp corner near the “shoulder” on the black line. The MTF measurement is better than reality. The absence of a sharp corner indicates that there is little MTF error.Clipping can usually be avoided with a correct exposure— neither too dark nor light— and by avoiding high contrast targets (like the old ISO-12233 chart). The maximum recommended edge contrast is 10:1; 4:1 contrast (recommended in the upcoming revision to the ISO-12233 standard) is even better. Low-contrast targets are more reliable overall: in addition to better exposure latitude (reduced risk of clipping), they tend to have less sharpening in cameras with variable signal processing, and MTF results are less sensitive to errors in estimating gamma.		Clipping warnings

SFRreg summary

• SFRreg analyzes images of registration mark patterns, which may be obtained from the Optikos Meridian compact projection system (where the images may be focused at infinity) or from an array or printed patterns, which can work with extreme wide angle fisheye lenses. There should be few or no interfering patterns (resembling registration marks) outside the registration mark charts.
• Lighting should be even and glare-free. Lighting and alignment recommendations are given in The Imatest test lab.
• The first time SFRreg is run, it should be run through Rescharts. This allows
  • parameters to be adjusted and saved for later use in the automatic version of SFRreg, which is opened by pressing Fixed modules (dropdown menu), SFRreg Auto in the Imatest main window.
  • results (listed above) to be examined interactively in the Rescharts window.

Previous: SFRreg Introduction        Next: Using SFRreg Part 3: Results