Color/Tone Auto (Multitest): fixed (batch-capable) analysis

Introduction

Imatest™ Color/Tone Auto (Multitest) (not available in Imatest Studio) analyzes images of a large variety of test charts for color accuracy, tonal response, noise, SNR (Signal-to-Noise Ratio) and ISO sensitivity using fixed interface that can run batches of images. It can measure white balance and color response for a wide range of lighting conditions and scenes. It can also display the tonal response of monochrome charts (or monochrome sections of color charts). A color correction matrix can be calculated.

Color/Tone Auto can analyze all the charts supported by Color/Tone Setup (Multicharts) and can produce most of the results plots. It can also incorporates all important functions of the Colorcheck and Stepchart modules: it is similar in operation, but able to analyze more charts and display more results.

Major test charts supported by Multitest

24-patch Colorchecker		IT8.7		Contrast Resolution		Colorchecker SG		DSC Labs Chroma- DuMonde 28/28R
36-Patch DR chart		Q-13/Q-14 and other grayscale stepcharts		Universal Test Target		ColorGauge (Micro, Nano, etc.)		RezChecker
SpyderCHECKR		ISO-15739		General mxn charts (L*a*b* reference file required)		TE269		ISO-14524
Also: SFRplus 20-patch color, SMPTE/Hale Color Bars (6 patches), TE226 (9×5) chart

A full list of supported charts can be found on Multicharts– Supported charts.

Getting started

Photograph or scan the chart, taking care to illuminate it evenly (±5%) and to avoid glare, which can be problematic in charts with semigloss (SG) and glossy surfaces such as the reflective IT8.7 and ColorChecker SG. Glare can be especially challenging to control with wide angle lenses. To minimize the effects of light falloff in lenses, the chart should typically not fill the image field.

A low-cost lighting setup is described Building a Low-Cost Test Lab. For testing white balance, you can photograph the chart in a scene under a variety of lighting conditions. Small versions of the ColorChecker (the Passport, the SpyderCheckr, or the ColorGauge charts) are especially suitable for this purpose. Some rectangular charts may be rotated by multiples of ±90° if they are geometrically symmetrical (these include the two Colorcheckers and the Kodak/Tiffen Q-13/Q-14, but not the IT8.7 (where top and bottom are different).

Before you run Multitest you should select the chart type to analyze. Click on the Settings button, just to the right of the Multitest button in the Imatest main window (shown on the right) or click on Fixed Modules, Multitest settings. These open the Multitest settings window, shown below. The Chart type dropdown menu (in the Multitest settings window, shown on the right) contains all available chart settings. After a selection has been made it is saved and displayed in the Multitest button, as shown above-right. Additional settings can be changed at this time or after an image file has been read and the Region of Interest (ROI) selected (unless you’ve selected Express mode)— but since Chart Type always affects the next run, a selection should be made before a new chart type is run. Chart selection dropdown menu Chart type should be selected prior to running Multitest.
		Chart type (for next Multitest run)

Automatic region detection

This is also a good time to select automatic or manual region detection, located at the bottom of the Multitest settings window and enabled for charts that allow the automatic option (initially Colorcheck and the 36-patch Dynamic Range chart, with more to come).

Manual/Automatic ROI selection, located at the bottom of the Multitest settings window

Running Multitest —  To start Multitest, run Imatest, then click the Multitest button. A window will open for selecting the file (or batch of files).

To calculate temporal noise, select either two files (for the Imatest-recommended difference method) or 4-16 files (for the ISO 15739 method). Details of the calculation are can be found here.

If you have selected manual ROI detection, after you make the file selection, Imatest will check to see if the image(s) are the same size as recently-run images (it saves several sizes). If it is the same size you’ll be asked if you want to repeat the same Region of Interest (ROI) selection:

ROI repeat window (appears when image is the same size as a previously-run image). Manual ROI detection.

If the image is not the same size as a previous image or if you click No, select new ROI(s) , a window will open asking you to select the region. Do this by drawing a rectangle from one to another corner of the chart you want to analyze. Don’t worry about getting the region exactly right— you’ll be able to refine it.

Initial (coarse) region selection. Draw a rectangle from corner to corner of the chart.
Image from Imaging-resource.com (the Multi-Target from the Samples page for individual camera reviews).

Refine the selection in the fine selection window that opens after you make the coarse selection. The inner squares are different for different charts. (Note: this window will appear if you’ve selected Automatic ROI detection with confirmation with one of the charts that support automatic detection.)

Fine region selection, shown for the ColorChecker SG

• The entire ROI can be moved ( ^ > v < , upper-left).
• The top, bottom, left, or right sides can be moved ( T^ Tv R> R< B^ Bv L> L< , middle-left).
• Corners can be moved individually (any of the 8 buttons in each of the four corners, middle-left).
• Pixel values for the four corners can be entered (X(tl), Y(br), …, where tl = top-left, br = bottom right, etc.). The origin is the upper-left. Be sure to press the Enter key after entering a value.
• You can choose between Fine, Coarse, or Extra Coarse (XC-25) movement (1, 5, or 25 pixels of movement per click).
• The display can be zoomed out or in.
• The display can be lightened to view the dark regions more clearly.
• For some charts (those with relatively large patches) a Fill factor (linear) slider to the left of the Distortion slider specifies the relative size of the squares. Its default value is 0.5. Smaller values can be used where lens distortion or pattern alignment makes alignment difficult.
• The Distortion slider allows highly distorted images (barrel or pincushion) to be analyzed.

When you have completed the fine adjustment, click one of the buttons on the bottom of the window. If you click Yes, Continue, the Multitest settings window, shown below, will open. If you click Yes, Continue in Express mode Multitest will run, display selected results Figures, and save selected results files. 

Color/Tone Auto settings window

Multicharts settings window.

Reference files

Most charts allow reference files to be entered using the Reference dropdown menu in the Settings area. (IT8.7 and CMP DT003 charts require them.) The available settings depends on the chart type. The previous file is generally available in the dropdown list. Here is a summary of the available reference sources.

Except for the IT8.7 and CMP DT 003, reference files must be in CSV (comma-separated file) or CGATS format CSV files can be opened and edited in Excel. Color files (L*a*b*, xyY D50, and xyY D65) have three entries per line; density files have have one entry per line.

If you have a spectrophotometer you can create a custom reference file (CGATS format) using techniques described in Measuring test chart patches.

Results figures

1. 3D Color Difference (with split)

In this plot, the height of the region is the selected color difference, in this case ΔE*_ab (plain Delta-E). The tops of the patch platforms are shown as split colors with the reference color on the upper-left and the actual input color on the lower-right. This plot enables you to compare color measured with perceptual color differences. (They may not match as well as the theories claim.)

This plot is shown with the thumbnails and EXIF data on the right (set in a chckbox in the Multitest Settings window. Other plots will be shown without these results, i.e., with the box unchecked (so they take up less space).

2. Split Color (Reference/Input) plot The Split Color plot shows the reference (ideal) color on the upper-left and the input color on the lower right. If a color correction matrix has been calculated (if the checkbox has been checked), this plot and also the 3D Color Difference plot (above) will display the reference color on the upper-left, the input color on the upper-right, and the corrected color on the bottom. 3D Color difference detail, Color Correction Matrix calculated Split color detail, Color Correction Matrix calculated
3. 2D a*b* plot Displays the a*b* plane of L*a*b* color space. Chroma differences are (very) approximately the distance between the Reference and Camera values. (This difference is ΔCab. The more complex ΔC94 and ΔC00 (CIEDE 2000) equations (described here) are more accurate. If a color correction matrix has been calculated, the corrected values will be displayed.
4. xy Chromaticity This plot is familiar, but less perceptually uniform than the a*b* plot, above. Green gets too much area in this plot. If a color correction matrix has been calculated, the corrected values will be displayed.
5. B&W Density & White Balance The Upper plot displays Log(Pixel level/255) (maximum of 1) vs. input density (-log10(exposure)), i.e., uses density units. The lower plot displays several types of ΔC as a function of patch density. See Colorcheck appendix. If a color correction matrix has been calculated, the corrected values will be displayed.
6. EXIF data and Color Matrix This figure shows selected EXIF data and the results of the Color matrix correction, including the color matrix a description of the settings used to calculate the matrix (Color difference used for optimization, weighting, linearization, …) Mean and maximum olor differences between the input and reference image and between the corrected and reference image.
7. Additional 2D color plot The vectorscope display is shown. One of several plots can be selected here: u’v’ Chromaticity Vectorscope 2D CbCr (from YCbCr) HS (Hue-sat) from HSV HS (Hue-Sat) from HSL These displays are described in Using Multicharts – Displays and Options.
Noise/SNR analysis (not for all charts) The Noise/SNR analysis is only available on charts that have patches large enough to provide good noise statistics. Noise analysis includes standard pixel noise, chroma noise, scene-referenced noise, sensor (raw) noise, and ISO-15739 visual noise. SNR (Signal-to-Noise Ratio) can also be displayed as a ratio or in dB. The patches on the ColorChecker SG shown on this page are too small, but fortunately the Imaging-Resource image has a standard 24-patch ColorChecker to the left of the SG that we’ve analyzed to provide the results shown on the right. Noise analysis is described in Multicharts/Multitest and eSFR ISO Noise. ISO 15739 visual noise was added in Imatest 4.0. The following selections are available. Noise vs. input density (RGB) Noise in pixels or % of maximum pixel level Signal/Noise (S/N) vs. input density (RGBY) The luminance (Y-channel) is used for S. SNR (dB) vs. input density (RGBY) SNR (dB) = 20 log10(S/N). Shown on the right, above. Chroma noise vs. input density Chroma S/N vs. input density Chroma SNR (dB) vs. input density CIELAB (L*a*b*) noise CIELAB (L*a*b*) S/N CIELAB (L*a*b*) SNR (dB) Noise vs. pixel (all patches) These measurements are of primary interest in characterizing sensors from RAW (unprocessed or minimally processed) images S/N vs. pixel (all patches) SNR (dB) vs. pixel (all patches) F-stop (scene-ref) noise & Dyn Rng F-stop (scene-ref) S/N & Dyn Rng. Levels for Dynamic Range are shown as thick horizontal lines (yellow, orange, etc.). Shown on the right. F-stop (scene-ref) SNR (dB) & Dyn Rng ISO 15739 Visual Noise (L*u*v*) ISO 15739 Visual S/N (L*u*v*) SNR, as a fraction ISO 15739 Visual SNR dB (L*u*v*) in decibels (dB) = 20*log10(fraction)