Imatest - Color correction matrix

Introduction

Starting with Imatest 2.4.1, Multicharts can calculate a color correction matrix that can be applied to images to achieve optimum color balance, as defined by minimizing a color error parameter on the test chart of choice. (The default is the mean of (Delta-E 94)² for all patches where L*>10 and L*<95.) The matrix can be used by imaging system designers in their cameras or image processing algorithms. For example, Micron sensors use a 3x3 correction matrix.

Some of the background for the calculation can be found in Color Correction Matrix for Digital Still and Video Imaging Systems by Stephen Wolf, though the Imatest calculation differs in many respects: there is no issue with outliers and optimization is performed using one of the standard color difference metrics.

The color correction matrix is initially included only in Multicharts. In the future it may be added to Colorcheck and additional linearization options will be offered.

The Math

The matrix

Color images are stored in m x n x 3 arrays (m rows (height) x n columns (width) x 3 colors). For the sake of simplicity, we transform the color image to a k x 3 array, where k = m x n. An Original (uncorrected) array O can be represented as

          | O_R1 O_G1 O_B1 |
      O = | O_R2 O_G2 O_B2 |
          |      ...       |
          |      ...       |
          | O_Rk O_Gk O_Bk |

where O_Ri, O_Gi, and O_Bi represent the normalized R, G, and B levels of pixel i. The transformed (corrected) array is called P, where

P = O A (case 1: A is a 3x3 matrix) — or —
P = [O 1] A (case 2: A is a 4x3 matrix; the added column of 1s provides a dc-offset)

A is the 3x3 or 4x3 color correction matrix. For the 3x3 matrix,

          | P_R1 P_G1 P_B1 |    | O_R1 O_G1 O_B1 |  
      P = | P_R2 P_G2 P_B2 | =  | O_R2 O_G2 O_B2 |   | A11 A12 A13 |
          |      ...       |    |      ...       | X | A21 A22 A23 |
          |      ...       |    |      ...       |   | A31 A32 A33 |
          | P_Rk P_Gk P_Bk |    | O_Rk O_Gk O_Bk |

X denotes matrix multiplication. In this case, for row m,

P_Rm = O_Rm*A11 + O_Gm*A21 + O_Bm*A31 (* denotes multiplication.)
P_Gm = O_Rm*A12 + O_Gm*A22 + O_Bm*A32
P_Bm = O_Rm*A13 + O_Gm*A23 + O_Bm*A33

For the 4x3 matrix, a column of 1s is added to provide a dc-offset,

          | P_R1 P_G1 P_B1 |    | O_R1 O_G1 O_B1 1 |   | A11 A12 A13 |
      P = | P_R2 P_G2 P_B2 | =  | O_R2 O_G2 O_B2 1 |   | A21 A22 A23 |
          |      ...       |    |       ...        | X | A31 A32 A33 |
          |      ...       |    |       ...        |   | A41 A42 A43 |
          | P_Rk P_Gk P_Bk |    | O_Rk O_Gk O_Bk 1 |

In this case, for row m,

P_Rm = O_Rm*A11 + O_Gm*A21 + O_Bm*A31 + A41
P_Gm = O_Rm*A12 + O_Gm*A22 + O_Bm*A32 + A42
P_Bm = O_Rm*A13 + O_Gm*A23 + O_Bm*A33 + A43

The goal of the calculation is to minimize the difference (the mean square error metric) between P and the reference array (the ideal chart values) R. The initial values of A (the starting point for optimization) for the 3x3 and 4x3 cases, are

               | k_R 0  0  |               | k_R 0  0  |
      A(3x3) = |  0 k_G 0  | ;   A(4x3) =  |  0 k_G 0  |
               |  0  0 k_B |               |  0  0 k_B |
                                           |  0  0  0  |

where

k_R = mean(R_Ri ; all i)/mean(O_Ri ; all i) for reference array R and original array O
k_G = mean(R_Gi ; all i)/mean(O_Gi ; all i)
k_B = mean(R_Bi ; all i)/mean(O_Bi ; all i)

These starting values are closer to the final values (have less mean square error) than the identity matrix (k_R = k_G = k_B = 1). They tend to converge slightly better.

Linearization

Although most digital image sensors are linear up to the point where they saturate, image files are highly nonlinear— they are designed for display at a specified gamma ( γ ), where display luminance = pixel level^γ. Gamma = 2.2 for the most commonly used color spaces (sRGB, Adobe RGB (1998) and Wide Gamut RGB (WGRGB)), although some well-known color spaces are designed for display at gamma = 1.8 (ProPhoto, Apply, ColorMatch; all RGB).

When cameras encode images (a part of the RAW conversion process), they apply a gamma that is the approximate inverse of the display gamma. Perhaps we should say very approximate: it may vary considerably from 1/γ, and it often includes a tonal response curve "shoulder" (an area of reduced contrast) to minimize highlight burnout. The shoulder makes the response more "film-like," improving pictorial quality in most instances.

The response should be linearized prior to applying the correction matrix, (though Imatest has the option of omitting linearization).

Optimization steps

Linearize the input image (optional). If Color space gamma linearization is selected, O_L = O^γ.
Call the optimizer, which
- calculates a (temporary) corrected image T_L = O_L A.
- Remove the linearization: T = T_L^(1/γ)
- Finds the mean of squares of errors between T and the reference (ideal) array R. The error is one of the standard error measurements: ΔE*_ab, ΔC*_ab, ΔE₉₄, ΔE₉₄, ΔE_CMC, ΔC_CMC, ΔE₀₀, or ΔC₀₀, described here. ΔE₉₄ is the recommended default. Although the CIEDE2000 color error metrics (ΔE₀₀, ...) are more accurate, they contain small discontinuities that can affect optimization, and hence they should be used with caution. See Sharma for details. The ΔC errors are similar to ΔE with luminance (L*) omitted.
- Adjusts A until a minimum value of the sum of squares of the errors is found, using nonlinear optimization.
- Report the final value of A.

In applying A (generally outside of Imatest), a similar linearization should be used. A may be applied during the RAW conversion process, prior to the application of the gama + tonal response curve.

There is no guarantee that A is a global minimum. Its final value depends to some extent on its starting value.

The Color correction matrix in Multicharts

Color matrix settings dialog box Color matrix calculation options can be set by clicking Settings, Color matrix in the Multicharts window. This brings up the dialog box shown on the right. The options are

4x3 or 3x3 matrix: Color correction matrix size. The 4x3 matrix (the default) includes a dc-offset (constant) term. It may be slightly more accurate, but it takes more computation time.
Optimize: Select the color error parameter whose mean of squares over patches with L*>10 (nearly black; color is invisible) and L*<95 (little chroma) is to be minimized. Choices include ΔE ab, ΔC ab, ΔE 94, ΔE 94, ΔE CMC, ΔC CMC, ΔE 00, and ΔC 00, described here. ΔE 94 is the default value, recommended because it gives less weight to chroma differences between highly chromatic (saturated; large a*² + b*²) colors, which is closer to the eye's perception than ΔE ab (the standard ΔE value: the geometrical distance between colors in L*a*b* space).
Linearization: Choose the method of linearizing the image (or leave it unchanged). Current choices include Color space gamma ( O_L = O^γ ) and None ( O_L = O ). In the near future an option will be added to linearze to a polynomial derived from the image data.

To calculate the color correction matrix, read the image into Multicharts, then press the Correction matrix button, shown on the right. The display will change, as shown below. The improvement for this image, which is quite good to begin with, is undramatic.

Split view, showing reference, input, and corrected patch colors

The image now shows the corrected colors on the bottom of each patch. The ideal (reference) color remains in the upper-left and the input (original) color remains in the upper right.

The Correction matrix button changes to Matrix calculated, highlighted with a pink background. The correction matrix cannot be recalculated until an image property changes (new image, color space, reference file, or color matrix setting). The Display input (or Corrected) dropdown menu, immediately to its left, is enabled. You can choose one of two selections.

Display input	Color differences (input − ideal) are shown in most displays, and [Input − ideal] Color differences are shown in the text in the lower left. Two displays are unaffected by this setting: Pseudocolor color difference and Split colors, where (corrected − ideal) is shown on the bottom.
Display corrected	Color differences (corrected − ideal) are shown in most displays, and [Corrected − ideal] Color differences are shown in the text in the lower left.

The EXIF data and Color matrix display has a summary of results.

Exif and Color matrix view

The color correct matrix, results summary, and both [input - ideal] and [corrected -ideal] color difference summaries are shown. The initial and final error numbers shows how much the selected metric (in this case the sum of squares of Delta-E 94 for all patches with L*>10 and L*<95) has changed.