Arbitrary Charts Analyses and Output

Current Documentation

All documentation versions

Analyses & OutputINI settingsChart Definition Files – Chart Definition Utility

 

Data outputs from the Arbitrary Charts Module are different from other modules in Imatest:

  • Only JSON files are output, not CSV or XML.
  • The structure of JSON is substantially different than other modules. 
  • If several images are run in a batch, results from all images may be contained in a single JSON.

Since the Arbitrary Charts Module is based on the concept of a custom layout of features, there is no fixed structure to the measurement results that the Module produces. Instead, the module simply produces results based on whatever it observes in the test image(s). The set of results produced depends on the set available for that type of feature (outlined below) and user options for enabling or disabling each analysis type.

Example: Slanted Edge ROI Outputs

If Imatest observes a slanted edge (based on a SlantedEdgeFeature or from one side of a SlantedSquareFeature) in a test image, it will produce all of the results that can be calculated from a slanted edge, such as MTF (and from this, acutance), chromatic aberration, edge roughness, etc. (If each of these has been requested for calculation in the user options, that is. By default, they typically are.) 

All of these results will then be grouped in the output JSON under a field entry named “Slanted_Edge_N”, where N is the unique index of the edge (more on this below). Each result based on this observed edge will be have its own entry, as seen in the diagram below.

 

 

ROI results vs System Results

Very roughly speaking, Imatest offers some measurements based specifically on a single ROI and some measurements which tell you about the whole “system” based on multiple ROI. 

For example, all the various results related to observing a single slanted edge (see the above figure) are ROI results while a geometric distortion result — which is based on many observed points around the entire image plane — would be a System result

  • Only one of each system result is produced per test image
  • An ROI result is generated for each instance of the corresponding ROI type

Sometimes, a system result will contain an array of per-ROI results, with the relevant measurements gathered in these sub-results. This is very typical of grayscale- and color-patch features, which are used to generate measurements of noise, tonal response, color accuracy, etc. Though Imatest could give all such results individually for each patch at the top level, they are more usefully when offered in the context of the system result.

 

Per-ROI Result Indexing

Per-ROI results are reported in a JSON object located under a field labeled “<ROI TYPE>_N”, where N is the unique index value which identifies each ROI of a given type. (For example: “Random_Field_1”.) These indices correspond directly to the “ind” value entry defining each feature in the chart definition file.

To relate entries found in the Arbitrary Results JSON to specific features on the chart, cross-reference with the chart definition file. (Hint: The Chart Definition Utility can be useful for this, if you choose to show feature indices.) 

 

Slanted Edge Indexing

Slanted edge ROI indexing extends the above rule. This is necessary because:

  1. there are two ways to create a slanted edge ROI on a chart (SlantedEdgeFeature and SlantedSquareFeature), and
  2. each SlantedSquareFeature implies four slanted edge ROIs, but itself only has only a single index.

The extended indexing rule for slanted edges is as follows (and is illustrated below):

  1. If there are S many SlantedEdgeFeatures defined on the chart, the first S indexed results correspond to those. (Note that S is simply 0 for charts that don’t have any SlantedEdgeFeatures defined, so any slanted-square-derived edges will start at index 1.)
  2. Every SlantedSquareFeature adds four edge indices to the list in the order of top, right, bottom, left edge. The order in which these are added is based on the index of the SlantedSquareFeatures.

Edge ROI, outlined in red, have unique and consistent identifying indices, in blue. These are based on the chart definition file only, not position in the test image (or whether or not they appear in the image at all).

Result JSON structure

JSON outputs from the Arbitrary Charts Module have the following hierarchical structure.

Top-level object (with fields) ;
     Info  (Object containing fields with information about the run time, Imatest version, etc)
     Results_array_sources (array of strings naming the image sources which were used to generate the following array of results)
     Results  (array of objects, one per entry in Results_array_sources)

The Results entry of the top level JSON object is an array, each entry of which contains the measurements taken from an image. The ordering of the string entries to Results_array_sources indicates the ordering of the source images which produced the results-bearing-objects in the array contained in Results

Note: Even in the case of a module run on a single image, the Results and Results_array_sources fields each contain a 1-element array containing their respective entries. This makes the JSON structure more consistent and predictable for simple machine parsing. 

Following is a listing of the hierarchical structure of the objects in the Results array. Below, we use the notation that text in bold indicates a  JSON object, and the “[…  something_n …]” construct indicates an array of objects each indexed by different values of n (typically relating to chart feature indices as described above). Standard text indicates a field in an object.

(Note: The following information about output JSONs indicates the structure as of the latest release of Imatest. Exact object names and organization may be slightly different in previous versions.)

Per-ROI Outputs

Slanted Edges

[ … 
     Slanted_Edge_n 
          ROI

               Top_left
               width
               height
          Edge_Properties
               orientation
               angle
               dark_mean
               light_mean
               dark_mean_lin_norm
               light_mean_lin_norm
          MTF
               
mtf
               freqs
               mtf50
               oversharpening_percent
          Acutance
               
acutance
               CPIQ_QL
          LCA
               area_pixels
               area_percent
               cross_pixels
               cross_percent
               R_G_delta_pixels
               R_G_delta_percent
               B_G_delta_pixels
               B_G_delta_percent
… ]

Dead Leaves/Spilled Coins Patterns

[ …
     Random_Field_n
          ROI
               Top_left
               width
               height
          Random_Direct
               freqs
               mtf
               texture_psd
               noise_psd
          Acutance
               acutance
               CPIQ_QL
… ]

Stars

[ …
     Star_n
          ROI
               Top_left
               width
               height
          MTF
               freqs
               mtf
               radii
               angles
… ]

Wedges

[ …
     Wedge_n
          ROI
               Top_left
               width
               height
          Wedge
               freqs
               mtf
               limiting_resolution_bars
… ]

System Outputs

Color
     Patch_Colors
          [ …
               patch_n
                    measured_RGB
                    measured_LAB
                    reference_LAB
                    Delta_C
                    Delta_E
                    Delta_C_94
                    Delta_E_94
                    Delta_H_94
                    Delta_C_2000
                    Delta_E_2000
                    Delta_H_2000
          … ]

     Max_Color_Errors
          Delta_C
          Delta_E
          Delta_C_94
          Delta_E_94
          Delta_H_94
          Delta_C_2000
          Delta_E_2000
          Delta_H_2000

     Mean_Color_Errors
          Delta_C
          Delta_E
          Delta_C_94
          Delta_E_94
          Delta_H_94
          Delta_C_2000
          Delta_E_2000
          Delta_H_2000

     CPIQ_Color_Saturation
          CPIQ_color_OM
          CPIQ_color_QL

System_LCA
     rg_fit_coeffs
     bg_fit_coeffs
     rg_delta_pixels
     bg_delta_pixels
     CPIQ_pixels
     CPIQ_metric
     CPIQ_QL

Tonal_Analysis
     reference_y
     measured_y
     measured_luma
     fit_gamma
     exposure_err_fstops

Noise
     ISOsnr
     saturationLum
     CPIQ_visual_noise_OM
     CPIQ_visual_noise_QL
     [ …
          patch_n
               luma_std
               r_std
               g_std
               b_std
               Lstar_std
               astar_std
               bstar_std
               sceneRefNoise
               sceneRefSNR
               CPIQ_Visual_Noise
                    visual_noise
                    mean_L
                    mean_a
                    mean_b
                    var_L
                    var_a
                    var_b
                    covLa
               ISO_Visual_Noise
                    visual_noise
                    mean_L
                    mean_u
                    mean_v
                    sigma_L
                    sigma_u
                    sigma_v
               horz_freq_content
               horz_val_content
               horz_freq_content_peak
               horz_val_content_peak
               horz_freq_content_interp
               horz_val_content_interp
               vert_freq_content
               vert_val_content
               vert_freq_content_peak
               vert_val_content_peak
               vert_freq_content_interp
               vert_val_content_interp
      … ]

Uniformity_Grid
     *selected channel*_Grid             (note: there may be multiple selected uniformity grid channels, each will have an entry)
     *selected color error*_deltaC     (note: there may be multiple selected uniformity grid color errors, each will have two entries)
     *selected color error*_deltaE
     CPIQ_Results
          Color_Uniformity
               OM
               QL

Video Analysis Results

Auto_Exposure
     Patch_Index
     Frame_Times_seconds
     Scene_Change
          Scene_Change_Onset_seconds
          Scene_Change_Time_seconds
          Scene_Change_Rise_or_fall
     Convergence
          Convergence_Time_Onset_seconds
          Convergence_Time_seconds
          Convergence_Target
          Convergence_Stability
     VCX_Results
          VCX_Version
          Patch_Index_VCX
          Scene_Change_Onset_VCX_seconds
          Convergence_Time_Onset_VCX_seconds
          Convergence_Time_VCX_seconds
          Convergence_Target_VCX
          Convergence_Stability_VCX   

Output File Name and Location

Output file names are formatted as:

 <chart name>-<test image name>.json,        if the INI field [arbcharts] combineOutput is 0 (see settings page) or:

 <chart name>-batch-<timestamp>.json,        if [arbcharts] combineOutput is 1.

 <chart name>-VideoAnalysis-<timestamp>.json,     if [arbcharts] doVideoAnalysis is 1 and valid video file is analyzed.

where:

  • <chart name> is the entry of name field of the chart definition file used during the module run.
  • <test image name> is the test image file name, including the full extension (e.g. “test_image_5.jpg”). 
  • <timestamp> is a time stamp of the starting time of analysis (safe for a filename on all systems as per ISO 8601), of the form ‘yyyymmddTHHMMSS’. 

The default location for the output files from this module is a “Results” subfolder of the user’s home directory. Output may be redirected to a location of the user’s choice by entering a valid path into the Arbitrary Charts GUI window’s “Output save directory” field (or equivalently by entering a valid path string in the [arbcharts] save_dir_name entry of the INI file). This path must both exist and allow writing to disk under the current user’s privileges. Doing so will force the creation of the Results subfolder there upon future runs, instead.