Imatest Documentation

Documentation – v23.1

Index of the Table of Contents
Offline documentation Download the website for offline view (with a web browser)
Image Quality 
IQ factors (KPIs) measured by Imatest, with links to detailed descriptions and instructions.
Sharpness  Introductions to sharpness and sharpening; comparisons of different charts; chart quality limitations & how to overcome them
Other IQ factors  Noise, SNR, Temporal noise, Dynamic Range (DR),  Chromatic Aberrations, Distortion, Veiling glare, Shannon information capacity, etc.
IQ Utilities  Image Processing, SSIM, Image Statistics, Radial geometry 
Getting started Why Imatest?, Test charts, Lighting, Image capture technique, Setting up your lab, Using Imatest software
Start using Imatest software Installation, Using Imatest, Image file formats & devices, Activation/Deactivation
General Instructions Special standards, Supported devices, Test manager, Raw files, Test lab, INI files, Pass/Fail Monitor
Acquiring images Files, formats, devices, and utilities for acquiring images
Knowledge Base highlights Links to articles for troubleshooting, activation, and general advice on operation.
Troubleshooting  What to do when Imatest doesn’t work as expected
Slanted-Edge sharpness modules SFR (manual ROIs), SFRplus, eSFR ISO, SFRreg, Checkerboard (auto ROI detection)
The four auto-detection slanted-edge sharpness module are compared here.
Other sharpness modules Log F-Contrast, Star, Random/Dead Leaves/Spilled coins, Wedge, Sharpness utilities & postprocessors
Sharpness utilities & postprocessors MTF Compare, Image stabilization, Batchview, Find Sharpest Files
Tone, Color, Noise & Dynamic Range modules Color/Tone (Interactive & Auto), CCM (Color Correction Matrix), CDP, Contrast Resolution, Colorcheck, Stepchart, Measuring patches, Gamutvision for evaluating ICC color profiles and gamut mapping.
Spatial & Flat field modules Flatfield, Flatfield Interactive, Distortion, Dot pattern, Testing displays
Miscellaneous modules and utilities Lighting Control, Arbitrary charts, Device Manager, Auto White Balance/Exposure/Focus (AWB, AE, AF), Printing & displaying test charts, Rename Files, Educational Apps, Database
Industrial Testing edition  Modules, Instructions for various interfaces (EXE, C, C++, Python, .NET), INI file reference, Pass/Fail, Operator Console, Speedup
Appendix FAQ, Versions, Troubleshooting, Change Log, INI file reference, License

Image Quality

General — Introduction to Image Quality Factors (Key Performance Indicators – KPIs)

Image quality factorsOverview and Imatest measurements  Recommended for getting started.

IntroductionSummary tableImage quality factorsSharpnessTextureNoiseInformation capacityTonal response – Dynamic rangeColor accuracyDistortionUniformityBlemishesExposure accuracy/ISO sensitivity – Lateral chromatic aberrationStray light (flare) – Veiling glareColor moiré – Software artifactsData compressionPrinter quality factors

Sharpness

SharpnessWhat is it and how is it measured?

IntroductionMTFMTF equationSpatial frequency unitsSummary metrics– MTF measurement matrix – comparing measurement techniques Slanted-Edge measurements – Why a slanted edge?ClippingNoise reductionInterpreting MTF50 – AutoFocus (AF) SpeedCalculation detailsSlanted-edge algorithmImatest vs. ISO calculationLinks

Sharpening and Standardized Sharpening for comparing cameras

IntroductionExamples – Oversharpening/UndersharpeningExamplesUnsharp masking (USM)Links – Standardized sharpening

Comparing sharpness in different cameras  The application: “Image-centric” (for pictorial images) or “object-centric” (for medical, machine vision, etc.) strongly affects how images are compared.

Validating the Imatest slanted-edge calculation

Slanted-edge versus Siemens StarA comparison of sensitivity to signal processing

IntroductionImagesRaw resultsSlanted-edge results – Sinusoidal (Log F-Contrast and Siemens star) resultsExtreme sharpeningSummaryConclusions

Slanted-edge versus Siemens Star, Part 2Results for four additional cameras

Slanted-edge measurement consistency and repeatability comparing different ISO speeds and ROI sizes. Incomplete, but still useful.

Nyquist frequency, aliasing, and Color Moire  introducing a color aliasing metric derived from hyperbolic wedges.

Diffraction, optimum aperture, and defocus – Lens aberrations – Diffraction – Pixel response limits and Q – Defocus 

Display (Monitor) Sharpness – Display the chartCapture the imageAnalyze – ResultsTV LinesMonitor gammaCMS systems

Camera Monitor Systems (CMS) – for automotive mirror replacement systems and some endoscopes

LSF (Line Spread Function) correction factor for slanted-edge MTF measurements

Compensating camera MTF measurements for chart and sensor MTF

Introduction – Calculation – MTF compensation files – Applying the compensation – Sensor MTF compensation for measuring lens MTFChart MTF measurements

Chart Quality Calculator – determine the suitability of a chart (based on MTF measurements for specific media & print methods) for a specific application
Test chart suitability for MTF measurements – charts for determining the MTF measurement suitability of several media types (inkjet and photographic; reflective and transmissive). Based on Chart Quality Calculator results.
Chart Quality Index (CQI) – older (now deprecated) method that relates chart quality and size to MTF measurements. Replaced by Chart Quality Calculator. The Chart Finder is an interactive website for selecting chart size and media.
MTF curves and Image appearanceCorrelating measurement with appearance
Acutance and SQF (Subjective Quality Factor): perceptual sharpness measurement
 

Other Image Quality Factors

Gamma, Tonal Response, and related concepts – why it’s used, how to measure it, and much more
Dynamic Range – a general introduction with links to Imatest modules that calculate it.
Chromatic Aberration – (Lateral Chromatic Aberration) AKA Color fringing
 
A recent white paper (May 2023, corresponding to Imatest 23.1), Measuring Information Capacity with Imatest, describes  information capacity and related measurements (NPS, NEQ, SNRi, and more) from widely-used slanted edges. 

Three November 2023 white papers (too many?) contain the latest information capacity measurements and results, including Edge SNRi and filter design. Many of the features are in Imatest 23.2. Until Imatest 24.1 is released in spring, 2024, they are available in the Imatest 24.1 Pilot Program.

Introduction to Image Information Metrics is the most concise introduction to image information capacity and related metrics, with a minimum number of equations and minimal technical detail. We may shrink it further for use in a brochure for marketing.

Image Information Metrics in Imatest is a concise and readable introduction to image information capacity and related object and edge detection metrics with moderate technical detail.

Image Information Metrics and Applications: Reference (41 pages) has all the equations and technical detail, but it may be difficult to read. It’s the reference for the other documents.

Information capacity news and links are on Solutions – Information capacity.
The Siemens star method, introduced in 2020, is slower but better for observing the effects of image processing artifacts (demosaicing, data compression, etc.).
 
Shannon information capacity from Siemens stars (2020) – a figure of merit that combines sharpness and noise
Information capacity measurements from Slanted edges: Equations and Algorithms (2023) – a figure of merit that combines sharpness and noise, conveniently measured from any slanted-edge. Also includes NPS, NEQ, and SNRi. Available in 23.1.
Information capacity measurements from Slanted edges: Instructions (2023) – instructions on the new calculations. Available in 23.1.
 

Image quality utilities let you degrade, enhance, examine, or analyze any image (not just test charts)

Image Processing Simulate several image processing operations (degradations: blur and noise; enhancements: sharpen, bilateral filter, tone mapping, etc.) and observe their effects on appearance and measurements (MTF, SSIM, etc.).
SSIM: Structural Similarity Index Measure image quality degradation and artifacts from signal processing such as compression. Also, measure PSNR.

Image Statistics – Interactively observe image statistics: cross sections, means, noise, SNR, histograms, and frequency spectra.
Radial Geometry – Add or correct distortion, lateral chromatic aberration, or rotation to images.

Getting Started with Image Quality Testing

Why Imatest?   |   Test Charts   |   Lighting   |   Image Capture Technique   |   Setting up Your Lab   |   Using Imatest Software 
Sample images for several modules from Github

 

Start Using Imatest Software

Installation – and getting started
Using Imatest – Getting started – Introduction and general instructions  Recommended for beginners.
Activation/Deactivation – Activating Imatest and moving it from one computer to another
Knowledge base — contains several links to help solve activation issues.
 
Please note:  Each Imatest license can be activated on only one computer at a time, but the activation can be easily moved between computers, especially if they’re online. You can install Imatest– and keep it installed– on as many computers as you like (Macintosh as well as Windows– your license works for both; you never have to uninstall it). To move the activation to another computer, you’ll need to deactivate it on the old computer then activate it on the new one, following the instructions above. You may do this as often as you like.

Imatest Instructions – general

Session LoggingLog level controls for display and log files
Target Detection SettingsSettings for guiding automatic detection of targets if default detection fails.

Skype video specification support Instructions and suggestions
Building a test lab – How to build a testing lab
Saving and retrieving Regions of Interest (ROIs) –  How to save and retrieve ROIs in named files for future use.

Saved settings – Imatest-v2.ini and INI files for use with Imatest IT

Pass/Fail Monitor – Realtime display and update of pass/fail results (useful with Imatest IT)

Acquiring images Files, formats, devices, and utilities

Autobatch – Combine several test images (from different modules) for a single device into an automatic batch run.

The Imatest Functional Interface – A simplified interface for running tests with consistent settings

The Functional InterfaceThe EditorPreparing a merge file with the INI editor  (may be deprecated in a future release)
Raw files Imatest modules can analyze raw files directly or after demosaicing.
 

Knowledge Base highlights Links to articles for troubleshooting, activation, and general advice on operation.

There are many more links like these on the full Knowledge Base.
 
Troubleshooting – What to do when Imatest doesn’t work

Slanted-Edge sharpness modules

SFRBasic analysis of slanted-edges (manual ROI selection)

Using SFR Part 1 – Setting up and photographing SFR targets
Using SFR Part 2 – Running Imatest SFR
SFR results: – Chromatic Aberration, Noise, and Shannon capacity plot

SFR results: Auto Focus (AF) plot

Table of slanted-edge MTF modules with automatic region detection  (Rescharts or batch-capable fixed modules)
All support ISO-compliant MTF (sharpness) analysis, Lateral Chromatic Aberration (LCA), and information capacity for chart contrast ratios between 2:1 and 10:1, with the ISO-standard 4:1 preferred.
Feature SFRplus
eSFR ISO
eSFRiso_enhanced_200W
SFRreg
Optikos_Meridian_200W
Checkerboard
checkerboard_ideal_framing-200W
Sharpness map detail   High   Medium   Depends on arrangement   High
ISO-standard chart design
Color analysis   (only for SFRreg center chart)
Tonal response (OECF)   (only for SFRreg center chart)
Noise analysis   Limited   Limited   Limited
Distortion ♦ Good   Limited Best
Geometry (rotation,
FoV, keystone, etc.)
Features and
recommended uses
Imatest’s original automatically-detected chart, in use since 2009. Robust and versatile. Some white space recommended above and below top and bottom bars. More spatial and distortion detail than eSFR ISO.
ISO-standard chart design. Includes wedge analysis and (optional) color patches. Supports detailed noise analysis.

Several individual charts are typically placed around the image field. For
– extreme fisheye lenses (>180º)
target projection systems
– Charts at different distances to test focus & depth of field.
– extreme high resolution cameras (>36MP)

Relatively insensitive to framing: can zoom in or out as long as distance is large enough so chart quality is not an issue and there are detectable corners. Well-suited for through-focus measurements. Very accurate distortion measurements.

♦ denotes strong support– denotes no support.

SFRplusAutomated analysis of slanted-edges

Using SFRplus Part 1 – The SFRplus chart: features and how to photograph it
Using SFRplus Part 2 – Running SFRplus
sfrplus_predistort_200WPre-distorted and special charts for Fisheye Lenses – with many applications such as automotive rear view cameras
              Includes SFRplus, eSFR ISO, and SFRreg charts.  Previewing pre-distortion


eSFRiso_enhanced_200WeSFR ISO – Automated analysis of the ISO 12233:2014 Edge SFR chart

Using eSFR ISO Part 1 – The ISO 12233:2014 E-SFR chart: features and how to photograph it


SFRreg – Automated analysis of registration mark patterns

  • Optikos_Meridian_200WTest at infinity focus with Long Range projection systems
  • Test imaging systems with ultrawide fisheye lenses (>180 degrees)
  • Test at very long camera-to-chart distances
  • Test at variable distances for Depth of Field.
Using SFRreg Part 1 – Registration mark patterns and how to photograph them
 

Checkerboard – Automated analysis of checkerboard patterns

checkerboard_ideal_framing-200WUsing Checkerboard Part 1 – Checkerboard patterns and how to photograph them
Using Checkerboard Part 2 – Running Checkerboard

Other (Rescharts) sharpness modulesLog F-Contrast, Star, Random, Wedge

Using Rescharts – Analysis of resolution-related charts
Log Frequency – Analysis of log frequency-varying charts
Log F-Contrast – Analysis of charts that vary in log frequency and contrast
Star Chart – Analysis of the Siemens star chart
Random/Dead Leaves – Scale-invariant test charts (including Imatest Spilled Coins)
           for measuring texture sharpness
Texture examples – More details of Imatest Spilled Coins texture sharpness measurements
Dead Leaves measurement issue Illustrates challenges posed by extreme signal processing
Random/Dead Leaves cross method  – Also called the Spilled Coins cross-correlation method.
Wedge Analysis – Analysis of hyperbolic or trapezoidal wedge patterns with the Wedge and eSFR ISO modules

Sharpness utilities & postprocessors

FocusField – Use batch results from a range of distances (or apertures) to measure Curvature of Field,
          Longitudinal (Axial) Chromatic Aberration, Depth of Field, and lens focal length.
Image Stabilization/Sharpness Compare – SFRplus postprocessor for analyzing (Optical) Image Stabilization
          and comparing sharpness of different images
Batchview – Postprocessor for viewing summaries of SFR, SFRplus, eSFR ISO, or Checkerboard results
Find Sharpest Files – Find the sharpest files in a batch
MTF Compare – Compare MTFs of different cameras and lenses (now used infrequently)

Tone, color, and spatial modules

Tone, Color, Noise, and Dynamic Range

New Imatest Dynamic Range film chart with Dmax = base+3.4Using Color/Tone Interactive (Multicharts)Interactive analysis of color & grayscale test charts
Using Color/Tone Auto (Multitest)Fixed (batch-capable) analysis of color & grayscale test charts
Color/Tone/eSFR ISO Noiseincluding chroma, sensor (raw), and visual noise
Contrast Detection Probability (CDP)  – Based on IEEE p2020 automotive image quality standard
Color/Tone Special ChartsAdditional charts, including ISO OECF and noise chats and circles arranged on a square
Color Correction Matrix (CCM) Calculate a matrix (usually 3×3) for correcting image colors (often from raw images)
Nonuniformity Correction in grayscale and color chart modules for Color/Tone Auto or Setup, Stepchart, and Colorcheck
Dynamic Range a general introduction with links to Imatest modules that calculate it.

Contrast ResolutionA special transmissive chart for measuring the visibility of low contrast features in larger fields over a wide dynamic range. Analyzed with Color/Tone Interactive or Auto.
GamutvisionA module for exploring ICC profiles and gamut mapping — illustrates how colors change when images are moved between files and devices. Will be in the Imatest 23.1 Alpha pilot program, available in October 2022. 
Colorcheck/Color/Tone AppendixColor difference algorithms and reference formulas
Dynamic Range (postprocessor)Calculate Dynamic Range from several Color/Tone Auto or Stepchart images
Measuring test chart patcheswith a Spectrophotometer and one of two software packages

Spatial and Uniformity (Flat field) modules

Light Falloff contour plotUsing Flatfield, Part 1Measure lens vignetting and image nonuniformity
Using Flatfield, Part 2 Features in Imatest Master and IS (not in Studio)
Using Flatfield-InteractiveInteractive measurement of vignetting and sensor nonuniformity (can be used with direct image acquisition)
Using Flatfield Blemish DetectMeasure visible sensor defects
Flatfield statistics – Measure PRNU & DSNU with methods based on EMVA-1288: Separate fixed-pattern (spatial) noise from temporal noise by processing multiple identical images.
Lightbox Comparison GuideComparing lightbox brightness and uniformity
Testing flat screen displayswith Imatest Blemish Detect
Using Distortion  (legacy module: not recommended for new work. Often fails with strongly distorted images. Checkerboard is now Imatest’s recommended module for measuring distortion: it is accurate, robust, and works well with strongly barrel-distorted (“fisheye”) lenses.
Dot PatternAnalysis of a grid of circular dots
 

Miscellaneous modules and utilities

Lighting ControlSet up and control DMX lights (Kino Flo) and Lightboxes; monitor lighting conditions with an Isolight puck.
Arbitrary Charts – Analyze a chart layout you have designed yourself. 
Device ManagerConnect to cameras, adjust settings and acquire images into interactive modules.
Video: Dynamic measurements These can also be performed with direct image acquisition.

Test ChartsCreate test charts for high quality inkjet printers
SVG Test ChartsScalable Vector Graphics charts for MTF and other measurements
Using Screen PatternsMonitor patterns for Light Falloff, SFR, Distortion, and monitor calibration
Rename Files – using EXIF data
Image Processing Simulate several image processing operations (degradations: blur and noise; enhancements: sharpen, bilateral filter, tone mapping, etc.) and observe their effects on appearance and measurements (MTF, SSIM, etc.).
Image Statistics – Interactively observe image statistics: cross sections, means, noise, SNR, histograms, and frequency spectra.

Database – Collect and store important information on image capture conditions, device state, and EXIF metadata
Composite ChartCreate a synthetic test chart from several individual image files. Can be used for Photon Transfer Curves.
Educational Apps – Learning resource for image quality factors and measurements.

 

Imatest IT – Industrial Testing (non-GUI)

Imatest IT-EXE instructions – Running Imatest IT (Industrial Testing)-EXE
Imatest IT-C instructions – Running Imatest IT (Industrial Testing)-C
Imatest IT-C++ instructions – Running Imatest IT (Industrial Testing)-C++
Imatest IT-Python instructions – Running Imatest IT (Industrial Testing)-Python
Imatest IT-.NET (C#) instructions – Running Imatest IT (Industrial Testing)-.NET (C#)
Imatest IT-.NET (Visual Basic) instructions – Running Imatest IT (Industrial Testing)-.NET (Visual Basic)
Imatest IT Parallel (IT-P) – Accelerating Imatest IT by performing tests in parallel
Imatest INI file reference – Reference to INI files for IT/EXE and IT/DLL users (sections used in several modules)
Direct Read Mode – Pass image data directly into Imatest IT, without writing to a file on disk
Implementing Pass/Fail – Setting up Pass/Fail criteria and Pass/Fail Reference. For Imatest IT and GUI versions.

Pass/Fail Monitor – Realtime display and update of pass/fail results (useful with Imatest IT)
Operator Console – User interface for production environments
Imatest IT Knowledge Base – Articles about Imatest IT