Imatest attended Electronic Imaging 2024.
Imatest exhibited our Benchtop Test Stand, new ISO 12233 chart, and webcam test setup.
- Keynote — Image information metrics from slanted edges: a toolkit of metrics to aid object recognition, machine vision, and artificial intelligence systems
- Evaluation of Signal and Noise Metrics of High Dynamic Range Image Sensors by IEEE P2020 Methodology
- Evaluating Camera Performance in Face-Present Scenes with Diverse Skin Tones
- Recommendations for the Detection and Analysis of the ISO 12233:2023 e-SFR Slanted Star
Recap Video
We have many of our talented imaging scientists who are presenting or contributing to the following papers:
Keynote — Image information metrics from slanted edges: a toolkit of metrics to aid object recognition, machine vision, and artificial intelligence systems
Author: Norman Koren
Download full text of paper | Download slides from talk (PDF)
Monday, January 22, 2024, 8:50am-9:50am
Part of Autonomous Vehicles & Machines (AVM) and Image Quality and Systems Performance (IQSP) Joint Session
There is increasing evidence that standard sharpness (MTF) and noise measurements correlate poorly with Machine Vision and Artificial Intelligence (MV/AI) system performance. This is not surprising because MV/AI algorithms operate on information rather than pixels.
We describe new techniques for measuring noise in the presence of slanted edge signals that enable the calculation of the key metric from information theory, information capacity, as well as several additional metrics.
We expect information capacity to be a strong predictor of MV/AI system performance, and because it is relatively unaffected by uniform image processing, it is the best metric for selecting (i.e., qualifying) cameras. Choosing a camera with the minimum number of pixels for the required information capacity should result in the fastest calculations and least power consumption.
The most important of the additional metrics, SNRi and Edge SNRi, measure the quality of object and edge detection, which can be enhanced by image processing. We show how to design filters that optimize object and edge detection, and we discuss the tradeoffs in applying them to real world scenarios. Finally, we discuss the mathematical framework that ties the new metrics together, resulting in a powerful and versatile toolkit of measurements.
The Image Information Metrics post has details on the new information metrics.
Evaluation of Signal and Noise Metrics of High Dynamic Range Image Sensors by IEEE P2020 Methodology
Authors: Orit Skorka, Norman Koren, Jackson Knappen, Paul Romanczyk, Jonathan Phillips, Radu Ispasoiu
Monday, January 22, 2024, 10:40am-11:10am
Part of Image Quality and Systems Performance (IQSP) Conference
High dynamic range (HDR) image sensors continue to evolve in range capability and ubiquitousness, particularly with the growth of cameras in automotive applications. The market desire is to keep increasing the dynamic range of image sensors, with boundaries being pushed beyond 140 decibels. A key image quality attribute for HDR image sensors is the quantification of noise, both temporal and fixed pattern. The increased, and increasing, range of HDR image sensors compared to standard dynamic range sensors in the market is a challenge to characterization of noise. Legacy procedures to quantify image sensor noise were not written with current HDR image sensors in mind, quantification is generally limited to portions of the operating range. The challenges for measuring both temporal and fixed pattern noise include optimizing lab capture methods to incorporate high dynamic range conditions, simplification of exposures needed, and comprehensive analysis. This paper will show applications of the IEEE P2020 noise standard with expanded protocols that account for verification of lab and chart conditions, and optimization of number of captures and number of pixels needed for analysis.
Evaluating Camera Performance in Face-Present Scenes with Diverse Skin Tones
Read summary article: Equity in Camera Technologies: How Consumer Cameras Perform Across Skin Tones
Authors: Megan Borek, Sarah Kerr, Amelia Spooner, Paul Romanczyk
Tuesday, January 23, 2024, 9:10am-9:30am
Part of Image Quality and Systems Performance (IQSP) Conference
Consumer cameras have become indispensable tools for communication, content creation, and remote work. Quality of digital images and videos varies significantly, influenced by lighting, hardware, scene content, face-detection, and automatic image processing algorithms. We investigate how camera systems perform in face-present scenes containing diverse skin tones, and how performance can be objectively measured using standard procedures and analyses. We closely examine image quality factors (IQFs) commonly impacted by scene content, emphasizing automatic white balance (AWB), automatic exposure (AE), and color reproduction. We assess the performance of several consumer cameras (web, phone, DSLR) with different lighting conditions and skin tones, expanding on Valued Camera Experience (VCX) standard procedures. Findings indicate that many automatic algorithms cannot accurately depict diverse skin tones. Color shifts, poor exposure, and reduced overall image fidelity are unfortunately common for scenes containing darker skin tones, revealing a major short-coming in modern day image processing. Moreover, we present a comprehensive evaluation of current standard procedures, highlighting strengths and limitations in measuring system performance for a range of skin tones. We propose novel targets and testing methodologies, paving the way for advancements in camera technology that cater to a diverse range of users and environments.
Recommendations for the Detection and Analysis of the ISO 12233:2023 e-SFR Slanted Star
Author: Sarah Kerr
Thursday, January 25, 2024, 9:40am-10:00am
Part of Image Quality and Systems Performance (IQSP) Conference
The recently published ISO 12233:2023 has introduced a new edge-based spatial frequency response (e-SFR) feature to the standard known as the slanted star. The 4 additional edge orientations available on the slanted star compared to the previously used slanted square feature facilitates comparison between sagittal and tangential SFR with vertical and horizontal SFR across the field of view. However, the extended utility granted by the extra edges simultaneously complicates reliable automatic placement of appropriate regions of interest (ROIs) for e-SFR analysis, making it difficult to properly compare the resolution across these orientations. This paper provides recommendations for the efficient and accurate detection and analysis of the ISO 12233 slanted star feature, based on simulated and experimentally validated results under a variety of challenging conditions.