Information Metrics for Performance and Optimization of Machine Vision Systems.

Norman Koren
Founder / CTO

We discuss the mathematical background and practical measurement techniques for a new set of image quality metrics that promise to be superior to MTF (sharpness) and noise as predictors of machine vision performance. The new metrics include Noise Equivalent Quanta (NEQ), Ideal Observer SNR (SNRi), and Edge location standard deviation (Edge σ). They can be measured from slanted edges as well as Siemens star and dead leaves (spilled coins) test chart images.¬ We cover

• the origin and significance of information capacity, and its use as a fundamental metric for qualifying cameras,
• the effect on measurements of common types of image processing, especially the bilateral filters found in most JPEG images,
• the distinction between total information capacity and information capacity for contrast-limited patterns (typically 4:1 or 60% Michelson contrast slanted edges), which can be measured for a wide range of exposures,
• obtaining the metrics for object detection (SNRi) and the object location (Edge σ),
• the effects of sharpening and lowpass (noise-reduction) filtering, showing how excessive sharpening degrades performance,
• design of matched filters to optimize SNRi and Edge σ for specific tasks, and tradeoffs for multiple tasks
• progress in correlating the new metrics with standard machine vison performance metrics, such as Mean Average Precision (mAP) and Intersection over Union (IoU), which are related to object detection and location.