|
Using Color/Tone Interactive – Interactive analysis of color & grayscale test charts Using Color/Tone Auto – Fixed (batch-capable) analysis of color & grayscale test charts Color/Tone & eSFR ISO noise measurements – describes how to measure image sensor noise from raw images Dynamic Range – a general introduction with links to Imatest modules that calculate it Noise in Photographic Images – a basic introduction Color Correction Matrix (CCM) Calculate a matrix (usually 3×3) for correcting image colors (often from RAW images) |
Input files – Opening ISPsim – Settings – Image processing blocks – Sharpening – Running – Processing –
Side-by-side display – Displays and analysis – Saving bayer raw (remosaiced) images –
Optical Character Recognition (OCR) – Face and People Detection
The Imatest ISP (Image Signal Processing) Simulator module, ISPsim, simulates ISP pipelines, and with some limitations, complete camera systems. It evolved from the Imatest Image Processing module, but is much more powerful and flexible, adding many processing blocks and allowing them to be performed in arbitrary order. ISPsim can greatly speed up the development of image processing pipelines.
ISP input includes
- Bitmap test charts produced by the Imatest Test Charts module,
- Bitmap test chart files that have been processed to include lens degradations and stray light by lens design programs, such as Code V,
- Raw (converted) images acquired from cameras with minimal processing (no sharpening, noise reduction, white balance, etc.). Highly processed images (most consumer camera JPEGs) are not recommended.
- Image sensor parameters, which can be measured from undemosaiced raw images in Color/Tone.
ISPsim processing includes
- Camera and lens image degradations, such as noise, fog (veiling glare), and blur,
- Image adjustments, such as exposure (and gain), gamma, and Beyer demosaicing/mosaicing,
- image enhancements, such as applying a Color Correction Matrix, tone mapping (used in High Dynamic Range (HDR) imaging), Sharpening (standard or Unsharp Mask (USM)), and bilateral filtering,
- ability to simulate motion blur, low light conditions, and more.
ISPsim results
- can be immediately observed in the ISPsim interactive window. You can view input and output images individually or side-by-side, as well as the quantitative effects of processing, including SSIM, PSNR, MTF, Optical Character Recognition (OCR), and Face or People Detection,
- can be exported to external programs, especially Imatest modules, to obtain all major image quality metrics, including image information metrics.
ISPsim includes an accurate image sensor noise model, based on just two measured parameters (well capacity at maximum signal and dark noise). No radiometric calculations are needed, though they can be useful for determining a camera’s absolute sensitivity. They may be added in the future.
ISPsim is expandable. Processing blocks will be added based on customer requests. The current set of blocks is primarily focused on operations that affect visible and measurable image quality.
ISPsim can operate on batches of images, automatically reading, processing and saving images in the batch.
ISPsim input files
ISP input can come from any image file, raw or demosaiced, that is readable by Imatest and listed in Image file formats and acquisition devices. Although ISPsim can work with any image, test chart images produce the most useful quantitative results. There are three major sources of input files.
Test Charts program — The most convenient source of test chart images is the Imatest Test Charts program. Here is an example, illustrating settings for creating a 2400×1600 pixel Enhanced eSFR ISO 2017 chart.
Test Charts window for the Enhanced eSFR ISO 2017 chart
Although we recommend reading the documentation for Test Charts and SVG Test charts, this page should contain enough information to get you started. You can also download the 2400×1600 PNG image (with added color patches) that was used for developing and testing ISPsim.
Test Charts lets you create SVG (Scalable Vector Graphics) or bitmap (TIFF or PNG) chart files that can be used for printing physical charts or for analyzing chart images. It contains a great many chart types, selectable in the Pattern dropdown menu (top-left) as well as a great many options. Chart patterns that have continuous tones (such as the sinusoidal Siemens Star) must be created as bitmap files, sized for printing or for Imatest analysis. SVG files can be opened by Inkscape (an excellent free program) for printing in any size or conversion to PNG bitmap files with bit depth = 8 or 16 (which can also be done for bit depth = 8 within Test Charts). eSFR ISO (ISO 12233-2017) charts, which are normally printed large, are created as SVG files, but can also be saved as 8 or 24-bit bitmap (PNG) files by checking the Save PNG (bitmap) file box, just below the chart image, and specifying the desired image height in pixels. For tonal response or dynamic range charts, we strongly recommend opening the SVG file in Inkscape and converting it to a 16 or 48-bit PNG or TIFF file.
Some Test Charts settings are hidden in dropdown menus. We recommend selecting eSFR LogWedge max = 4000 LW/PH in the Settings dropdown menu.
When your settings are complete, press Create Test Chart , and save the files (you’ll get both SVG and PNG) in a convenient location.
Lens design programs — such as Code V let you process chart images, degrading them (adding blur, chromatic aberration, vignetting) using a realistic lens model. The chart images can be created with the Imatest Target Generator, which is a back-end for Test Charts, converting SVG charts to bitmap files. Charts for input to lens design programs must have gamma = 1 (linear gamma). When a lens design program has been used, the (blur) filter blocks won’t be needed, unless you need to simulate an anti-aliasing (optical lowpass) filter.
Images acquired from cameras — Raw or minimally-processed images, converted to RGB with LibRaw or ReadRaw, work well. These include the actual images from development systems that need to be tuned. Blur and noise are already present in these images, so (blur) filter and noise blocks won’t be needed.
Most JPEG images from consumer cameras should be avoided because they are already heavily processed, including complex tonal response curves (with “shoulders”) and nonlinear and nonuniform bilateral filtering, making it difficult to obtain useful results.
Opening ISPsim
Press ISP Simulator in the the Utility dropdown menu or in the Utility tab on the right of the Imatest main window. This opens initial ISP Simulator window, which contains basic instructions and has many buttons grayed out (to minimize confusion). The instructions will be periodically updated. To perform an analysis, you will need to
- read an image file, and
- make sure you have the correct settings, which are saved from the previous run. You can update them from the ISP Settings window if needed.
The order of pressing Read image file and ISP Settings is irrelevant.
ISP Simulator opening window. Click on image to view full-sized. The text will be updated.
The settings window lets you set up the analysis: you can select processing blocks, and adjust their parameters and the order of operations. To open the settings window, press the ISP Settings button on the upper-left of the ISP Simulator window. Since ISP settings are saved from the previous run, you may not need to open it. Settings can be made before or after reading an image (but always before for running batches of images).
The Settings window contains a large number of processing blocks. Several, like Sharpen or Bilateral Filter, have several associated parameters and are contained inside rectangular frames. Others, like Exposure or Gain, which have at most one associated parameter, are not displayed inside frames. Their order in the window is not related to the order of operations to be selected.
The window is divided into four regions: 1. Input processing & degradations, 2. Color and tone adjustments, 3. Sharpening/noise reduction, and 4. Image file save settings.
ISP settings screen
The Image Signal Processing (ISP) Blocks window, on the upper-right, contains the sequence of operations. Blank lines are intended to make it easy to insert, arrange, and delete entries, and are ignored when the settings are transferred to the main window or processed.
Selecting and ordering image processing blocks
| 1. Choose a block to include in the image processing pipeline. | |
| 2. Adjust parameters — sliders, popup menus, or edit boxes, as needed. The corresponding Set window will become White with a red background , and remain that way until the setting is copied to the ISP Blocks box or canceled. | |
| 3. Use the Set popup menu button next to each block to enter it into the Image Processing Blocks list box on the right. All the Set popups have the same contents: Set, 01, 02, 03, …, 20. Click on the Set popup, then click on the line number where you want the block to be entered. Choosing Set (instead of a line number) cancels the operation. | |
| 4. Since the block parameters are copied to the list box, you can reuse blocks by changing their settings and entering them on a different line. | |
| 5. The tint of the Set box indicates the status of the block. White with a green background indicates that the block has included in the Image Processing Blocks list box. White with a red background indicates that the block has been adjusted, but not (yet) selected. Press Set then the line number to select it. Press Set then Set (instead of a line number) to reset the box to Black/white . In the above window, both the Add and Correct cos4 lens shading popups are white/red because they are both affected by the FL/diag (Focal Length/Sensor diagonal) setting; they will be both reset to Black/White when one has been selected. | |
6. The Image Processing (ISP) Blocks list box contains many blank lines. They are there to make it easy to add and edit processing blocks. They are ignored when the calculations are performed. The order of the processing blocks can be conveniently edited using the buttons below the box, which include
|
|
| 8. Pressing OK saves the settings, closes the Settings window, and returns control to the main ISPsim window. The selected image processing blocks (with blank lines removed) appear in the ISPsim window. | |
| SUMMARY: Settings are made in two steps: (1) Adjust the parameter, and (2) move it into the appropriate line in the Image Processing blocks box, which can be edited as needed. Since it’s easy to forget the second step, the Set window is changed to White with a red background when the corresponding parameter has been adjusted, but not yet selected. |
For example, Filter (on the bottom-left of the settings window) was copied to line 03 by selecting the Set dropdown on the left of the Filter frame, then selecting 03 in the menu (which contains 1-20). In most cases, parameters are copied along with the lock name. Note that there is space in the Settings window for additional blocks. Several are planned, and we are always interested in customer feedback and requests.
Image processing blocks
As we indicated, updating the calculations applies image processing blocks in the selected sequence. Depending on the input image source, some of the degradation or enhancement steps may have been performed prior to entering the image into ISPsim. For example, JPEGs from consumer cameras typically have nonlinear, nonuniform bilateral filters (we don’t recommend them for ISPsim). Noise and filtering (blurring) have already been implicitly applied to images acquired from cameras (i.e., you don’t have to add noise and blur because they are present in the image).
Here is a list of the available processing functions, as of January 2025). Most are standard Matlab functions, from the MATLAB File Exchange, or custom Imatest code. Notes: (L) recommended only for linear images.
| Processing block | Description | Function: described in link |
||||
| Input gamma | Not actually a processing block, Input gamma is required for keeping track of the image gamma at each state of the processing — important because some operations require the image to be linear — set to gamma = 1 — (indicated by (L). There will almost certainly be errors if it is entered incorrectly. Input gamma is typically 1 for (linear) raw images or around 0.45 (0.5 is close enough) for interchangeable color space images (sRGB, Adobe RGB, etc.) | Gamma, Tonal Response Curves, and related concepts | ||||
| Image degradations and initial operations | ||||||
|
Exposure Gain = 1/Exposure Gain (set by slider) |
Exposure/gain multiplier (gamma = 1 (linear) recommended) Exposure (a multiplier set by the slider; usually < 1) is typically applied to simulated images before noise is added. It represents an actual change in exposure. Gain = 1/Exposure is typically applied after noise has been added. The effective Exposure Index (AKA “ISO speed”) can be adjusted by combining Exposure and Gain. Multiplying the image by Exposure (typically <1) before applying noise, then multiplying by Gain = 1/Exposure afterwards is equivalent to multiplying EI by 1/Exposure. This decreases the Signal-to-Noise Ratio. For example, for a base EI of 100, applying Exposure = 0.25 before the noise and 1/Exposure = 4 afterwards is equivalent to increasing the EI by 4× to 400. Gain (multiplier) can be applied at any point in the pipeline. |
(L) |
||||
|
Bayer mosaic Demosaic |
Demosaic converts a mosaiced Bayer raw file from an image sensor with a Bayer Color Filter Array (CFA; which must be monochrome) into a 3-color RGB file using the MATLAB demosaic function. Demosaicing is a complex process that can generate artifacts and affects the Noise Power Spectrum. Bayer raw files can also be demosaiced by the Imatest LibRaw and ReadRaw modules. Bayer mosaic takes converts a 3-color RGB image into a monochrome, mosaiced pseudo-raw image, good for adding noise to the image. The image needs to be demosaiced to be converted back into an RGB image (which isn’t always needed). It can be useful for studying the effects of demosaicing when ISPsim starts with simulated (RGB) images. It uses a simple, straightforward algorithm (with no artifacts) to convert RGB images into one of the four Bayer alignments: {RGGB, GRBG, GBRG, or BGGR}. |
|||||
|
RGB Balance |
Adds or corrects RGB imbalance, which is found in raw images, where the Green channel is typically larger than the Red or Blue channels. R, G, or B can be entered in separate boxes. Values can be found in Color/Tone. Multiply to add the imablance, which is appropriate before noise is added, or divide to correct it, which is appropriate later in the pipeline. |
|
||||
|
Fog (flare or veiling glare) |
Fog the image: equivalent to adding flare light or veiling glare. Negative values of fog can be used to correct for black level offset, if present. |
|
||||
| Monochrome | Convert color images (m×n×3) into monochorme (m×n) using either a luminance channel equation (typically 0.2125×R + 0.7154×G + 0.0721×B) or the mean of the three channels. |
|
||||
| AWB (Auto White Balance) | Perform Auto White Balance using the simple “gray world” algorithm, which is adequate for most test chart images, but not for real-world images, especially if they have a dominant color. |
|
||||
|
Invert |
Invert the image, i.e., make a positive image negative. |
|
||||
| Linearize | (equivalent to Gamma target 1, which is recommended) Linearize the image (remove the gamma encoding). Images can be linearized, operations can be applied, then a final encoding gamma can be reapplied near the end of the ISP pipeline. Gamma = 1 is recommended for most processing blocks, such as standard sharpening (the 2D version of the 1D sharpening described on the Sharpening page. Not recommended for USM sharpening. | |||||
| Gamma | Apply gamma encoding to an image. Gamma can be selected by a slider (shown above) and a popup menu that contains Target and Mult. Important because the input and output may be gamma-encoded (often with gamma ≅ 0.5), but many operations may need to be performed with gamma = 1. For Mult, imageout = imageinputγ. For Target, where the current value of gamma is γ_current , imageout = imageinput(γ/γ_current). |
Gamma, Tonal Response Curves, and related concepts | ||||
|
Add, correct cos4 lens shading |
Also called lens “vignetting” (gamma = 1 (linear) recommended) Adding lens shading can be done for simulated images early in the ISP pipeline, before noise has been added. It doesn’t need to be added for real camera images. Lens shading can be corrected later in the pipeline, while gamma is still 1. |
|||||
| Negative | Invert the image, i.e., turn it into a negative. Infrequently used. Best performed on linear images (gamma = 1 (linear) recommended). | |||||
| Image degradations: blur, noise, vignetting Often found in images acquired from cameras prior to processing. Operations are generally linear. |
||||||
| Noise (Gaussian) |
For Luma fraction = 0, the noise in the RGB channels are uncorrelated. For Luma fraction = 1, the noise is correlated. (We are still working out details.) If the noise is applied to a raw image (after Bayer mosaic has been applied), the setting doesn’t matter because the raw image is monochrome. This results in the best estimate or luma and chroma noise in the demosaiced image. If the Use sensor noise box is unchecked, a simple noise is applied, indicated by sigma (standard deviation: the upper Edit box). (The lower edit box is not displayed.) If Use sensor noise is checked, a full model of image sensor noise is applied, with the photon shot noise in the upper Edit box and dark noise in the lower Edit box. Image sensor noise is summarized below and described in great detail in Color/Tone & eSFR ISO noise measurements. |
imnoise: Add noise to the HSV V channel. Parameters sigma [] and Luma noise []. | ||||
| Filter | Filter (blur) the image. Best before applying noise, but can be applied after noise has been added (it will affect the noise spectrum). There are four types of noise. The four types of noise can simulate lens blurring, motion blur, misfocus, or diffraction. Affects the noise spectrum if applied after noise.
|
imgaussfilt: 2-D Gaussian filtering of images with parameter sigma, fspecial: creates a 2D kernel, filter2, imfilter |
||||
| Add or correct cos4 lens shading | Add or correct vignetting (light falloff) using the standard cos4 angular equation. The lens focal length/diagonal distance must be entered using the slider. | |||||
| Tonal enhancement: Correct colors or apply one of three tone mapping algorithms, used for displaying High Dynamic Range (HDR) images. Tonal response may be highly nonlinear. |
||||||
| Color Correction Matrix (CCM) |  Apply a Color Correction Matrix (CCM), which can be calculated and saved in Color/Tone. using the processed image from ISPsim at the stage where the CCM is to be applied (usually near the end of the ISP pipeline). The Set CCM button to the right of the CCM checkbox opens a window (shown on the right; click on thumbnail to view full-sized) that lets you set up the matrix (read it from a file or paste it from the clipboard and select options (linearization, etc.)). |
Color Correction Matrix describes the CCM, which is calculated in Color/Tone. | ||||
| Tone mapping |
Render high dynamic range (HDR) image for viewing in displays with limited dynamic range. Tone mapping is a technique used to approximate the appearance of high dynamic range images on displays with a more limited dynamic range. Nonuniform and nonlinear |
tonemap: Render high dynamic range image for viewing | ||||
| Local tone mapping | Alternative tone mapping algorithm for HDR images using Laplacian filtering. Has a different set of adjustments. Produces different results (more saturated images). Nonuniform and nonlinear | localtonemap | ||||
| Contrast-Limited Histogram Equalization | CLAHE operates on small regions in the image, called tiles, rather than the entire image. Each tile’s contrast is enhanced, so that the histogram of the output region approximately matches the histogram specified by the ‘Distribution’ parameter. The neighboring tiles are then combined using bilinear interpolation to eliminate artificially induced boundaries. The contrast, especially in homogeneous areas, can be limited to avoid amplifying any noise that might be present in the image. Nonuniform and nonlinear | adapthisteq: Contrast-limited adaptive histogram equalization (CLAHE) |
||||
| Spatial enhancement: sharpening and/or noise reduction. Output depends on image content— may be highly nonlinear. |
||||||
| Sharpen USM (Unsharp Mask) |
Sharpens the grayscale or RGB input image using the unsharp masking (USM) method. Most visible on edges. USM works by subtracting a gaussian blur function from the original image. If threshold > 0, low-contrast regions may not be sharpened. See Wikipedia. This function is not linear. It caused some inconsistencies with the image information metrics. Designed to work directly on images, which should not be linearized. Nonlinear (a surprise; not indicated in MATLAB documentation) |
imsharpen: Unsharp Masking (USM) | ||||
| Sharpen Standard |
Sharpens the grayscale or RGB input image using a 2D version of the 1D sharpening method described in the Sharpening page. Most visible on edges. It works by subtracting an amplitude-reduced replica of the original image, arranged on a circular locus with radius R, from the original image. To maintain linearity, Linearize should be checked. This causes the image to be linearized (gamma-encoding removed) at the start of the calculations. Then sharpening is applied to the linearized image, then gamma encoding is reapplied. Recommended for testing the effects of sharpening on image information metrics, such as SNRi or Edge SNRi. The 2D sharpening kernel is copied to the clipboard when this operation is performed. Because standard sharpening is not a MATLAB function, we include more explanation in the box below. |
Custom Imatest function (L) Either USM or Standard sharpen can be applied (but it doesn’t make sense to apply both). |
||||
| Bilateral filter 1 imbilatfilt (MATLAB) |
Bilateral filtering was proposed by Tomasi and Manduchi in 1998 as a non-iterative method for edge-preserving smoothing.
|
imbilatfilt (MATLAB) |
||||
| Bilateral Filter 2 bfilt2 (Lanman) |
Slow. (The Mathworks File Exchange has other bilateral filters, some of which may be faster but didn’t work reliably.) Parameters:
|
bfilter2 (from the Mathworks File Exchange): |
||||
| Image file save settings: applied only when an image is saved in a file. | ||||||
| Save file precision |
Settings
|
|||||
| Offset (DN) | Black Level Offset in pixels (Digital numbers) to be added to the saved file. | |||||
| Saturation (DN) | Saturation Level (maximum pixel or DN level) for the saved file. Used to represent m bit data in an n bit container (e.g., 4095 for m = 12-bit data, which could be stored in an n = 16-bit file, where the maximum DN is 65535. | |||||
|
Sharpening (standard; not USM) is an Imatest (not a MATLAB) function, designed to maintain linearity. As we mentioned above, this is a 2D version of the 1D sharpening method described in the Sharpening page. It was created when we were trying to determine if USM, which is implemented with the MATLAB imfilter function, is nonlinear (it is). It is also closer to typical camera sharpening than USM, which is more computationally intensive. For sharpening radius R and amount A, define ksharp = A/(A+1). The filter kernel is a 2R+1×2R+1 matrix H (units of pixels), calculated by (1) setting each entry at a distance r from the center to H(r) = max(|R-r|,0), (2) setting H = H/∑H, then setting the entry at the center, H(R+1,R+1) = 1. For a filter with Radius = 2, Amount = 3 (strongly sharpened), the 5×5 kernel is -0.0092 -0.0409 -0.0535 -0.0409 -0.0092 Standard sharping should be used with Linearize γ for gamma-encoded images. It is a linear operation (as long as excessive sharpening doesn’t saturate the image). |
Image sensor noise summaryThis section summarizes how to measure image sensor noise for entering into the ISPsim Noise block, described above. The full details are in Color/Tone/eSFR ISO Noise. For linear (non-HDR) image sensors, noise can be characterized by an equation derived from the Digital number (pixel level) of the signal + noise at the image sensor. DN = DNmeas – DNoff where DNoff is an offset frequently included in undemosaiced images. Estimating DNoff , which is frequently present in undemosaiced raw files, requires some care. For dark noise power, Nd2 Expected Noise Voltage = VN = (Nd2 + kN DN)1/2 = (Nd2 + kN (DNmeas – DNoff ))1/2 — or — Expected Noise Power = PN = VN2 = Nd2 + kN (DNmeas – DNoff ) for linear sensors Nd2 and kN are the key parameters for the image sensor noise model in ISPsim. Now for the steps.
Once again, the full details of the calculation for finding dark noise and photon shot noise are in Color/Tone/eSFR ISO Noise. |
Running ISPsim
Read an input image file if it hasn’t been read in already. It doesn’t have to be a test chart; any pictorial image can be used. It doesn’t matter whether you read the image or update the settings first. Most types of image can be used, though as we indicated above, we don’t recommend highly processed JPEGs from consumer cameras.
|
Before running a batch of images you should run a single image to be sure the settings are correct. Note that the batch calculations are not optimized for speed, so batch processing, which typically takes several seconds per image, is not suited for high volumes of images. |
Update the settings, as described above. It doesn’t matter whether this is done before or after reading the image. The input image and the processing blocks chosen in the settings window will appear.
ISPsim input image
Process the image using Update calculations or another of of the buttons in the Calculation group (light blue box) on the left of the ISP simulator window. Here is a list of controls in the group.
| Update calculations | Starting with the input image, process each block shown in the List box. Display the results when processing is complete. This is the most common calculation. |
| Input | Restore the input image view. |
| Display results for 1s during updates (checkbox) | Display the results for 1 second after each block when running Update calculations. Step through calculations is generally preferred. |
| Step through calculations | Step through the calculations manually, pushing the button once for each block. Cumulative results are displayed. |
| Step & show single block | Step through the calculations manually, pushing the button once for each block. Results for the current block-only (not the cumulative results) are displayed after each block. |
After the image has been processed, the output image and the light green Analysis and view area below the image are displayed. Pressing View input or View processed just below the image (on the left) toggles between them. Because the two views are very similar, we show the more interesting side-by-side view, zoomed in to make the differences visible.
Side-by-side view
The side-by-side view allows you to compare the input and processed images side-by-side, i.e., right next to each other. Click the Side-by-side view checkbox just below View input or View processed (below the image, to the left). When the box is first checked the two images are displayed in their entirety. If you crop one, the other will have the identical crop. This makes for meaningful comparisons.
Here is the ISP Simulator display for the 36-patch dynamic range chart image (cropped and shown in the Side-by-side view, input on the left; processed on the right). The input image is straight out of Test Charts. The processed image had had blur, noise, and sharpening added.
ISPsim side-by-side view. Input image on left; processed on right. Zoomed in (for display-only)
What to do with the results
Several things can be done with the processed image.
Using Save processed image, it can be saved in a file with the precision selected in the Settings window, then read into Imatest for analysis. Image Information metrics, available for slanted edges and Siemens stars, are of particular interest for predicting the performance of Machine Vision/Artificial Intelligence systems.
It can be copied to the clipboard, then pasted into Imatest analysis modules. While this is faster than saving the files, it’s not recommended because the imclipboard function only supports 8-bit file depth. 16 bits is especially recommended for tonal or dynamic range measurements.
It can be directly opened in Image Statistics for several types of analysis, including cross-sections, histograms, and Fourier transforms.
You can try out different processing settings to see what effect they have on system performance.
Controls at the bottom of the ISPsim window
The Analysis and view (light green) box will be listed separately. Controls are from left-to-right in column-major order. The View image/processed and Side-by-side view buttons have already been described.
| Save processed image | Save the processed file with the bit depth specified in the light green box at the lower-right of the Settings window. Frequently used. |
| Image statistics | Open the image in the Image Statistics module, which can perform cross-section, histogram, and Fourier Transform analysis — and more. |
| Help | Open the online Help window. |
| also save settings (checkbox) | Save settings along with the processed image file. |
| Read image file (2) | Read a second image file (into im_processed) for comparison with the original file (which remains in im_input). |
| Proc img > clipbd | Copy the processed image to the clipboard. Unfortunately limited to bit depth. |
| Zoom out | Zoom the image out (display the entire image). This can be helpful when the side-by-side display has framing issues. |
| Zoom to crop | Ths refers to the Crop function in the light green Analysis and View box, described below, which is used for the MTF and SSIM measurements. This button lets you zoom to the crop. You can also use standard click-and-drag to zoom into the image for display-only without affecting the MTF calculation. |
| Processed > input | Move the processed image to the input so that analysis start with the new (formerly processed) image. |
| Save screen | View and/or save the screen. An image viewer must be specified. |
| Save settings | Save the current settings in the standard imatest-v2.ini file. Settings can be saved in a named file from the Settings window. |
| Exit | Bye! |
Utilities in the dropdown menus
Several of the utilities in the dropdown menus at the top of the ISP Simulator window are are duplicates of buttons in the window (intentional redundancy).
| File | Display and/or Save screen | Display the screen in an image viewer (Irfanview recommended) and optionally save it. This function is in several interactive modules. |
| Save image | Save the processed image. Normally images will be saved in full precision (except if 8-bit max (below) is checked). | |
| Save pseudo-raw/remosaiced image | Save the image in a pseudo-raw (remosaiced) or special format, including Bayer, RCCC, or specific channel can be selected. Many options. See Synthetic RAW images. | |
| Save image as monochrome | Save the image (which may be color) as a monochrome image. | |
| Save image as HDR | Save the image in HDR (High Dynamic Range) format | |
| Swap Endian (16, 32-bit input) | Swap the order of bytes within pixels (useful in rare instances where they are incorrect) | |
| Save image files with 8-bit max precision | Unchecked (off) by default. If set, images will be saved in 8-bit precision. (This should normally be unchecked). | |
| Save image as HDR | Save the image in HDR (High Dynamic Range) format | |
| View | Colorbar (grayscale or color) | Grayscale or color: Used in SSIM and PSNR displays. The colors in the color bar can be changed in Options II. |
| Crop | Lets you select a crop for image analysis operations (MTF, SSIM, etc.) Same as the Crop button in the Analysis and View area. | |
| MTF log axes | Toggle the MTF display between linear and logarithmic axes. | |
| Analysis | Most functions duplicate buttons in the Analysis and View section, but there are several additions. | |
| Image Statistics | Call the Image Statistics module for viewing image statistics such as histograms and cross-section profiles. | |
| Radial Geometry | Send the processed image to the Radial Geometry module, which can return a further processed image with added or corrected optical distortion, lateral chromatic aberration, and/or rotation. | |
| OCR | Optical Character Recognition using Matlab’s OCR function | |
| Detect faces | Uses the Matlab CAMShift routine, described below. | |
| Detect people | Uses the Matlab detectPeopleACF routine, described below. Less effective than Face detection. | |
| Help | Open web pages with descriptions of various ISP Simulation blocks. | |
Analysis and view controls (in the light green box near the bottom of the ISPsim window)
The display is controlled by buttons below the image or by the Analysis dropdown menu. You can choose between a visual display of the Input or Processed image or a calculated display (an analysis comparing the input and processed images).
| MTF | Display the MTF of the processed / input image as a function of spatial frequency. Sensitive to cropping. |
| MTFnn | Display a polar plot of MTF70-MTF10 (spatial frequencies where MTF is nn% of low frequency value) |
| MTFnnP | Display a polar plot of MTF70P-MTF10P (spatial frequencies where MTF is nn% of peak value) |
| SSIM | Display the Structural Similarity Index, which is discussed in detail on the page for the SSIM module. Better correlated to visual differences (degradations) than PSNR. |
| PSNR & DIff | Display the Peak Signal-to-Noise Ratio and the difference between images. |
| OCR | Optical Character Recognition results using Matlab’s OCR function |
| Crop | Crops the image for calculations as well as display. This brings up the standard Imatest coarse and fine crop windows. This approach affects calculations such as the MTF curve (which is image-dependent for several processing blocks) or the mean SSIM calculation (though the detailed SSIM plot is unaffected). Pressing the Crop button enables the Zoom out and Zoom to crop buttons on the lower-right of the window. To remove the crop, press the Crop button and select the entire image. |
| Detect faces | Face detection results using the Matlab CAMShift routine. |
| Detect people | People detection results using the Matlab detectPeopleACF routine, described below. Less effective than Face detection. |
The View input button on the left, just below the image, toggles between View input (when the processed image is selected) and View processed (when the input image is selected). This enables rapid visual comparison of the two images. The zoom setting is preserved when you switch images. The View input/processed button performs the same functions as the Input and Processed buttons in the Analysis and View area. The other buttons in this area display calculated results.
SSIM displays the Structural Similarity Index, which is discussed in detail on the page for the SSIM module. It’s primary use is in measuring the visible effects (i.e., degradation) of image compression from saving files (for example, JPEGs of varying quality) and from data transmission.
MTF (Modulation Transfer Function; equivalent to Spatial Frequency Response) calculates the transfer function between the Processed and Input images using a 2D Fourier transform similar to the calculation in the Random module, but with one important difference. SInce this calculation has no noise reduction. the MTF measurement is extremely sensitive to noise.
MTF for noisy Gaussian-filtered image after Unsharp Masking
|
The high frequency rolloff is the effect of Gaussian filter, which is applied after the noise has been added (and hence shapes the spectral power of the noise). You can see the MTF for Unsharp Mask (USM)-only (in this case with Radius = 1.5 and Amount = 2.5) by unchecking Noise (gaussian) and Gaussian filter 2 in the ISP Simulation area to the left of the image, then pressing Update calculations. Unsharp Masking has a different spectral response from the type of sharpening described in the Sharpening page: It’s response is not cyclical. It rises to an asymptotic level of Amount+1 and stays high. This is a good example of how the ISP Simulator can be used to explore the effects of processing blocks on both appearance and measurements. |
 MTF for USM (Unsharp Mask: Radius = 1.5; Amount = 2.5) |
Saving Bayer raw (remosaiced; undemosaiced) images
By clicking File, Save pseudo-raw (undemosaiced) image (after a file has been read), you can save the image in a synthetic Bayer RAW format as well as one of several other formats. The output can be a standard image file format or pure binary. A dialog box opens that allows a number of options.
Mosaicing or channel selection None (Normal color file output), Remosaic (Bayer format), RCCC (Red-Clear-Clear-Clear): R bright, RCCC (Red-Clear-Clear-Clear): R = C/3, Monochrome (Y = luminance), Monochrome (mean: equal weights), Red, Green, Blue. For remosaicing (creating Bayer RAW files), Remosaic (Bayer format) would be selected.
Red in RmCn lets you select one of the four Bayer configurations.
Gamma adjust lets you adjust the gamma of the output curve. Keep it at 1 for no change. Use 2 (or 2.2) to go from a typical color space gamma to linear (gamma = 1).
Save file format lets you select a standard image file format (PNG, JPG (90% quality), TIF (large), BMP, Binary 8-bit, or Binary 16-bit. The binary files can be read into Generalized Read Raw.
Green channel emphasis lets you emphasize the green channel so that the synthetic RAW image resembles actual raw images, which typically are strongest in green. Several Blue/Red levels are available.
When you select Save, a dialog box lets you select the location and file name. The default is root_file_name_png (with the selected extension).
Optical Character Recognition (OCR)
|
OCR has been added in Imatest 5.0, using Matlab’s OCR function, which is an implementation of Tesseract-OCR. To use it,
This opens the OCR window and performs a text recognition operation on the crop of the image. Results for the sample image on the right (mildly degraded with Noise (gaussian) = 0.002; Gaussian filter 2 = 0.7) are shown below. You can download this image and experiment with to see how various degradations and enhancements affect OCR performance. Be forewarned. Matlab’s OCR performance is not impressive. Text that appears to be quite readable does not do well, especially for font sizes under 12 points. Performance is comparable to several free or low-cost OCR programs such as gImageReader (a GUI front-end for Tesseract-ocr) and TopOCR. |
The left side sample image consists a set of characters (QWERTYUIOPASDFGHJKLZXCVBNM1234567890) of different sizes ranging from 20 to 7 points in normal, bold, and negative bold Calibri font. These are characters used in automotive license plates. The file (here) was created in Microsoft Word. The remainder of the image is a portion of the Enhanced eSFR ISO chart, with wedges and two high contrast squares. These can be used to correlate MTF measurements with OCR performance. |
Sample image: Click for a
OCR window, showing results for mildly-degraded crop of the sample image
OCR Settings
Character Subset Results can be improved by entering a character subset to limit the valid output characters (all characters are allowed if Character Subset is blank), then pressing CALCULATE OCR. For the sample image, the subset consists of Latin capital letters and numbers (typical of automobile license places). It makes only a small improvement.
QWERTYUIOPASDFGHJKLZXCVBNM1234567890
Standard OCR does not work very well with license plates. Several license plate detection routines are available in the Matlab File Exchange. We may add one or two if they work well and the license allows.
Display Text (shown) is the default. You can also select Confidence & words, which shows the Word Confidence and individual word. Here are a few lines from the display.
0.8545 QWERTYUIOPASDFGHJKLZXCVBNM1234567890
0.8361 QWERTYUIOPASDFGHJKLZXCVBNM123456789O
0.9018 18
0.7842 QWERTVU|OPASDF6HJKLZX(VBNM1234567890
0.8339 16
0.7468 QWEKTYUIOPASDFGHJKLZXCVBNM1234567B90
0.8254 14
0.7252 QWERYYUIOPASDFGI-1JKLZX(\/ENM1Z34567E9(!
0.7784 12
0.5777 QWERTYUWDDASDFGHJI(LZXCVBNM113A567E9u
0.6549 11
0.6276 uwEuHu\onAsur:.r<n<.zxn’aNM:23-4ss7s9u
0.7658 m
Copy to clipboard You can copy the current display into the clipboard for use in text editors or other applications.
Save results saves the current display.
Variable or fixed width font can be selected in the Settings dropdown menu.
Face and People detection
Face and people detection can be called from the Analysis dropdown menu. Both currently use the Matlab default settings. (We may add an options window in the future.) See
Face Detection and Tracking Using CAMShift and detectPeopleACF
Face detection works somewhat better than people detection. Here is an example, using an image shamelessly stolen from http://epaperpress.com/portfolio/columbia.html. The soggy author of Imatest is on the left. Click Update Calculations to remove the annotations (to reset the image).
Example of Face Detection (Imatest author on the left)
Face detection works pretty well much of the time, but seriously… How can you miss Paul and John?