Related web pages

Simatest ISP/Camera Simulator — Instructions and Reference

Using Color/Tone Interactive – Interactive analysis of color & grayscale test charts

Using Color/Tone Auto – Fixed (batch-capable) analysis of color & grayscale test charts

Color/Tone & eSFR ISO noise measurements – describes how to measure image sensor noise from raw images

Dynamic Range – a general introduction with links to Imatest modules that calculate it

Noise in Photographic Images – a basic introduction

Color Correction Matrix (CCM) Calculate a matrix (usually 3×3) for correcting image colors (often from RAW images)

 
 Introduction – Modeling image sensor noise – Measuring the noise – Simulating the noise 
Modeling and measuring image sharpness – Simulating image sharpness 

This page illustrates how Simatest can be used for modeling and measuring camera systems and, most importantly, Image Signal Processing (ISP), using a compact digital camera as an example. It’s designed to be much more readable than the full instructions and reference in ISP/Camera Simulator (Simatest).

Introduction

Simatest can model and measure

  1. the effects of Image Signal Processing (ISP) starting with either
    1. Raw camera image — the raw output of a digital camera, which includes lens blur and image sensor noise. Test chart images are typically used for this case, but any image can be used, or
    2. Simulated (test chart) image, which needs to be blurred blurred to match the MTF (SFR) of a lens (typically using a simple gaussian or Airy disk filter) and includes noise from an image sensor model (described below). 
  2. complete camera systems starting with an image (typically a test chart image) that has been degraded by a lens design program. This enables the “soft prototyping” of a camera before the hardware is specified and built. The image sensor noise (described below) is required. A complete ISP chain can be applied. 

The camera used for the examples is the Panasonic Lumix LX5, an older (2010) 10.1 megapixel compact digital camera with 2.14 μm pixel pitch and a high quality Leica-branded zoom lens. Its image processing is simple by 2025 standards — it has no Computational Imaging or Artificial Intelligence, and most importantly, it has raw output, which can be processed with or without demosaicing. We will show how to

  • Model the image sensor noise using a raw image of a grayscale test chart.
  • Model the approximate camera spatial domain response and MTF (SFR) using a simple gaussian or Airy disk filter.
  • Use the camera  model to show how camera performance varies as a function of Exposure Index (widely known as ISP speed).

 

Modeling image sensor noise

A key step — perhaps the most important step — in simulating digital camera performance is to model the image sensor noise, which consists of dark noise from several sources and photon (or electron) shot noise. The Simatest model is derived from measurements of raw images of grayscale test charts, preferably from transmissive test charts, which have much larger tonal ranges than reflection charts. 

Once an image sensor’s noise has been properly characterized, Simatest can be used to predict camera performance for a wide variety of camera configurations — lenses, image processing, etc.

Image sensor noise is also described in Simatest instructions and reference and in Color/Tone/eSFR ISO Noise measurements.

For linear (non-HDR) image sensors, noise can be characterized by an equation derived from the Digital number (pixel level DN ) of the signal + noise at the image sensor.

DN = DNmeas – DNoff  where DNoff  is an offset frequently included in undemosaiced image files.

Estimating DNoff , which is frequently present in undemosaiced raw files, requires some care. It can be estimated from EXIF metadata or from measurements the signal in the darkest regions of the test chart image. 

For dark noise power, Nd2 

Expected Noise Voltage = VN = (Nd2 + kN DN)1/2 = (Nd2 + kN (DNmeas – DNoff ))1/2   — or —    

Expected Noise Power    = PN = VN2 = Nd2 + kN (DNmeas – DNoff )    for linear sensors

This model is a variant of the noise model in the EMVA 1288 standard for linear sensors, section 2.4.

Measuring the image sensor Noise

Nd2 and kN are the key parameters for the image sensor noise model in Simatest. Measurements must be made carefully from raw (undemosaiced and unprocessed) images.

Now for the steps.

  1. Acquire an image of a grayscale test chart, preferably a transmissive chart designed for measuring dynamic range.
  2. Save the image in a raw format.
  3. Read the raw image into Color/Tone Interactive without demosaicing. For commercial raw files, the Bayer RAW 16-bit linear preset is recommended. (Color Tone Auto also works, but is less flexible.) The converted image is saved as a TIFF. Details are given in Color/Tone/eSFR ISO Noise.
  4. Since the undemosaiced image has a monochrome format, the Standard monochrome or Bayer RAW? box will appear. Select the appropriate Pixel color mapping (Red in R1C2 (GR/BG)) for the Lumix LX5. (If the image contains recognizable color, you can open it in Rawview to find the color mapping.) Then select (Analyze image as) Bayer RAW.
  5. Estimate the offset, DNoff , and saturation level, using methods detailed in Color/Tone/eSFR ISO Noise, and enter it into the Color/Tone settings window (part of which is shown on the right). The values can often be found in the EXIF metadata (in the raw image, but not the JPEG), but the names are different for each manufacturer. DNoff , is always less than the lowest patch level, but can be larger than the lowest individual pixel level. If too large a value is entered, the response will be cut off (DN will be 0) for the darker patches.  Plots 10-12, which show noise and SNR as a function of pixel level, are of greatest interest. (These plots are only meaningful for raw images.) 
    Fortunately, we have found that the model is relatively insensitive to the offset estimate. For the Lumix LX5, the offset is 128, and the saturation level is 4095 (both numbers from the EXIF data, shown here).
  6. Select Display 10. Noise analysis with one of the plots (10-12) that display the noise or SNR as a function of input pixel level. This runs a Levenberg-Marquardt optimizer that finds the best values of Nd2 and kN that best match the data. Plot 10 (noise vs. pixel (dB)) is shown below. (We didn’t use SNR plots 11 or 12 because they are affected by offset + stray light, which can be difficult to quantify. 

Measured and fit noise vs. input pixel level the LX5 (2.14 μm pixels)

The key results to enter into Simatest are shown on the lower-left:  Noise = 0.01143 and Dark noise = 0.00067. The well capacity of 7345 electrons (e-) is interesting to compare with the well capacity of 33882 pixels for the the EOS-6D, which has much larger 6.5 μm pixels, shown on the Simatest instructions page.

Another result of importance is RGB = 0.549 1.000 0.537, which represents the color of the middle gray patches, normalized to 1. Because of the incandescent light source and sensor sensitivity, the image (thumbnail shown on the left with no color correction) has a greenish tint.

More details of the noise calculations are in Color/Tone/eSFR ISO Noise. Dark noise is actually a combination of several signal-independent noise sources, described in OnSemi Application Note AND9189/D. Electronic (Johnson) noise has a strong temperature dependence. It is proportional to (4kT)1/2.

Plot 2 (S/N vs. input density) for the LX5, shown on the right, closely resembles the Photon Transfer Curve in the OnSemi App Note, where S/N ≅ Noise (electrons (e-)) and Well capacity = {S/N)2max = 7345. The spike in the S/N plot is apparently caused by noise, most likely a dirt speck on the chart. The flat areas on the left side — the dark signal and dark SNR — are affected by stray light and possible uncertainties in the offset setting.

We expect to add more detail to the noise model in future releases. We will also move the noise calculations so results are available in the CSV and JSON output. And (a sort of a holy grail of calculations), we expect to simulate multi-region HDR sensors. 

S/N vs. input density: Photon Transfer Curve

 

Simulating the image sensor noise

The measured parameters for photon shot noise noise, dark noise, and RGB balance can be entered into a Simatest model of the image.

  • Open Simatest and read in an ideal test chart image file created by the Imatest Test Charts program. Our examples use two charts.
  • Open the Simatest settings window, and enter the following settings to replicate the raw image we just measured. 
         Gamma Targ 1, Filter gaus 0.75, RGB mult 0.55 1 0.53, Bayer grbg, Noise 0.0114 1 1 0.00067. These settings are described in the large table in Simatest — instructions and reference. Click OK when the settings are complete.
  • Press Update calculations in the main window. The resulting image will be a Bayer raw image.
  • To get an image with the same offset and maximum value as the camera (128 and 4095 for the LX5), press Save processed image to save the image in a named file.
  • Open the image in Color/Tone Interactive, with the 36-patch Dynamic Range chart selected. Because the undemosaiced image file is monochrome, you will get the window shown below. Key settings are circled.

Monochrome or Bayer raw? window opens for monochrome files

  • Select the ROIs or confirm the ROI selection, as required. The results will appear.

Simulated noise vs. input pixel level the LX5 (2.14 μm pixels)

The simulated noise is nearly identical to the measured noise (above), but the plots look slightly different because the minimum x-axis value is different: 4×10-3 for the measured value vs. 7×10-4 for the simulation, which goes lower because it has no veiling glare (stray light; flare light). Veiling glare could have been added with the Fog ISP processing block. 

Modeling and measuring image sharpness

The next step is to measure and model the image sharpness (SFR or MTF). Since we don’t have the degraded image from a lens design program, we need to estimate the lens blur by trial-and-error using a gaussian or Airy disk filter. Once we have a good model of the raw image, we ae free to try out any adjustments or enhancements. 

Since a raw image is required only for finding the noise model, we demosaic (convert) the raw image using LibRaw with the Color 24-bit sRGB preset. The resulting curves are typical for unsharpened images.

Measured Edge/MTF plot for the Lumix LX5, demosaiced to 24-bit sRGB. Key parameters are 10-90% rise and MTF50.

Simulating image sharpness

Now we run Simatest, using an input image of the 36-patch chart with slanted edges generated by the Imatest Test Charts module.

Simatest window for matching LX5 simulation with measurement: key setting: Filter gaus 0.75

The following sequence were repeated until we got a good match between the measured Edge/MTF results (above) and the simulated Edge/MTF results.

  •                       Filter settings


    Click Simatest Settings (upper left of Simatest window).
  • Filter settings:  We tried both Gaussian and Airy disk filters. Gaussian was slightly better, but not perfect. We adjusted the slider several times until we settled on a final value of sigma = 0.75 (pixels). Recall that the selection requires two steps: set the slider to the desired value, then press the Set dropdown menu to copy the setting to the selected the line in the ISP Blocks box.
  • Click OK to return to the main Simatest window.
  • Click Update Calculations.
  • (optional) You can display the side-by-side view, with the input image on the left and the processed image on the right, shown above. Although this view is occasionally interesting, we typically skip over it.
  • Click Rescharts near the bottom of the Simatest window (on the left of the light green area).
  • Make sure 1. SFR is chosen for chart, then click on Analyze image.
  • Select the region (for the first try) and make a fine adjustment if necessary, or confirm the saved selection (for succeeding tries) by pressing Yes, Express mode.
  • If the Edge/MTF plot is selected, the simulated Edge and MTF are displayed. Many other plots are available in both the Rescharts and Color/Tone modules.

Simulated Edge/MTF plot for the LX5. Results are very close to the measured plot, above.
Key results — 10-90% rise distance, MTF50, Information capacities C4 and Cmax — are all within a few percent.
The match is not perfect, but it’s \good enough for testing ISP.