Related web pages

Simatest ISP/Camera Simulator — Instructions and Reference

Image Sensor Noise – measurement and modeling – using raw files for image sensor Dynamic Range and Simatest

Using Color/Tone Interactive – Interactive analysis of color & grayscale test charts

Using Color/Tone Auto – Fixed (batch-capable) analysis of color & grayscale test charts

Color/Tone & eSFR ISO noise measurements – describes how to measure image sensor noise from raw images

Dynamic Range – a general introduction with links to Imatest modules that calculate it

Noise in Photographic Images – a basic introduction

Color Correction Matrix (CCM) – Calculate a matrix (usually 3×3) for correcting image colors (best from RAW images)

Image information metrics – new measurements, based on Shannon information theory, that are superior predictors of Machine Vision system performance. 

 
 Introduction – Simatest block diagram –  Modeling image sensor noise – Measuring the noise 
Simulating the noise – Modeling and measuring image sharpness – Simulating image sharpness 
Simulating camera JPEGs – Summary

This page illustrates how Simatest can be used for modeling and measuring camera systems and, most importantly, Image Signal Processing (ISP), using a compact digital camera as an example. It’s designed to be more practical and readable than the full instructions and reference in ISP/Camera Simulator (Simatest).

Until Imatest 2025.1 is released, Simatest will be available through the Imatest Pilot program. 

Introduction

Simatest can model and measure the effects of Image Signal Processing (ISP) starting with

  1. Raw camera image — the raw output of a digital camera (demosaiced or not), which includes lens blur and image sensor noise. Test chart images are typically used for this case, but any image can be used, or
  2. Simulated (test chart) image, which needs to be blurred to match the MTF (SFR) of a raw camera image, typically using a simple gaussian or Airy disk filter. Must also include image sensor noise from the model described below. 
  3. Simulated images degraded by a lens design program. This enables the “soft prototyping” of a camera before the hardware is specified and built.  Image sensor noise (described below) is required. A complete ISP chain can be applied to images from any source.

 

The camera used for the examples on this page is the Panasonic Lumix LX5, an older (2010) 10.1 megapixel compact digital camera with 2.14 μm pixel pitch and a high quality Leica-branded zoom lens. Its image processing is simple by 2025 standards — it has no Computational Imaging or Artificial Intelligence, and most importantly, it has raw output, which can be processed with or without demosaicing. We will show how to

  • Model the image sensor noise using a raw image of a grayscale test chart.
  • Model the approximate camera spatial domain response and MTF (SFR) using a simple gaussian or Airy disk filter.
  • Use the camera  model to show how camera performance varies as a function of Exposure Index (EI; widely known as ISO speed), where increasing the EI is equivalent to decreasing the illumination.

Modeling image sensor noise

A key step — perhaps the most important step — in simulating digital camera performance is modeling the image sensor noise, which consists of dark noise from several sources, photon (or electron) shot noise, and PRNU (photo response nonuniformity). The Simatest model is derived from measurements of raw images of grayscale test charts, preferably from transmissive test charts, which have much larger tonal ranges than reflective charts. 

Once an image sensor’s noise has been properly characterized, Simatest can be used to predict camera performance for a wide variety of camera configurations — lenses, image processing, etc.

Image sensor noise is described in Image Sensor Noise – measurement and modeling and is also referenced in Simatest instructions and reference.

For linear (non-HDR) image sensors, noise can be derived from the Digital number (pixel level DN ) of the signal + noise at the image sensor.

DN = DNmeas – DNoff  where DNoff  is an offset frequently included in undemosaiced image files.

Estimating DNoff , which is common in undemosaiced raw files, is described below and in Black level offset and saturation level. 

The equation for noise σN uses the normalized Digital Number, V = DN / DNmax, as described in Image sensor noise model — equations.

\(\sigma_N = \sqrt{k_{Ndark}^2 + k_{Nshot}^2 V + k_{FPN}^2 V^2}\)

This model is a variant of the noise model in the EMVA 1288 standard for linear sensors, section 2.4.

Measuring the image sensor Noise

kNdark , kNshot , and kFPN are the key parameters for the image sensor noise model in Simatest. Measurements must be made carefully from raw (undemosaiced and unprocessed) images.

The steps for modeling the noise

  1. Acquire an image of a grayscale test chart, preferably a transmissive chart designed for measuring dynamic range.
  2. Save the image in a raw format.
  3. Read the raw image into Color/Tone Interactive without demosaicing. For commercial raw files, the Bayer RAW 16-bit linear preset is recommended. (Color Tone Auto also works, but is less flexible.) The converted image is saved as a TIFF. Details are given in Color/Tone/eSFR ISO Noise.
  4. Since the undemosaiced image is monochrome, the Standard monochrome or Bayer RAW? box will appear. Select the appropriate Pixel color mapping (Red in R1C2 (GR/BG) for the Lumix LX5). (If the image contains recognizable color, you can open it in Rawview to find the color mapping.) Then select (Analyze image as) Bayer RAW.
  5. Estimate the Black Level offset, DNoff  and saturation level using techniques described in Black level offset and saturation level.  For example, DNoff can be larger than the smallest DN (image pixel level) but must be smaller than the DN of the darkest patch; it’s OK for the starting guess to be 0. Check the Optimize checkbox to the right of DN Offset in the Color/Tone settings to obtain a refined value for DNoff .  Photon Transfer Plots 10-12, which show noise and SNR as a function of pixel level, are of greatest interest for raw images. Results are not highly sensitive to the exact value of DNoff . Fortunately, the results are relatively insensitive to the exact offset value. For the Lumix LX5, the EXIF data says offset is 128 and the saturation level is 4095, but the actual offset is closer to 142.
  6. Press OK when your settings are complete. A Levenberg-Marquardt optimizer will be run to find the values of kNdark , kNshot , and kFPN  (the coefficients for dark noise, photon shot noise, and fixed-pattern noise) that best match the data. These are the coefficients to enter into Simatest.
  7. Select Display 10. Noise analysis to display the noise as a function of Digital Number — the Photon Transfer Curve. SNR plots 11 or 12 are sensitive to offset + stray light, making them less robust than the simple noise plot. 

Photon Transfer Curve, similar to OnSemi Application Note AND9189/D
Measured and fit noise vs. normalized Digital Number for the LX5 (2.14 μm pixels)

The key results to enter into Simatest, shown below the plot, ar  the [Dark, shot, FP noise coefficients for Simatest] = 0.0005425, 0.00985, 0.00615. The saturation (well) capacity of 7079 electrons (e-) corresponds to 38.5 dB SNR (a little lower than the measured SNR which includes the PRNU fixes-pattern noise). The DN offset (DNoff = 143) and saturation level (DNmax = 4095) must also be entered.

Another result of importance is RGB = 0.429 1.000 0.603, which represents the color of the middle gray patches, normalized to 1. Because of the light source and sensor sensitivity, the image (thumbnail shown on the left with no color correction) has a greenish tint.

More details of the noise calculations are in Image Sensor Noise – measurement and modeling and is also in Simatest instructions and reference.

Dark noise is actually a combination of several signal-independent noise sources, described in OnSemi Application Note AND9189/D. Electronic (Johnson) noise has a strong temperature dependence. It is proportional to (4kT)1/2. Dari current noise increases with exposure time.

We expect to add more detail to the noise model in future releases. We will also move the noise calculations so results are available in the CSV and JSON output. And (a sort of a holy grail of sensor modeling), we expect to simulate multi-region HDR sensors. 

Simulating the image sensor noise

The measured parameters for photon shot noise noise, dark noise, and RGB balance can be entered into a Simatest model of the image.

  • Open Simatest and read in an ideal test chart image file created by the Imatest Test Charts program. Our examples use two charts.
  • Open the Simatest settings window, and enter the following settings to replicate the (undemosaiced) raw image we just measured. The noise and filter settings are shown on the right.
         Gamma Targ 1
         Filter gaus 0.75
         RGB mult 0.43 1 0.60
         Bayer grbg
         Noise (gaussian) – sensor
         Noise (dark) 0.0005498
         Shot noise 0.000985
         Fixed Pattern Noise 0.00615
         Noise 0.0114 1 1 0.00067.
    These settings are described in the large table in Simatest — instructions and reference. Settomgs prefixed by % are disabled (commented out). Click OK when the settings are complete.
  • We recommend checking Auto update in the lower-right of the Settings window, next to OK. This causes the calculations to be updated when OK is pressed to close the Settings window. If Auto update hasn’t been checked, press Update calculations in the main window. The resulting image will be a Bayer raw image.
  • To get an image with the same offset and maximum value as the camera (143 and 4095 for the LX5), Set  Save images – 16 bit depth, Offset (DN) 142, Saturation (DN) 2095  at the bottom-right of the settings window, then press Save processed image to save the image in a named file.
  • Open the image in Color/Tone Interactive, with the 36-patch Dynamic Range chart selected. Because the undemosaiced image file is monochrome, you will get the window shown below. Key settings are circled.
  • Select the ROIs or confirm the ROI selection, as required. The results will appear.

Simulated noise vs. input pixel level the LX5 (2.14 μm pixels)

The simulated noise is virtually identical to the measured noise (above), but the plots look slightly different because the minimum x-axis value is different: 4×10-3 for the measured value vs. 7×10-4 for the simulation, which goes lower because it has no veiling glare (stray light; flare light). 

Simulating low light performance

One of the advantages of the image sensor noise model is that it lets you simulate low light performance without low light measurement equipment.

Cameras use two strategies for low light conditions

  1. Increase the exposure time. This generally applies only to still camreras. Video cameras have limits on their maximum exposure times, typically around 1/30 second. For long exposures, dark current noise can be significant. In EMVA 1288, it is measured in units of electrons per second of seposure (e-/s). At this time, Imatest does not currently (as of (March 2025) have a measurement of dark current noise, but it can use EMVA 1288 measurements by substituting iDark in e-/s and exposure time s into \(\displaystyle k_{NDark} = \sqrt{\sigma^2_{Dk}+DSNU^2 + (i_{Dark}\ s)^2} \times \text{Gain}(V/e-) \)
  2. Increase the analog gain, or equivalently, increase the ISO Speed (Exposure Index). Simatest simulates this by decreasing the signal, adding the image sensor noise, then restoring the signal to its original level. Here is an example.

We measured a raw (ARW) image file from the Sony A6000, a 24 MP Micro Four-Thirds camera with 3.88 μm pixels set to ISO (EI) 100. The results to enter into Simatest were:  kNdark = 0.0001672;  kNshot = 0.005554;  kFPN = 0.005477;  and  RGB = 0.306, 1.0, 0.606.

Modeling and measuring image sharpness

The next step is to measure and model the image sharpness (SFR or MTF). Since we don’t have the degraded image from a lens design program, we need to estimate the lens blur by trial-and-error using a gaussian or Airy disk filter. Once we have a good model of the raw image, we ae free to try out any adjustments or enhancements. 

Since a raw image is required only for finding the noise model, we demosaic (convert) the raw image using LibRaw with the Color 24-bit sRGB preset. The resulting curves are typical for unsharpened images.

Measured Edge/MTF plot for the Lumix LX5, demosaiced to 24-bit sRGB. Key parameters are 10-90% rise and MTF50.

Simulating image sharpness

Now we run Simatest, using a demosaiced, minimally processed input image of the 36-patch chart with slanted edges generated by the Imatest Test Charts module.

Simatest window for matching LX5 simulation with measurement: key setting: Filter gaus 0.75

The following sequence were repeated until we got a good match between the measured Edge/MTF results (above) and the simulated Edge/MTF results.

  •                   Move the Filter setting to ISP Blocks, line 3.


    Click Simatest Settings (upper left of Simatest window).
  • Filter settings:  We tried both Gaussian and Airy disk filters. Gaussian was slightly better, but not perfect. We adjusted the slider several times until we settled on a final value of sigma = 0.75 (pixels).

Recall that Simatest settings require two steps:  (1) set the slider (or other adjustment) to the desired value, then (2) press the Set dropdown menu to copy the setting to the selected the line in ISP Blocks, as shown by the red arrow on the right.

  • Click OK to return to the main Simatest window.
  • Click Update Calculations.
  • (optional) You can display the side-by-side view, with the input image on the left and the processed image on the right, shown above. Although this view is occasionally interesting, we typically skip over it.
  • Click Rescharts near the bottom of the Simatest window (on the left of the light green area).
  • Make sure 1. SFR is chosen for chart, then click on Analyze image.
  • Select the region (for the first try) and make a fine adjustment if necessary, or confirm the saved selection (for succeeding tries) by pressing Yes, Express mode.
  • If the Edge/MTF plot is selected, the simulated Edge and MTF are displayed. Many other plots are available in both the Rescharts and Color/Tone modules.

Simulated Edge/MTF plot for the LX5. Results are very close to the measured plot, above.
Key results — 10-90% rise distance, MTF50, Information capacities C4 and Cmax — are all within a few percent.

The match is not perfect. The shape of the MTF is a little off, but it’s good enough for testing ISP. 

Simulating the camera JPEG image

Now that we have simulated image sensor noise and the sharpness of a demosaiced, minimally processed image (from the LX5) we can move on to something more challenging: simulating the JPEG image. 

Camera JPEGs typically have several additional image processing steps beyond the demosaiced raw image. 

  • Tonal response curve — can be simple gamma encoding (close to gamma specified for the color space, but sometimes slightly larger to give the image a more snappy appearance).
  • Color correction (including White Balance) — Often done by a 3×3 Color Correction Matrix (CCM), but we use simple “gray world” white balance, which is much too simple to handle real-world challenges, in this example. 
  • Sharpening — standard sharpening (instead of USM) is used in most cameras because it’s faster. 
  • Noise reduction — typically with a bilateral filter, which keeps edges sharp, but blurs (smooths) the image elsewhere. Bilateral filters are nonuniform and nonlinear. They are nearly universal in consumer camera JPEG images.
  • Note that the high quality JPEG image compression in most consumer cameras has little effect on image quality.  

Here is the measured JPEG for the LX5, acquired at ISO 100 (Exposure Image). 

Measured LX5 JPEG Edge/MTF response

The second plot is the Edge noise, used to calculate image Information metrics. Edge noise is the lower plot of the Edge & Info Capacity noise display.  Measured LX5 JPEG Edge noise

To simulate this noise, we sharpened the image and applied the MATLAB bilateral filter, which required considerable trial-and-error because it’s not well-documented. Here are the settings.

 

Simatest settings for JPEG image

Here is a quick summary of the iterative procedure. 

Simulated LX5 JPEG Edge/MTF response
Results are similar to the measured image, except near fNyq, where MTF is higher.

Some considerations

  • The same input image (from Imatest Test Charts) as the previous simulations was used. The central slanted edge was used to obtain the MTF results.
  • Standard sharpening with  Radius = 2 and Amount = 1.4 was chosen for a good match to the measured JPEG image. There was some interaction between the sharpening and Bilateral filter settings. To maintain the asymmetry between the larger overshoot and smaller undershoot, sharpening was applied after the encoding gamma (≅0.45) was restored.
  • The MATLAB imbilatfilt filter was used. Since Its documentation is not perfectly clear, we had to do a lot of trial-and error to arrive at the settings. We selected Degree of Sharpening = 0.0025, Spatial sigma = 0.7, and Neighborhood area = 5 (must be an odd integer). The noise away from the peak appeared to decrease as Degree of Sharpening was increased. 
These results show how Simatest ISP simulation agrees well (though not perfectly) with measured results. Since we don’t know the properties of the bilateral filter in the LX5, we correctly guessed that it could be approximated by MATLAB imbilatfilt. 

Summary

Using the humble Panasonic Lumix LX5 as an example, we have shown how to measure, model and simulate

  • image sensor noise, which can be measured from a raw (undemosaiced) image,
  • a minimally processed RGB image (converted from raw), and
  • a JPEG image (which includes sharpening and bilateral filtering).

The techniques presented here can be used with

  • minimally-processed images, converted from raw. These images don’t require the sensor noise or lens blur to be simulated since they’re implicitly included in the image.
  • simulated image, especially ideal test chart images. Sensor noise and blur models are required.
  • simulated images blurred by a lens design program. The sensor noise model is required.

These cases are illustrated in the block diagram near the beginning of this page.

A great many image processing blocks can be simulated. We have focused on blocks that affect MTF, SFR, noise, information metrics, and color accuracy. The early 2025 release is just a beginning. We expect to add more image processing blocks that customers find useful.