Benchmarking beamformers using simulated data

Adaptive array signal processing methods are increasingly used to improve image quality in ultrasound based surveys. However, adaptive methods can be very sensitive to the circumstances they are used in. Therefore, choosing the right one from the many available candidates requires extensive testing in various scenarios, which is often not feasible in practice. 

Benchmarking beamformers using a simulated underwater scene.

Output figures of the benchmarking software. The images on the left show the risk of the beamformers for each pixel with positive numbers for false positive and negative numbers for missed detections. The figures on the left show the ROC curves and the AUC values.

Ultrasound channel data can be generated by computer simulations for a large number of scenarios with a great freedom in choosing the parameters. These virtual recordings then can be used to benchmark adaptive array signal processing methods with the additional advantage that the ground truth is known, which can be difficult to ensure under field conditions.

Realistic simulation of complex scenarios using the Field II Ultrasound Simulation Program (see https://field-ii.dk/) in combination with adaptive array signal processing of the output using the open access package UltraSound ToolBox USTB (see https://www.ustb.no/) has been previously demonstrated. A Matlab based software, the Ultrasound Beamfomer Benchmarking Pipeline (see https://doi.org/10.5281/zenodo.7057237) has been developed to run a large number of random generated scene variants with added noise through Field II and USTB and evaluate the beamformers capability to detect the true position of the scatterer points (see sidenote).

In this project, we aim to test adaptive beamformers using the benchmarking pipeline under various scenarios and attempt to understand their strenghts and limitations, and if possible improve them or make them more suitable for specific applications.

In the project, you will do the following:

  • Study the literature on detection theory and adaptive beamforming.
  • Implement various beamformers in USTB.
  • Design and simulate test scenarios using the Beamformer Benchmarking Pipeline.
  • Evaluate the beamformer performance and attempt to find the underlying cause.

Qualifications:

  • Excellent MATLAB programming skills (Object oriented MATLAB)
  • Signal Processing (IN3190/4190, IN5450, IN5340 and preferably IN3015/4015)
  • Git Version Control

Literature:

  • A. R. Molares et al., "The Generalized Contrast-to-Noise Ratio: A Formal Definition for Lesion Detectability", IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, vol. 67, no. 4, 2020, doi: 10.1109/TUFFC.2019.2956855.
  • T. I. B. Lønmo, A. Austeng, and R. E. Hansen, "Low Complexity Adaptive Beamforming Applied to Sonar Imaging", in UACE2015 - 3rd Underwater Acoustics Conference and Exhibition, Crete, 2015 [Online]. Available: https://www.duo.uio.no/handle/10852/55568
Publisert 20. sep. 2022 14:14 - Sist endret 20. sep. 2022 14:14

Veileder(e)

Omfang (studiepoeng)

60

The benchmarking software evaluates the detection performance of the selected beamformers by calculating the probability of true positive and false positive detections using the random iterations of the beamformed images and comparing it with the truth image. The algorithm thresholds each pixel of the normalized beamformed images at various threshold levels and consider the pixel values above the threshold as positive detections, then if the pixel contains scatterer points, it registeres a true positive, otherwise a false positive detection. Using the detection probabilities, the function calculates the average risk of the beamformer for each pixel of a collection of image realizations, and then calculates the Receiver Operation Curve (ROC). Finally it measures the Area Under the Curve (AUC) value that can be used to directly compare the performance of multiple beamformers.