Low cost real time LED detection using CIS

Modern street scene consists of many LEDs, and some of those LEDs, for example: vehicle headlight, road sign and billboard board, contain information which are important for proper operation of autonomous vehicle. However, LED uses pulse width modulation (PWM) which means it pulsates between on and off state at a certain frequency. To make things worse, different LED manufacturers use different PWM frequencies. As a result, capturing LEDs using CMOS Image Sensor (CIS) without flickering is a challenging task which requires special processing to mitigate flickering.

But LED flicker mitigation (LFM) processing can cause artifacts in other places in the image, for example, blurring of road texture. Therefore, it is important to find location of LEDs in an image and apply special processing only on those locations instead of the whole image.

The main goal of this project is to research and develop a low-cost LED detection algorithm suitable for deploying in FPGA that can detect LED in real time video frames of different lighting (e.g. night vs day) and weather (e.g. sunny vs rain) conditions.

The student will investigate different existing LED detection algorithms and eventually propose and implement an algorithm. The focus should be on accuracy of LED detection at varied condition as well as lower cost in terms of complexity and memory usage.