PhD Oral Exam - Ameen Abdelmutalab, Electrical and Computer Engineering

When studying for a doctoral degree (PhD), candidates submit a thesis that provides a critical review of the current state of knowledge of the thesis subject as well as the student’s own contributions to the subject. The distinguishing criterion of doctoral graduate research is a significant and original contribution to knowledge.

Once accepted, the candidate presents the thesis orally. This oral exam is open to the public.

Abstract

Pedestrian detection is essential in various applications, such as self-driving vehicles, video surveillance, and intelligent traffic management. However, the wide variations in pedestrian sizes, postures, locations, and backgrounds make the detection a complex task. In particular, the detection becomes significantly challenging due to the lack of pedestrian information when pedestrians are occluded by other objects, such as vehicles or trees, or when they appear as objects of small-scale in an input image. Such situations occur frequently in the real world. The objective of this thesis is to design CNN-based pedestrian detection models to improve the detection of occluded and small-scale pedestrians.

The first part of this work addresses the occlusion problem by proposing a specific detection model referred to as Multi-Branch Center and Scale Prediction (MB-CSP). The proposed model employs a multi-branch structure to optimize the utilization of the features extracted from the visible parts of pedestrians. This structure enables the feature data from the upper, middle, and lower parts of a pedestrian, as well as those of the full body, to be processed separately. By doing so, the data representing the true pedestrian appearances, whether partially or fully visible, can be more dominating in the final decision making. As a result, the interference from non-pedestrian data in the detection can be minimized. To optimize the fusion of the detection outcomes generated by the multiple branches, a new method referred to as Boosted Identity Aware-Non-Maximum Suppression (BIA-NMS) is developed and applied in the design of the MB-CSP detection system. The BIA-NMS method eliminates redundant detections across branches and boosts the scores of the preserved detections. To implement the proposed model, a part annotation algorithm has been introduced to enable the training of the multi-branch structure. It is anticipated that the proposed model can boost the overall performance of the pedestrian detection system.

The second part of this work provides a number of approaches to improving the detection of small-scale pedestrians, besides the occluded ones. One can use two CNNs designated to detect pedestrians of large and small scales, respectively, to achieve a good detection in each of the two cases. Instead of two designated CNNs, one can use only one and incorporate a specific branch in the proposed MB-CSP model to process the features of small-scale pedestrians. The other approach proposed in this thesis is to segment the original input image into multiple partially overlapped sub-images, the likelihood of the presence of small-scale pedestrians in each sub-image is measured, and those of high scores are selected and enlarged. The detection is performed by two CNNs, of which one is designed for the original image and the other for the selected/enlarged sub-images, in order to enhance the detection of small-scale pedestrians while preserving the detection quality of the occluded pedestrians.

The detection systems presented in this thesis have been trained and evaluated using image samples from the Caltech-USA and CityPersons datasets. The tests have confirmed the effectiveness of the proposed multi-branch system in detecting occluded pedestrians. The test results have also demonstrated that the approaches to enhance the small-scale pedestrian detection produced a visible improvement in this aspect without affecting the detection of occluded pedestrians.