PhD Oral Exam - Linhan Qiao, Mechanical Engineering

When studying for a doctoral degree (PhD), candidates submit a thesis that provides a critical review of the current state of knowledge of the thesis subject as well as the student’s own contributions to the subject. The distinguishing criterion of doctoral graduate research is a significant and original contribution to knowledge.

Once accepted, the candidate presents the thesis orally. This oral exam is open to the public.

Abstract

This work focuses on the increasingly serious and urgent environmental problem of wildfire, studying and testing the possible schemes, strategies and solutions in real application of autonomous wildfire management and fighting.

To efficiently tackle these wildfire fighting challenges with unmanned aerial vehicles (UAVs), several intelligent computer vision algorithms are studied, updated, and finetuned. These algorithms are designed to work in conjunction with UAV motion and path planning algorithms to detect (early) wildfire spots and efficiently approach these spots for firefighting.

The main contribution of this work is the design of an intelligent wildfire perception system using visible and infrared (VI) image information with UAVs, primarily through a deep learning-filtered oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (ORB features) mechanism. This work proposes a novel concept of combining deep learning models and ORB based simultaneous localization and mapping (SLAM) technologies as UAV applications in vision-based wildfire perception and management/fighting.

There are three main functional aspects of the intelligent wildfire perception system of this work.

The first aspect is wildfire detection, which includes wildfire image classification, wildfire semantic segmentation, and wildfire spot(s) detection (object detection). For wildfire image classification, an optimized ResNet-based model is utilized to achieve higher classification accuracy. For wildfire semantic segmentation, this work focuses on the U-shaped deep network models (UNets) and proposes the application of original UNet, an attention gate enhanced UNet, and a SqueezeNet lightweight attention gate UNet for early wildfire smoke and flame spot segmentation. For online wildfire object detection, the model of you only look once version 5 (YOLOv5) and updated model of you only look once version 8 (YOLOv8) are utilized to obtain accurate wildfire spot bounding boxes, YOLOv8 model can avoid pre anchors and straightforwardly detect and track the center of wildfire spots.

The second aspect comprises several smaller functions to complete the wildfire perception system. Most of these functions use the infrared information, because the energy radiation information could support the deep learning wildfire detection to have more confident and robust detection results. A geometry based visible and infrared (VI) image alignment and registration scheme is designed in this work. It is the basement of the visible-infrared image fusion. Infrared images can also be used to estimate wildfire spot temperature to guide the safe flight of UAVs. After that, there is a design of online water retardant release mechanism, and it is slightly stated.

The third aspect is cooperated work with other team to achieve UAV-wildfire distance estimation and wildfire geo-positioning thought monocular ORB-SLAM technology (SLAM2 and SLAM3). This function has two main designs: The first one designs an attention gate UNet to filter ORB feature points for wildfire distance estimation, achieves more robust results and detailed segmentation at the edges of wildfire smoke and flame spots. This design can be deployed on ground workstations for detailed missions. The other one designs YOLOv8 filtering ORB-SLAM3 features for online wildfire distance estimation and geo-positioning, which can be deployed on onboard computers for real-time wildfire spot recognition and geo-positioning. This lightweight and fast-responding application can combine with UAV path and motion planning for online firefighting.