PhD Oral Exam - Ahmed Alagha, Information Systems Engineering

When studying for a doctoral degree (PhD), candidates submit a thesis that provides a critical review of the current state of knowledge of the thesis subject as well as the student’s own contributions to the subject. The distinguishing criterion of doctoral graduate research is a significant and original contribution to knowledge.

Once accepted, the candidate presents the thesis orally. This oral exam is open to the public.

Abstract

In a world that increasingly relies on autonomous systems, swarm robotics hold the promise of revolutionizing how complex tasks, such as target search and localization, are approached. The ability of multiple autonomous agents, such as robots and UAVs, to work together, exchange information, and adapt to dynamic environments is critical for target localization applications ranging from search and rescue missions to environmental monitoring. However, efficiently coordinating a swarm of robots to search for targets in uncertain and complex environments poses significant challenges. Most existing solutions for target search and localization still possess challenges and limitations in terms of adaptability to different environments and scalability. These challenges become even more intricate when the target may not exist (i.e. false alarms) or is unreachable.

In this thesis, the main motivation is to leverage AI, specifically Multi-Agent Deep Reinforcement Learning (MDRL), to address the target search and localization problem. The aim is to develop MDRL solutions where the agents intelligently and autonomously learn to tackle the problem and its different complexities, with minimum human intervention. The capacity of MDRL in producing agents capable of learning from their experiences in the environment proves efficient in handling complex and dynamic scenarios, such as coordinating with other agents, translating data readings into actions that lead to the target, and navigating obstacles in the environment. This research is motivated by four main needs: (1) Adaptable solutions for collaborative target search and localization for varying environment complexities; (2) autonomous and intelligent sensing agents with decision-making that addresses scenarios like false alarms and target unreachability; (3) scalable and efficient AI-based learning process for the sensing agents, and (4) mechanisms for knowledge exchange across different users and parties from different domains for better accessibility to AI solutions.

The aforementioned needs are addressed in this thesis by: (1) Developing novel MDRL algorithms for collaborative target search in both simple and cluttered environments by modeling the problem as a Markov Decision Process (MDP), (2) designing novel methods based on ideas from MDRL, Imitation Learning (IL), and reward shaping for enhanced and quick learning performance, (3) enhancing the proposed MDRL algorithms by integrating complex decision-making, where agents can take actions ranging from mobility to deciding on the existence and reachability of the target, (4) developing a blockchain-based platform for Deep Reinforcement Learning as a Service (DRLaaS) allowing collaborative training and better accessibility to DRL solutions for target localization problems, and (5) ensuring the scalability of all the proposed solutions through the use of concepts such as Centralized-Learning and Distributed Execution (CLDE) MDRL methods coupled with Convolutional Neural Networks (CNNs) for optimized analysis of the agents' collected observations. Besides these contributions, we present several experimental studies and simulations that validate the proposed methods and compare them against existing state-of-the-art benchmarks in the literature.