PhD Oral Exam - Ningsheng Zhao, Information and Systems Engineering

When studying for a doctoral degree (PhD), candidates submit a thesis that provides a critical review of the current state of knowledge of the thesis subject as well as the student’s own contributions to the subject. The distinguishing criterion of doctoral graduate research is a significant and original contribution to knowledge.

Once accepted, the candidate presents the thesis orally. This oral exam is open to the public.

Abstract

Explainable AI (XAI) is an emerging field focused on providing human-interpretable insights into complex and often black-box machine learning (ML) models. Shapley value attribution (SVA) is an increasingly popular XAI method that quantifies the contribution of each feature to a model’s behavior, which can be either an individual prediction (local SVAs) or a performance metric (global SVAs). However, recent research has highlighted several limitations in existing SVA methods, leading to biased or incorrect explanations that fail to capture the true relationships between features and model behaviors. What's worse, these explanations are vulnerable to adversarial manipulation.

Additionally, global SVAs, while widely used in applied studies to gain insights into underlying information systems, face challenges when applied to ML models trained on imbalanced datasets, such as those used in fraud detection or disease prediction. In these scenarios, global SVAs can yield misleading or unstable explanations.

This thesis aims to address these challenges and improve the reliability and informativeness of SVA explanations. It makes three key contributions: 1) Proposing a novel error analysis framework that comprehensively examines the underlying sources of bias in existing SVA methods; 2) Introducing a series of refinement methods that significantly enhance the informativeness of SVA explanations, as well as their robustness against adversarial attacks; 3) Developing a standardization method for evaluating global model behaviors on imbalanced datasets, advancing the development of an explainable model monitoring system. Our experiments demonstrate that these methods substantially improve the ability of SVAs to uncover informative patterns in model behaviors, making them valuable tools for knowledge discovery, model debugging, and performance monitoring.