Seminar: Pragmatic Prioritization Software Quality Assurance Efforts

Dr. Emad Shihab (Rochester Institute of Technology)

Monday, January 13, 2014, 10:30AM-12:00PM, EV 11.119

Abstract

Software Quality Assurance (SQA), which involves the processes and methods of ensuring that software does not break and meets its intended purpose, is one of the major focus points of Software Engineering research today. Researchers have found that a software system's past can be a very good indicator of its future quality. For example, prior work showed that the number of pre-release changes is a good predictor of fault-prone files. However, to date these predictions have limited adoption in practice. The most commonly cited reason is that the prediction identifies too much code to review without distinguishing the impact of these defects.

Our work focuses on making SQA research more pragmatic, i.e., making advanced SQA techniques applicable in practical settings. This work is based on our experience working with some of the world's largest software companies - namely Avaya, BlackBerry and Microsoft. We will share experiences and present two pragmatic approaches used to ensure software quality. First, we focus on understanding and identifying high impacting defects, i.e., defects that catch practitioners off-guard. Practitioners are much more interested in these high-impact defects since, as we have found, focusing on high-impact defects reduces the amount of code that needs to be reviewed by 40-50% and helps practitioners focus on defects that significantly impact the quality of their software systems.

In addition, we present a proactive approach where risky changes, i.e., changes that may break or cause errors in the software system, are flagged so defects can be avoided before they are widely integrated into the code. We will present the results of a year-long study involving more than 450 developers, spanning more than 60 teams to better understand and identify these risky changes. We find that attributes such as the number of lines of code added and the history of the files being modified by the change can be used to accurately identify risky changes with a recall of more than 67% and a precision that is 37-87% higher than a baseline model. Our change risk models are being used today by an industrial partner to manage the risk of their software projects.

Bio

Dr. Emad Shihab is a Dr. Emad Shihab is an Assistant Professor in the Department of Software Engineering at the Rochester Institute of Technology, New York, USA. His general research area is Software Engineering. He is particularly interested in Mining Software Repositories, Software Quality Assurance, Software Maintenance, Empirical Software Engineering and Software Architecture. He mines historical project data and applies Data Mining, Artificial Intelligence and Statistical Analysis techniques in order to build pragmatic solutions that practitioners can use to maximize their software quality with the least amount of resources. Some of his research has been done in collaboration with and/or adopted in industry by companies such as Avaya, Microsoft and Research In Motion.