AI-Assisted Automated Programing and Automated Program Repair

Abstract:

Recent advancements led by large-scale language models (LLMs) show that Artificial General Intelligence (AGI) is potentially within reach, as demonstrated in numerous applications such as AI agents and code generation. LLMs have also been leveraged for high-level task planning of robotics in the real world. These progresses are contributed by the logistic reasoning capabilities embedded in the LLMs. However, it remains an open question how we can achieve intelligent systems that can interact with the physical world and understand its underlying principles.

In this talk, I will first briefly review the previous efforts in computer vision, focusing particularly on how these pioneering works perceive the physical world computationally and how they advanced the development of intelligent autonomous systems. Then, I will introduce the idea of integrating the first-principle rules within neural networks through optimization with learned objective function and regularization. This idea, first adopted for the task of 3D reconstruction of static scenes, bridges the gap between end-to-end learning-based methods and conventional multi-view geometry methods. Following this, I will introduce subsequent works that generalize this idea to a variety of other low-level vision tasks. Specifically, I will discuss how to decouple the task-specific objective function from the model, making it possible to learn a single model for multiple tasks with shared model weights, and even enabling zero-shot transfer between tasks.

Biography:

He Ye is a postdoctoral researcher at Software and Societal Systems (S3D) at Carnegie Mellon University. Her main research interest is in Software Engineering (SE), focusing on building automated tools to improve software developers productivity and software quality, to help developers predict, detect, localize, and fix software bugs automatically. Her research also involves the synergy between Machine Learning and Software Engineering. She obtained her Ph.D. from KTH Royal Institute of Technology in Sweden and her B.Sc. in software engineering from Sichuan University in China. For more information, please visit her personal web page: https://heye.me.