Machine Learning for Safety-Critical Applications

Advances in artificial intelligence, and specifically in machine learning, are enabling new capabilities across nearly every sector of the economy. Many of these applications - such as automated vehicles, the power grid, or surgical robots - are safety critical: where malfunctions can result in harm to people, the environment, or property. While machine learning is already being deployed to enhance the capabilities of some physical systems, extending the rigorous practices of safety engineering to include machine learning components brings significant challenges.

Machine Learning for Safety-Critical Applications explores ways to safely integrate machine learning into physical systems and presents research priorities for improving safety, testing, and evaluation. This report finds that designing machine learning algorithms in a way that aligns with safety engineering standards will require changes in research, training, and engineering practice - as well as a shift away from focusing on algorithmic performance in isolation.

Table of Contents

Front Matter Summary 1 Engineering Safety-Critical Systems in the Age of Machine Learning 2 State of the Art, Promises, and Risks of Machine Learning 3 System Engineering with Machine Learning Components for Safety-Critical Applications 4 A Research Agenda to Bridge Machine Learning and Safety Engineering 5 Societal Considerations to Build Public Understanding and Confidence in Safety-Critical Systems with Machine Learning Components Appendix A: Statement of Task Appendix B: Briefings to the Committee Appendix C: Committee Member Biographical Information