Artificial intelligence (AI) and Machine Learning (ML) systems are increasingly seen in many domains such as self-driving land vehicles, autonomous aircraft, and medical systems. AI systems should equal or surpass human performance, but given the consequences of failure in these systems, how do we determine that the data gathered to train an AI system is suitably representative of the real world? How do we assure the public that these systems work as intended and will not cause harm? These questions have given rise to a new term: “assured autonomy.” In this workshop, issues in assured autonomy such as explain ability, bias, verification, validation, privacy, trust and more for AI and ML systems will be explored. Research, experiences and best practices will be presented to illustrate the challenges and possible approaches to assured autonomy. Finally, the road ahead will be explored.
This workshop will bring together participants from industry, academia and government to tackle these challenging issues. In addition to paper presentations, two panel discussions including international experts from government, industry and academia will be conducted.
Workshop will occur on October 31st. All times are EDT.
8:00
Welcome: Steven Li, IEEE Reliability Society President
Session 1: Chair Phil Laplante
8:45
Assuring Safety-Critical Machine Learning Enabled Systems: Challenges and Promise
Alwyn Goodloe (NASA Langley Research Center)
9:30
Machine-Learned Specifications for the Verification and Validation of Autonomous Cyberphysical Systems
Matthew Litton, Doron Drusinsky and James Michael (Naval
Post Graduate School)
10:00
Break
10:30
Assurance Guidance for Machine Learning in a Safety-Critical System
Martin Feather (Jet Propulsion Laboratory, California Institute of Technology), Philip Slingerland (The Aerospace Corporation),
Steven Guerrini (Jet Propulsion Laboratory, California Institute of Technology) and Max Spolaor (The Aerospace Corporation)
11:00
A Taxonomy of Critical AI System Characteristics for Use in Proxy System Testing
Joanna DeFranco, Mohamad Kassab and Phillip Laplante, (The Pennsylvania State University)
11:30
AI and Stochastic Terrorism Should it be done?
Bart Kemper (Kemper Engineering Services, LLC)
12:00-
Lunch
Session 2: Chair Phil Laplante
1:30
Combinatorial Coverage for Assured Autonomy
Rick Kuhn, M. S. Raunak and Raghu Kacker (NIST)
2:00
XAI for Communication Networks
Sayandev Mukherjee, Jason Rupe and Jingjie Zhu (CableLabs)
2:30
Investigating Bugs in AI-Infused Systems: Analysis and Proposed Taxonomy
Mohamad Kassab, Joanna DeFranco and Phillip Laplante (Pennsylvania State University)
3:00
Break
3:30
Safety-Critical Adaptation in Self-Adaptive Systems
Simon Diemert and Jens Weber (University of Victoria)
4:00
Evaluating Human Locomotion Safety in Mobile Robots Populated Environments
Boyi Hu, Yue Luo and Yuhao Chen (University of Florida)
4:30
Classification Analysis of Bearing Contrived Dataset under Different Levels of Contamination
Shamanth Manjunath, Ethan Wescoat, Vinita Gangaram Jansari, Matthew Krugh and Laine Mears (Clemson University)
Legend:
In Person Presenter
Virtual Presenter
Best papers will be considered for expansion and publication in IEEE Computer magazine.
Submissions must adhere to the IEEE Computer Society Format Guidelines as implemented by the following LaTeX/Word templates:
Papers should not exceed ten pages in IEEE style, including references and bios.
Paper submission will be done electronically through EasyChair, selecting the track "The First IEEE International Workshop on Workshop on Assured Autonomy, AI and Machine Learning".
Each paper must be submitted as a single Portable Document Format (PDF) file. All fonts must be embedded. We also strongly recommend you print the file and review it for integrity (fonts, symbols, equations etc.) before submitting it. A defective printing of your paper can undermine its chance of success. Please take a note of the following:
Authors of accepted papers will be expected to supply electronic versions of their papers and are encouraged to supply source code and raw data to help others replicate and better understand their results.
Phil Laplant (Chair), Penn State
Ben Amaba, Clarifai
Rick Kuhn, NIST
Feras A. Batarseh | Virginia Tech |
Jagan Chandrasekaran | Virginia Tech |
Ken Costello | NASA |
Joanna DeFranco | Penn State |
Laura Freeman | Virginia Tech |
Alwyn E. Goodloe | NASA - Langley |
Mohamad Kassab | Penn State |
Nir Kshreti | University of North Carolina-Greensboro |
Lady Linares | Johnson & Johnson |
Dejan Milojicic | HP |
Kishor Trivedi | Duke University |
Jeff Voas | NIST |
Eric Wong | University of Texas, Dallas |
Matt Zeiler | Clarafai |