NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Collections About News Help Login
Back to Results
Improving Trust in Deep Neural Networks with Nearest NeighborsDeep neural networks are used increasingly for perception and decision-making in UAVs. For example, they can be used to recognize objects from images and decide what actions the vehicle should take. While deep neural networks can perform very well at complex tasks, their decisions may be unintuitive to a human operator. When a human disagrees with a neural network prediction, due to the black box nature of deep neural networks, it can be unclear whether the system knows something the human does not or whether the system is malfunctioning. This uncertainty is problematic when it comes to ensuring safety. As a result, it is important to develop technologies for explaining neural network decisions for trust and safety. This paper explores a modification to the deep neural network classification layer to produce both a predicted label and an explanation to support its prediction. Specifically, at test time, we replace the final output layer of the neural network classifier by a k-nearest neighbor classifier. The nearest neighbor classifier produces 1) a predicted label through voting and 2) the nearest neighbors involved in the prediction, which represent the most similar examples from the training dataset. Because prediction and explanation are derived from the same underlying process, this approach guarantees that the explanations are always relevant to the predictions. We demonstrate the approach on a convolutional neural network for a UAV image classification task. We perform experiments using a forest trail image dataset and show empirically that the hybrid classifier can produce intuitive explanations without loss of predictive performance compared to the original neural network. We also show how the approach can be used to help identify potential issues in the network and training process.
Document ID
20200000328
Acquisition Source
Ames Research Center
Document Type
Conference Paper
Authors
Lee, Ritchie (Stinger Ghaffarian Technologies Inc. (SGT Inc.) Moffett Field, CA, United States)
Clarke, Justin (Stinger Ghaffarian Technologies Inc. (SGT Inc.) Moffett Field, CA, United States)
Agogino, Adrian K. (NASA Ames Research Center Moffett Field, CA, United States)
Giannakopoulou, Dimitra (NASA Ames Research Center Moffett Field, CA, United States)
Date Acquired
January 16, 2020
Publication Date
January 6, 2020
Subject Category
Cybernetics, Artificial Intelligence And Robotics
Report/Patent Number
ARC-E-DAA-TN76279
Meeting Information
Meeting: SciTech Forum
Location: Orlando, FL
Country: United States
Start Date: January 6, 2020
End Date: January 10, 2020
Sponsors: American Institute of Aeronautics and Astronautics (AIAA)
Funding Number(s)
CONTRACT_GRANT: NNA14AA60C
Distribution Limits
Public
Copyright
Public Use Permitted.

Related Records

There are no records associated with this record.
No Preview Available