Image classifiers in automated vehicles: An evaluation framework for robustness

Abstract

This thesis reports the development and evaluation of a general safety evaluation procedure for image classifiers that can be applied to automated vehicles. Artificial Intelligence (AI) systems are widely used in automated vehicles, for example to process camera inputs. For the deployment of safety-critical systems, their safety assessment is of critical importance. However, current type approval and safety tests in cars are not suitable for evaluating machine learning components due to their black-box nature and general instability. To fill this research gap, I developed a methodological evaluation framework that tests the robustness of image classifiers. The testing procedure consists of two separate tests. The first test evaluates the ability to function in challenging and novel conditions. The second test evaluates the system’s behavior when confronted with out-of-distribution (OOD) examples. Evaluation measures for these tests are accuracy, separability of OOD examples, and overall quality of certainty estimates. I tested the framework on two different classifier architectures: a baseline Deep Convolutional Neural Network (DCNN) and Monte Carlo (MC) Dropout, which is used due to its more accurate certainty predictions. Results of the evaluation showed that different image transformations provide critical tests for both evaluating the classifier on challenging in-distribution images and realistic OOD examples. In this way, the evaluation framework can be used to identify weak points in the classifier’s robustness and show their unsuitability for safety-critical applications. Comparing the two classifiers with the evaluation shows the better generalization of MC Dropout, which was not detectable when evaluating with a traditional test set. The evaluation can ultimately be used to define acceptance criteria for the safe use of AI systems, for example, in type-approval for automated vehicles.