The importance of nonlinearity for physical neural networks

Abstract

This thesis investigates the effects of nonlinearity on physical neural networks as part of the Vaporware network project. The Vaporware network, which has not been implemented yet, will do computations which are done by neural networks using the propagation of laser light. For large enough networks this will speed up the evaluation of these neural networks. In this network the role of a nonlinear activation function is taken on by propagating the light through dense atomic vapour. To get nonlinear effects the frequency of the light has to be near an absorption line and the intensity high enough that the vapour is partially saturated. We simulated the Vaporware model numerically using Keras to see how different values of the susceptibility affected the accuracy of the network. A susceptibility of zero corresponds to propagation in a vacuum and this corresponds to having no activation function. As expected this showed the lowest accuracy for the MNIST dataset. We found a maximum accuracy of 96%. We found that the sign of the susceptibility only matters to a small extent for the accuracy.