Predictive coding as an emergent phenomenon

Abstract

Predictive coding is a prominent theory in neuroscience for understanding neural processing in the brain. The theory states that the brain constantly predicts incoming sensory stimuli and improves its predictions by calculating the error between the predictions and the actual stimuli. It is still an open question whether this theory requires hard-wired neural circuitry or whether it could be an emergent phenomenon. Here, we explore whether predictive coding emerges in recurrent neural networks that receive predictable stimuli in the form of image sequences, when they are trained in an unsupervised way to reduce network activity. We show that these networks can successfully learn to predict future stimuli. Without imposing architectural constraints we see that a natural distinction emerges between units that calculate prediction errors and units that generate predictions. These findings suggest that these properties of predictive coding may be emergent phenomena in neural networks that efficiently encode predictable stimuli.