Master thesis research conducted at the Systems, Information, Learning & Society (SILS) Lab, investigating predictive coding mechanisms in neural networks. This project explores how predictive coding principles from neuroscience can be implemented in artificial neural networks to improve learning and inference.
Abstract
We study predictive coding (PC) on arbitrary directed graphs, unifying classification, generation, and occlusion within one model. We evaluate hierarchical, fully connected, and stochastic block models (SBMs) on MNIST-scale tasks. We show that shallow PC models are competitive for classification, but deeper models amplify training instability and update divergence compared with traditional backpropagation (BP). Generative PC models querying via clamped labels yield plausible yet weakly controllable MNIST image samples and are sensitive to hyperparameters. Fully connected graphs favor shortcut credit paths that collapse into near direct (sensory–label) mappings, bypassing rich latent structure. In contrast, clustered sparse topologies discourage direct sensory-to-label bypasses and scale to larger node counts without degrading accuracy. Overall, PC is promising for parallel, local learning on sparse graphs, but remains limited by sensitivity to initialization, step sizes, and topology.
Keywords: predictive coding, inference-learning, graph topology