Seonglae ChoAdds a Lagrange multiplier at each layer to accumulate prediction error and feed it back into the local activation/weight updates. At convergence, effectively becomes the BP adjoint/gradient signal.
In linear networks it recovers the exact BP gradient at equilibrium, and in nonlinear MLPs it nearly matches BP performance under the same inference budget — with the largest gains in the deep + narrow regime where standard PC breaks down. Instead of error diffusing slowly as in PC, the primal-dual dynamics propagate it to deep layers quickly, like a ballistic wave. The downsides: the multiplier state roughly doubles activation memory, and experiments are still limited to MNIST/Fashion-MNIST + residual MLP + 1 epoch.
Augmented Lagrangian Predictive Coding: training 1000-layer networks without backpropagation
A local alternative to backpropagation. PC-ALM trains residual MLPs up to 1000 layers, nearly matching backprop's performance despite using only layer-local dynamics. PC-ALM equips each layer with a feedback control dynamical system that distributes and propagates supervision credit throughout a network.
https://pub.sakana.ai/pc-alm/
arxiv.org
https://arxiv.org/pdf/2605.31022