In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study artificial neural networks whose internal layers preserve a continuous symmetry. We study the phase structure of the model and demonstrate that this symmetry can be spontaneously broken and therefore support analogous Goldstone-like degrees of freedom. We demonstrate both theoretically and empirically that in the spontaneously broken phase the Goldstone modes allow information to flow coherently through the layers of the network, leading to advantages in training very deep networks and in long-sequence modelling tasks. 

After Nabil Iqbal’s talk, at 14.00, we will have a kick-off meeting of the DIEP journal club on foundations of AI. Please feel free to join if you are interested in.

If you wish to attend this seminar online, please send an email to m.t.pham@uva.nl to receive the zoom-link.