About

Congratulations to authors Maximilian Weimar, Lukas M. Rachbauer, and Stefan Rotter (TU Wien), as well as Ilya Starshynov and Daniele Faccio (University of Glasgow), Linara Adilova (Ruhr University Bochum), and Dorian Bouchet (Université Grenoble Alpes) on their recent paper published in the Physical Review X journal titled “Fisher Information Flow in Artificial Neural Networks”.

See the Quantum Physics special interest group for more related news.

Summary

Artificial neural networks have become powerful tools for analyzing data, yet we still know little about how they process and organize information internally. This limitation is especially important in scientific tasks such as parameter estimation, where extracting precise information from noisy data is essential. In our work, we introduce a new technique that makes it possible to track how information moves through a neural network during training.

We focus on the Fisher information, a quantity that measures the information content within a set of data. Normally, calculating the Fisher information inside a neural network is feasible only for very simple models. To overcome this, we develop an algorithm that determines the Fisher information content in each of the layers of a network. Using this approach, we study a network trained to estimate the position of an object from noisy camera images, and we directly observe how Fisher information flows and transforms during the learning process.

Our findings provide new insight into how neural networks manage to compress high-dimensional input data into a small set of meaningful outputs. This understanding can guide the design of neural networks that preserve essential information while discarding irrelevant noise. Looking ahead, these ideas could be applied not only to improve data compression and network design but also to physical neural systems and neuromorphic devices, where tracking and controlling information flow will be key for building efficient hardware and training algorithms.