Triangle Seminars
Thursday, 17 Sep 2026
Deep-layered machines have a built-in Occam's razor
📍 London
Thomas Fink
(LIMS)
Abstract:
Many complex systems map vast numbers of microscopic descriptions onto far fewer macroscopic outcomes. Surprisingly, these input-output maps strongly favor simple outputs. We analyze an exactly solvable deep-layered Boolean machine and prove that increasing depth drives the distribution of the output \(F\) toward exponential decay in Kolmogorov complexity, \(P(F) \propto 2^{-K(F)}\), before ultimately collapsing onto the two constant functions. Our results show that hierarchical composition alone can generate an intrinsic Occam's razor, providing an analytical explanation for simplicity bias in learning and evolution.
Many complex systems map vast numbers of microscopic descriptions onto far fewer macroscopic outcomes. Surprisingly, these input-output maps strongly favor simple outputs. We analyze an exactly solvable deep-layered Boolean machine and prove that increasing depth drives the distribution of the output \(F\) toward exponential decay in Kolmogorov complexity, \(P(F) \propto 2^{-K(F)}\), before ultimately collapsing onto the two constant functions. Our results show that hierarchical composition alone can generate an intrinsic Occam's razor, providing an analytical explanation for simplicity bias in learning and evolution.
Posted by: Kymani Armstrong-Williams