Singular Learning Theory

Regular statistical models, which only work well when the parameter space is smooth and identifiable. Deep learning models like neural networks are singular models, where the parameter space is non-smooth, breaking conventional inference theories (such as Laplace approximation). The prediction error bound is expressed using learning coefficient instead of Kolmogorov dimension, but it necessarily requires the

iid assumption