Minimum Description Length
Selects models that explain data in the most concise and efficient way possible.
A principle that minimizes both the error (by minimizing the size needed to describe data using the model) and model complexity (the size needed to describe the model itself).