Seonglae ChoInformational Entropy
A characteristic value that represents the shape of probability distribution and amount of information. It measures of information content; the number of bits actually required to store data; How random it is, How broad it is (Uniform distribution has maximum entropy)
Average Information Content
The entropy of a probability distribution can be interpreted as a measure of uncertainty, or lack of predictability
Information Content of Individual Events
Information content must be additive, meaning the total information content equals the sum of individual event information
Boltzmann–Gibbs Entropy = Shannon entropy (up to constant scaling) in Information Theory
