ABSTRACT
Exploiting entropy is not going to untie people's cords. But it will help people solve some problems in choosing distributions. Statistical models force many choices upon people. Some of these choices are distributions that represent uncertainty. Choosing the distribution with the largest entropy means spreading probability as evenly as possible, while still remaining consistent with anything people think they know about a process. In the context of choosing a prior, it means choosing the least informative distribution consistent with any partial scientific knowledge they have about a parameter. The principle of maximum entropy applies this measure of uncertainty to the problem of choosing among probability distributions. Perhaps the simplest way to state the maximum entropy principle is: The distribution that can happen the most ways is also the distribution with the biggest information entropy. To begin to understand maximum entropy, forget about information and probability theory for the moment.
