ABSTRACT
This chapter presents a conceptual introduction to generalized linear models and the principle of maximum entropy. The posterior distribution has the greatest entropy relative to the prior among all distributions consistent with the assumed constraints and the observed data. Variables with different constraints get different maximum entropy distributions, but the underlying principle remains the same. Entropy counts up the number of different ways a process can produce a particular result, according to our assumptions. To build a regression model from any of the exponential family distributions is just a matter of attaching one or more linear models to one or more of the parameters that describe the distribution’s shape.
