ABSTRACT
The average length of a source code is at least equal to the entropy of the source.
Information can be reliably sent over a noisy channel at any rate below the channel capacity.
A code C for a source with n states is a sequence w1, . . . , wn of binary strings, none the prefix of another. The wi are the words of the code. We may now write our source as
x1 x2 . . . xnp1 p2 . . . pn w1 w2 . . . wn
.
