The average length of a source code is at least equal to the entropy of the source.

Information can be reliably sent over a noisy channel at any rate below the channel capacity.

A code C for a source with n states is a sequence w1, . . . , wn of binary strings, none the prefix of another. The wi are the words of the code. We may now write our source as

 x1 x2 . . . xnp1 p2 . . . pn w1 w2 . . . wn

  .