ABSTRACT
Information signals, such as television, wireless, voice and data, which are han-
dled by communication systems are by nature probabilistic. The same is true of
automatic control systems, large radar with antennae, airplanes flying in gusts of
wind, or ships sailing in stormy seas. Each situation like this is characterized by a
“stochastic model.” In this chapter, we present the probability definitions, random
variables, probability distributions and density functions, and other concepts such
as the mean and variance of a random variable. The Central Limit Theorem and var-
ious other inequalitites are derived. We shall discuss the representation of Wiener
or Brownian Motion via random walk. The geometric representation of random
variables and stochastic processes in Hilbert space is also presented. Various types
of random processes such as stationary, ergodic, etc., are described along with the
ideas of auto-and cross-correlation of system inputs and system outputs. This is fol-
lowed by the series representation of stochastic processes via orthogonal functions
and Karhunen-Loeve expansion. Wiener and Kalman filters minimizing the effect
of additive noise in signals are also derived. Since our audience is engineers and
physical scientists, we have tried to find a good balance between mathematical rigor
and understandable proofs.
