ABSTRACT
To study those real-world phenomena in which systems evolve randomly, we need probabilistic models rather than deterministic ones. Such systems are usually studied as a function of time, and their mathematical models are called stochastic models. The building blocks of stochastic models are stochastic processes, defined as sets of random variables {Xn : n ∈ I} for a finite or countable index set I , or
{ X(t) : t ∈ T} for an uncountable
index set T . For example, let Xn be the number of customers served in a bank at the end of the nth working day. Then {Xn : n = 1, 2, . . .} is a stochastic process. It is called a discrete-time stochastic process since its index set, I = {1, 2, . . .}, is countable. As another example, let X(t) be the sum of the remaining service times of all customers being served in a bank at time t. Then
{ X(t) : t ≥ 0} is a stochastic process, and since its index
set, T = [0,∞), is uncountable, it is called a continuous-time stochastic process. The set of all possible values of Xn’s in the discrete-time case and X(t)’s in the continuous-time case is called the state space of the stochastic process, and it is usually denoted by S . The state space for the number of customers served in a bank at the end of the nth working day is S = {0, 1, 2, . . .}. The state space for the sum of the remaining service times of all customers being served in a bank at time t is S = [0,∞). Other examples of stochastic processes follow.
