Homework 4
MAS 442 Spring 2020
Due: 11:59 PM Apr 6
Problem 1: One of the most widely used policies to manage inventory subject to random
demand is the “(s, S)” policy. When the inventory level at the end of the day falls at s or
below, we order enough inventory to bring it back at S. For simplicity, we assume that there
is no lead time.
Suppose now that an electronics store sells a video game console and uses an inventory
policy with s = 2 and S = 6. That is, if the inventory level at the end of the day is either 0 or
1 or 2, then they order enough new units so that the total inventory level at the beginning of
the next day is 6. Let Xn denote the inventory level at the end of day n. The daily demands
are independent random variables following the Binomial (4, 0.6) distribution. Today’s (day
0) end-of-day inventory is 3.
(a) Model the inventory level at the end of each day (day n) as a Markov Chain. Clearly
define S, α(0), P . Also present the p.m.f of the demand. (20 Points)
(b) Compute the expected inventory level at the end of the day 5. (5 Points)
(c) Compute the probability of a stock-out during day 6. (5 Points)
1
Problem 2: (Ehrenfest model) Gas molecules move about randomly in a box which is
divided into two halves symmetrically by a partition. A hole is made in the partition.
Suppose there are 20 molecules in the box. Think of the partitions as two urns (urn 0 and
1) containing balls labeled 1 through 20. Molecular motion can be modeled by choosing a
number between 1 and 20 at random and moving the corresponding ball from the urn it
is presently into the other. This is a historically important physical model introduced by
Ehrenfest in the early days of statistical mechanics to study thermodynamic equilibrium. The
Ehrenfest model has also been applied to study the dynamics of social network formations.
Let Xn denote the number of molecules at the left partition of the box (urn 0) after n
transitions.
(a) Draw a transition diagram of the process. You can draw this for the first three and last
three states, and leave . . . in between. Is {Xn : n ≥ 0} irreducible (10 Points)
(b) Argue that {Xn : n ≥ 0} is a DTMC. In particular, argue that Xn is a finite-space
random walk. Clearly define S, and P . For P , complete the following matrix. (20
Points)
P =
