Strategy

This paper considers the solution of Markov decision problems whose parameters can be obtained only via approximating schemes, or where it is computationally preferable to approximate the parameters, rather than employing exact algorithms for their computation. Various models are presented in which this situation occurs. Furthermore, it is shown that a modified value-iteration method may be employed, both for the discounted version and for the undiscounted version of the model, in order to solve the optimality equation and to find optimal policies.

Recent papers have shown that Π∞k = 1 P(k) = limm→∞ (P(m) ... P(1)) exists whenever the sequence of stochastic matrices {P(k)}∞k = 1 exhibits convergence to an aperiodic matrix P with a single subchain (closed, irreducible set of states). We show how the limit matrix depends upon P(1).

For the multi-server queue with Poisson arrivals and general service times we present various approximations for the steady-state probabilities of the queue size. These approximations are computed from numerically stable recursion schemes which can be easily applied in practice. Numerical experience reveals that the approximations are very accurate with errors typically below 5%. For the delay probability the various approximations result either into the widely used Erlang delay probability or into a new approximation which improves in many cases the Erlang delay probability approximation.

Analysis of state and privately owned enterprises in industrialized market economies leads to the identification of differences in objectives and strategy between the two enterprise types. A series of propositions is developed that contrasts the behavior of state and privately owned corporations.

We consider the Policy Iteration Algorithm for undiscounted Markov Renewal Programs. Previous specifications of the policy evaluation part of this algorithm all required the analysis of the chain structure for each policy generated. The purpose of this paper is to provide a unique specification of the value sectors as well as an anticycling rule which avoids parsing the transition probability matrices into their subchains.

This paper considers two-person zero-sum sequential games with finite state and action spaces. We consider the pair of functional equations (f.e.) that arises in the undiscounted infinite stage model, and show that a certain class of successive approximation schemes is guaranteed to converge to a solution pair whenever an equilibrium policy with respect to the average return per unit time criterion (AEP) exists. Existence of the latter thus implies the existence of a solution to this pair of f.e. whereas the converse implication is shown only to hold under special circumstances.

This paper examines the hypothesis that the expected rate of return to speculation in the forward foreign exchange market is zero; that is, the logarithm of the forward exchange rate is the market's conditional expectation of the logarithm of the future spot rate. A new computationally tractable econometric methodology for examining restrictions on a k-step-ahead forecasting equation is employed. Using data sampled more finely than the forecast interval, we are able to reject the simple market efficiency hypothesis for exchange rates from the 1970s and the 1920s.

This paper presents a simple and computationally tractable method which recursively computes the stationary probabilities of the queue size in an M/G/1 queueing system with variable service rate. For each service two possible service types are available and the service rule is characterized by two switch-over levels. The computational approach discussed in this paper can be applied to a variety of queueing problems.