Operations & Supply Chain Management

This paper provides a new approach for solving a wide class of Markov decision problems including problems in which the space is general and the system can be continuously controlled. The optimality criterion is the long-run average cost per unit time. We decompose the decision processes into a common underlying stochastic process and a sequence of interventions so that the decision processes can be embedded upon a reduced set of states.

In a preceding paper we have introduced a new approach for solving a wide class of Markov decision problems in which the state-space may be general and the system may be continuously controlled. The criterion is the average cost. This paper discusses two applications of this approach. The first application concerns a house-selling problem in which a constructor builds houses which may be sold at any stage of the construction and potential customers make offers depending on the stage of the construction.

Service systems’ behavior can be affected by multiple factors. In the case of intensive care units (ICUs), which admit patients from four primary loci (the emergency department (ED), scheduled patients, planned transfers from other ICUs, and unplanned transfers), it is known that admission rates of some patients decrease as occupancy increases. It is also known that, for at least some conditions, ICU admission is not just a function of patients’ illness, and that a significant proportion of the variation in ICU admission rates is due to hospital, not patient, factors.

Problem definition: In many service systems, individual server’s workload can have a substantial impact on service time and quality. Such effects are particularly important in healthcare systems which often operate under resource and time constraints. In much of the literature, this has been primarily considered at the system level rather than the individual level. In this study, we investigate this relationship in the context of cardiac surgery, i.e., how surgery duration and patient outcomes are affected by individual surgeon’s daily workload.

Determining emergency department (ED) nurse staffing decisions to balance the quality of service and staffing cost can be extremely challenging, especially when there is a high level of uncertainty in patient-demand. Increasing data availability and continuing advancements in predictive analytics provide an opportunity to mitigate demand uncertainty by utilizing demand forecasts.

Rationing of healthcare resources is a challenging decision that policy makers and providers may be forced to make during a pandemic, natural disaster, or mass casualty event. Well-defined guidelines to triage scarce life-saving resources must be designed to promote transparency, trust and consistency. To facilitate buy-in and use during high stress situations, these guidelines need to be interpretable and operational.

We study a stylized revenue maximization problem for a provider of cloud computing services, where the service provider (SP) operates an infinite capacity system in a market with heterogeneous customers with respect to their valuation and congestion sensitivity. The SP offers two service options: one with guaranteed service availability, and one where users bid for resource availability and only the “winning” bids at any point in time get access to the service.