Bhatnagar, Shalabh and Lakshmanan, K (2012) An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes. In: JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 153 (3). pp. 688-708.
MAT_STA_153-3_688-708_2012.pdf - Published Version
Restricted to Registered users only
Download (870Kb) | Request a copy
We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.
|Item Type:||Journal Article|
|Additional Information:||Copy right for this article belongs to Springer Science|
|Keywords:||ASYNCHRONOUS STOCHASTIC APPROXIMATIONS;NETWORKS|
|Department/Centre:||Division of Electrical Sciences > Computer Science & Automation (Formerly, School of Automation)|
|Date Deposited:||17 Jul 2012 11:22|
|Last Modified:||17 Jul 2012 11:22|
Actions (login required)