Mohan Babu, K and Bhatnagar, Shalabh (2007) Two-timescale Q-learning Algorithms with an Application to Routing in Networks. In: International Conference on Advances in Control and Optimization of Dynamical Systems, ACODS- Bangalore, Feb. 2007, Bangalore.
10.1.1.130.7691.pdf - Published Version
We propose two variants of the Q-learning algorithm that (both) use two timescales. One of these updates Q-values of all feasible state-action pairs at each instant while the other updates Q-values of states with actions chosen according to the ‘current ’ randomized policy updates. A sketch of convergence of the algorithms is shown. Finally, numerical experiments using the proposed algorithms for routing on different network topologies are presented and performance comparisons with the regular Q-learning algorithm are shown.
|Item Type:||Conference Paper|
|Keywords:||Q-learning based algorithms;Markov decision processes;two- timescale stochastic approximation;simultaneous perturbation stochastic approximation (SPSA);normalized Hadamard matrices|
|Department/Centre:||Division of Electrical Sciences > Computer Science & Automation (Formerly, School of Automation)|
|Date Deposited:||17 Oct 2011 05:22|
|Last Modified:||17 Oct 2011 05:22|
Actions (login required)