An iterative decision-making scheme for Markov decision processes and its application to self-adaptive systems

Su, Guoxin, Chen, Taolue, Feng, Yuan, Rosenblum, David S. and Thiagarajan, P. S. (2016) An iterative decision-making scheme for Markov decision processes and its application to self-adaptive systems. Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2-8, 2016, Proceedings. In: 19th International Conference Fundamental Approaches to Software Engineering (FASE 2016), 02-08 Apr 2016, Eindhoven, The Netherlands. pbk-ISBN 9783662496640, e-ISBN 9783662496657. ISSN 0302-9743 [Conference or Workshop Item] (doi:10.1007/978-3-662-49665-7_16)

[img] PDF - Final accepted version (with author's formatting)
Restricted to Repository staff and depositor only

Download (487kB)


Software is often governed by and thus adapts to phenomena that occur at runtime. Unlike traditional decision problems, where a decision-making model is determined for reasoning, the adaptation logic of such software is concerned with empirical data and is subject to practical constraints. We present an Iterative Decision-Making Scheme (IDMS) that infers both point and interval estimates for the undetermined transition probabilities in a Markov Decision Process (MDP) based on sampled data, and iteratively computes a confidently optimal scheduler from a given finite subset of schedulers. The most important feature of IDMS is the flexibility for adjusting the criterion of confident optimality and the sample size within the iteration, leading to a tradeoff between accuracy, data usage and computational overhead. We apply IDMS to an existing self-adaptation framework Rainbow and conduct a case study using a Rainbow system to demonstrate the flexibility of IDMS.

Item Type: Conference or Workshop Item (Paper)
Additional Information: Published as a chapter in: Fundamental Approaches to Software Engineering, Volume 9633 of the series Lecture Notes in Computer Science, pp 269-286
Research Areas: A. > School of Science and Technology > Computer Science > Foundations of Computing group
Item ID: 19199
Notes on copyright: The final authenticated version is available online at
Useful Links:
Depositing User: Taolue Chen
Date Deposited: 12 Apr 2016 09:59
Last Modified: 12 Jun 2021 18:14

Actions (login required)

View Item View Item


Activity Overview
6 month trend
6 month trend

Additional statistics are available via IRStats2.