Online learning in structured Markov decision processes

buir.advisorTekin, Cem
dc.contributor.authorAkbarzadeh, Nima
dc.date.accessioned2017-08-07T09:41:12Z
dc.date.available2017-08-07T09:41:12Z
dc.date.copyright2017-07
dc.date.issued2017-07
dc.date.submitted2017-08-03
dc.descriptionCataloged from PDF version of article.en_US
dc.descriptionIncludes bibliographical references (leaves 80-86).en_US
dc.description.abstractThis thesis proposes three new multi-armed bandit problems, in which the learner proceeds in a sequence of rounds where each round is a Markov Decision Process (MDP). The learner's goal is to maximize its cumulative reward without any a priori knowledge on the state transition probabilities. The rst problem considers an MDP with sorted states and a continuation action that moves the learner to an adjacent state; and a terminal action that moves the learner to a terminal state (goal or dead-end state). In this problem, a round ends and the next round starts when a terminal state is reached, and the aim of the learner in each round is to reach the goal state. First, the structure of the optimal policy is derived. Then, the regret of the learner with respect to an oracle, who takes optimal actions in each round is de ned, and a learning algorithm that exploits the structure of the optimal policy is proposed. Finally, it is shown that the regret either increases logarithmically over rounds or becomes bounded. In the second problem, we investigate the personalization of a clinical treatment. This process is modeled as a goal-oriented MDP with dead-end states. Moreover, the state transition probabilities of the MDP depends on the context of the patients. An algorithm that uses the rule of optimism in face of uncertainty is proposed to maximize the number of rounds in which the goal state is reached. In the third problem, we propose an online learning algorithm for optimal execution in the limit order book of a nancial asset. Given a certain amount of shares to sell and an allocated time to complete the transaction, the proposed algorithm dynamically learns the optimal number of shares to sell at each time slot of the allocated time. We model this problem as an MDP, and derive the form of the optimal policy.en_US
dc.description.statementofresponsibilityby Nima Akbarzadeh.en_US
dc.embargo.release2019-07-21
dc.format.extentxi, 86 leaves : charts (some color) ; 29 cm.en_US
dc.identifier.itemidB156076
dc.identifier.urihttp://hdl.handle.net/11693/33535
dc.language.isoEnglishen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.subjectOnline Learningen_US
dc.subjectMarkov Decision Processen_US
dc.subjectMulti-armed Banditsen_US
dc.subjectReinforcement Learningen_US
dc.subjectDynamic Programmingen_US
dc.subjectClinical Decision Makingen_US
dc.subjectLimit Order Booken_US
dc.titleOnline learning in structured Markov decision processesen_US
dc.title.alternativeÖzel yapılı Markov karar süreçlerinde çevrimiçi öğrenmeen_US
dc.typeThesisen_US
thesis.degree.disciplineElectrical and Electronic Engineering
thesis.degree.grantorBilkent University
thesis.degree.levelMaster's
thesis.degree.nameMS (Master of Science)

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
thesis.pdf
Size:
1.31 MB
Format:
Adobe Portable Document Format
Description:
Full printable version

License bundle

Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description: