• About
  • Policies
  • What is open access
  • Library
  • Contact
Advanced search
      View Item 
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Industrial Engineering
      • View Item
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Industrial Engineering
      • View Item
      JavaScript is disabled for your browser. Some features of this site may not work without it.

      Reward-rate maximization in sequential identification under a stochastic deadline

      Thumbnail
      View / Download
      2.8 Mb
      Author(s)
      Dayanık, S.
      Yu, A. J.
      Date
      2013
      Source Title
      SIAM Journal on Control and Optimization
      Print ISSN
      0363-0129
      Electronic ISSN
      1095-7138
      Volume
      51
      Issue
      4
      Pages
      2922 - 2948
      Language
      English
      Type
      Article
      Item Usage Stats
      143
      views
      121
      downloads
      Abstract
      Any intelligent system performing evidence-based decision making under time pressure must negotiate a speed-accuracy trade-off. In computer science and engineering, this is typically modeled as minimizing a Bayes-risk functional that is a linear combination of expected decision delay and expected terminal decision loss. In neuroscience and psychology, however, it is often modeled as maximizing the long-term reward rate, or the ratio of expected terminal reward and expected decision delay. The two approaches have opposing advantages and disadvantages. While Bayes-risk minimization can be solved with powerful dynamic programming techniques unlike reward-rate maximization, it also requires the explicit specification of the relative costs of decision delay and error, which is obviated by reward-rate maximization. Here, we demonstrate that, for a large class of sequential multihypothesis identification problems under a stochastic deadline, the reward-rate maximization is equivalent to a special case of Bayes-risk minimization, in which the optimal policy that attains the minimal risk when the unit sampling cost is exactly the maximal reward rate is also the policy that attains maximal reward rate. We show that the maximum reward rate is the unique unit sampling cost for which the expected total observation cost and expected terminal reward break even under every Bayes-risk optimal decision rule. This interplay between reward-rate maximization and Bayesrisk minimization formulations allows us to show that maximum reward rate is always attained. We can compute the policy that maximizes reward rate by solving an inverse Bayes-risk minimization problem, whereby we know the Bayes risk of the optimal policy and need to find the associated unit sampling cost parameter. Leveraging this equivalence, we derive an iterative dynamic programming procedure for solving the reward-rate maximization problem exponentially fast, thus incorporating the advantages of both the reward-rate maximization and Bayes-risk minimization formulations. As an illustration, we will apply the procedure to a two-hypothesis identification example.
      Keywords
      Bayes-risk minimization
      Dynamic programming
      Reward-rate maximization
      Sequential multihypothesis testing
      Speed-accuracy trade off
      Bayes-risk minimization
      Computer science and engineerings
      Dynamic programming techniques
      Identification problem
      Iterative Dynamic Programming
      Multi-hypothesis testing
      Optimal decision-rule
      Trade off
      Costs
      Dynamic programming
      Economic and social effects
      Equivalence classes
      Intelligent systems
      Inverse problems
      Iterative methods
      Optimization
      Stochastic systems
      Decision making
      Permalink
      http://hdl.handle.net/11693/26706
      Published Version (Please cite this version)
      http://dx.doi.org/10.1137/100818005
      Collections
      • Department of Industrial Engineering 702
      • Department of Mathematics 653
      Show full item record

      Browse

      All of BUIRCommunities & CollectionsTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartmentsThis CollectionTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartments

      My Account

      LoginRegister

      Statistics

      View Usage StatisticsView Google Analytics Statistics

      Bilkent University

      If you have trouble accessing this page and need to request an alternate format, contact the site administrator. Phone: (312) 290 1771
      © Bilkent University - Library IT

      Contact Us | Send Feedback | Off-Campus Access | Admin | Privacy