Browsing by Subject "Dynamic programming."

Now showing 1 - 8 of 8

Open Access
Approximate dynamic programming approach for sequential change diagnosis problem
(2013) Akbulut, Elif
We study sequential change diagnosis problem which is the combination of change diagnosis and multi-hypothesis testing problem. One observes a sequence of independent and identically distributed random variables. At a sudden disorder time, the probability distribution of the random variables change. The disorder time and its cause are unavailable to the observer. The problem is to detect this abrupt change in the distribution of the random process as quickly as possible and identify its cause as accurately as possible. Dayanık et al. [Dayanık, Goulding and Poor, Bayesian sequential change diagnosis, Mathematics of Operations Research, vol. 45, pp. 475-496, 2008] reduce the problem to a Markov optimal stopping problem and provide an optimal sequential decision strategy. However, only a small subset of the problems is computationally feasible due to curse of dimensionality. The subject of this thesis is to search for the means to overcome the curse of dimensionality. To this end, we propose several approximate dynamic programming algorithms to solve large change diagnosis problems. On several numerical examples, we compare their performance against the performance of optimal dynamic programming solution.
Open Access
Bounded rationality and learning in dynamic programming environments
(2001) Erdem, Mahmut
The purpose of this thesis is to explain “excess sensitivity” puzzle observed in consumption behavior an alternative way. By deviating from full optimization axiom, in a dynamic extension of Arthur’s stochastic decision model, it was observed that a tendency of excess consumption following temporary income shock prevails. Another main technical contribution achieved in this thesis is in modelling behavior and learning in intertemporal decision problems. In particular, an extension of Arthur’s type of behavior to dynamic situations and comparison of the corresponding values with those of Bellman’s dynamic programming solution is achieved. Moreover it was shown by using stochastic approximation theory that classifier systems learning ends up at the ‘strength’ values corresponding to the Arthur’s value function.
Open Access
Comparative evaluation of spectrum allocation policies for dynamic flexgrid optical networks
(2013) Yümer, Ramazan
A novel class-based first-fit spectrum allocation policy is proposed for dynamic Flexgrid optical networks. The effectiveness of the proposed policy is compared against the first-fit policy for single-hop and multi-hop scenarios. Event-based simulation technique is used for testing the spectrum allocation policies under both Fixed Routing and Fixed Alternate Routing algorithms with two shortest paths. Throughput is shown to be consistently improved under the proposed policy with gains of up to 15 % in certain scenarios.
Open Access
Development of a supervisory controller for energy management problems
(2011) Akgün, Emre
Multi energy source systems, like hybrid electric vehicles in automotive industry, started to attract attention as a remedy for the greenhouse gas emission problem. Although their environmental performances are better than conventional technologies such as the case of gasoline vehicles versus hybrid electric vehicles in automotive industry, their operational management can be challenging due to their increased complexity. One of these challenges is the operational management of the energy flow among these multiple sources and sinks which in this context referred as the energy management problem. In this thesis, a supervisory controller is developed to operate at a residential environment with multiple energy sources. First, dynamic optimization techniques are applied to the available mathematical models of the multi-energy sources to create a non-causal optimal controller. Then, a set of implementable rules are extracted by analyzing the optimal trajectories resulted from the dynamic optimization to create a causal supervisory controller. Several simulations are conducted with Matlab/Simulink to validate the developed controller. The supervisory controller achieves not only a daily cost reduction between 6-7.5% compared to conventional energy infrastructure used in residential areas but also performs 2% better than heuristic control techniques available in the literature. Another simulation study is conducted, with different demand cycles, for verification of the controller. Although its performance reduces as expected, it still performs 1% better than heuristic control strategies. In the final part of this thesis, the formulation used in the residential problem which was originally adopted from an example in automotive industry, is generalized so that it can be used in all types of energy management problems. Finally, for exemplary purposes, a formulation for energy management problem in mobile devices is created by using the developed generic formulation.
Open Access
Dynamic capacity adjustment for virtual-path based networks using neuro-dynamic programming
(2003) Şahin, Cem
Dynamic capacity adjustment is the process of updating the capacity reservation of a virtual path via signalling in the network. There are two important issues to be considered: bandwidth (resource) utilization and signaling traffic. Changing the capacity too frequently will lead to efficient usage of resources but has a disadvantage of increasing signaling traffic among the network elements. On the other hand, if the capacity is adjusted for the highest possible value and kept fixed for a long time period, a significant amount of bandwidth will be wasted when the actual traffic rate is small. We proposed two formulations for dynamic capacity adjustment problem. In the first formulation cost parameters are assigned for bandwidth usage and signalling, optimal solutions are reached for different values of these parameters. In the second formulation, our aim is to maximize the bandwidth efficiency with a given signaling requirement. In this formulation, a leaky bucket counter is used in order to regulate the signaling rate. We used dynamic programming and neuro-dynamic programming techniques and we applied our formulations for voice traffic scenario (voice over packet networks) and a general network architecture using flow-based Internet traffic modelling. In the Internet traffic modelling case, we tested two different control strategies: event-driven control and time-driven control. In event-driven control, capacity update epochs are selected to be the time instants of either a flow arrival or a flow departure. In time-driven control, decision epochs are selected to be the equidistant time instants and excessive amount of traffic that cannot be carried will be buffered.
Open Access
Dynamic threshold-based algorithms for communication networks
(2009) Toksöz, Mehmet Altan
A need to use dynamic thresholds arises in various communication networking scenarios under varying traffic conditions. In this thesis, we propose novel dynamic threshold-based algorithms for two different networking problems, namely the problem of burst assembly in Optical Burst Switching (OBS) networks and of bandwidth reservation in connection-oriented networks. Regarding the first problem, we present dynamic threshold-based burst assembly algorithms that attempt to minimize the average burst assembly delay due to burstification process while taking the burst rate constraints into consideration. Using synthetic and real traffic traces, we show that the proposed algorithms perform significantly better than the conventional timer-based schemes. In the second problem, we propose a model-free adaptive hysteresis algorithm for dynamic bandwidth reservation in a connection-oriented network subject to update frequency constraints. The simulation results in various traffic scenarios show that the proposed technique considerably outperforms the existing schemes without requiring any prior traffic information.
Open Access
A model of boundedly rational learning in dynamic games
(1997) Aksoy, Hakan
There are various computer-based algorithms about boundedly rational players’ learning how to behave in dynamic games, including classifier systems, genetic algorithms and neural networks. Some examples of studies using boundedly rational players are Axelrod (1987), Miller (1989), Andreoni and Miller (1990) who use genetic algorithm and Marimon etal. (1990) and Arthur (1990) who use classifier systems. In this dissertation, a Two Armed Bandit Problem and the KiyotakiWright (1989) Economic Environment are constructed and the learning behaviour ol the boundedly rational players is observed by using classifier systems in computer programs. From the simulation results, we observe that experimentation and imitation enables faster convergence to the correct decision rules of players in both repeated static decision problems and dynamic games.
Open Access
Routing, spectrum allocation and regenerator placement in flexible-grid optical networks
(2013) Kahya, Alper
Tremendous increase in the number of wireless devices has been resulting in huge growth in the Internet traffic. This growth necessitates efficient usage of resources in the optical networks, which form the backbone of the Internet. Recently proposed flexible optical networks can adjust the optical layer transmission parameters to take advantage of existing channel conditions thereby increasing the resource utilization efficiency. Therefore, flexible optical network is a promising solution to fulfill growing future demand of IP traffic. Apart from efficient usage of the optical spectrum, the degradation of the optical signal as it propagates over the fiber is another problem. In such cases, the optical signal must be regenerated when a lightpath travels longer than the maximum optical reach. However, regenerators are expensive devices with high operational costs. Therefore, they should be placed carefully to reduce the capital and operational network costs. In this dissertation, we deal with the joint routing, spectrum allocation and regenerator placement (RSA-RP) problem for flexible optical networks. Our aim is to find the route and allocate spectrum for each traffic demand by assigning minimum number of nodes as regenerator sites. Firstly, we introduce a novel mixed integer linear programming (MILP) formulation for the joint RSA-RP problem. Since this formulation is not practical for large networks, we propose a decoupled formulation where the RSA-RP problem is decomposed into two phases. In the first step, we find routes and locations of regenerators assuming a full wavelength converting network. Then, we allocate the spectrum to each demand in the second phase. The decoupled model can be used to solve the RSA-RP problem for reasonably sized optical networks. We show that the decoupled model can find optimum solutions for 92% of the all cases tested for the NSFNET topology and 99% of the all cases tested for the Deutsche Telecom topology. We also show that the locations of regenerator sites significantly depend on network parameters such as the node degree and lengths of the links adjacent to the node.