Browsing by Subject "Stochastic approximation"
Now showing 1 - 3 of 3
Results Per Page
Sort Options
Item Embargo Feature selection using stochastic approximation with Barzilai and Borwein non-monotone gains(Elsevier Ltd, 2021-08) Aksakallı, V.; Yenice, Z. D.; Malekipirbazari, Milad; Kargar, KamyarWith recent emergence of machine learning problems with massive number of features, feature selection (FS) has become an ever-increasingly important tool to mitigate the effects of the so-called curse of dimensionality. FS aims to eliminate redundant and irrelevant features for models that are faster to train, easier to understand, and less prone to overfitting. This study presents a wrapper FS method based on Simultaneous Perturbation Stochastic Approximation (SPSA) with Barzilai and Borwein (BB) non-monotone gains within a pseudo-gradient descent framework wherein performance is measured via cross-validation. We illustrate that SPSA with BB gains (SPSA-BB) provides dramatic improvements in terms of the number of iterations for convergence with minimal degradation in cross-validated error performance over the current state-of-the art approach with monotone gains (SPSA-MON). In addition, SPSA-BB requires only one internal parameter and therefore it eliminates the need for careful fine-tuning of numerous other internal parameters as in SPSA-MON or comparable meta-heuristic FS methods such as genetic algorithms (GA). Our particular implementation includes gradient averaging as well as gain smoothing for better convergence properties. We present computational experiments on various public datasets with Nearest Neighbors and Naive Bayes classifiers as wrappers. We present comparisons of SPSA-BB against full set of features, SPSA-MON, as well as seven popular meta-heuristics based FS algorithms including GA and particle swarm optimization. Our results indicate that SPSA-BB converges to a good feature set in about 50 iterations on the average regardless of the number of features (whether a dozen or more than 1000 features) and its performance is quite competitive. SPSA-BB can be considered extremely fast for a wrapper method and therefore it stands as a high-performing new feature selection method that is also computationally feasible in practice.Item Open Access Simulation optimization: a comprehensive review on theory and applications(Taylor & Francis, 2004) Tekin, E.; Sabuncuoglu, I.For several decades, simulation has been used as a descriptive tool by the operations research community in the modeling and analysis of a wide variety of complex real systems. With recent developments in simulation optimization and advances in computing technology, it now becomes feasible to use simulation as a prescriptive tool in decision support systems. In this paper, we present a comprehensive survey on techniques for simulation optimization with emphasis given on recent developments. We classify the existing techniques according to problem characteristics such as shape of the response surface (global as compared to local optimization), objective functions (single or multiple objectives) and parameter spaces (discrete or continuous parameters). We discuss the major advantages and possible drawbacks of the different techniques. A comprehensive bibliography and future research directions are also provided in the paper.Item Open Access State-dependent control of a single stage hybrid system with poisson arrivals(2011) Gokbayrak, K.We consider a single-stage hybrid manufacturing system where jobs arrive according to a Poisson process. These jobs undergo a deterministic process which is controllable. We define a stochastic hybrid optimal control problem and decompose it hierarchically to a lower-level and a higher-level problem. The lower-level problem is a deterministic optimal control problem solved by means of calculus of variations. We concentrate on the stochastic discrete-event control problem at the higher level, where the objective is to determine the service times of jobs. Employing a cost structure composed of process costs that are decreasing and strictly convex in service times, and system-time costs that are linear in system times, we show that receding horizon controllers are state-dependent controllers, where state is defined as the system size. In order to improve upon receding horizon controllers, we search for better state-dependent control policies and present two methods to obtain them. These stochastic-approximation-type methods utilize gradient estimators based on Infinitesimal Perturbation Analysis or Imbedded Markov Chain techniques. A numerical example demonstrates the performance improvements due to the proposed methods. © 2011 Springer Science+Business Media, LLC.