Variance reduction techniques: experimental comparison and analysis for single systems

We provide a thorough analysis of the effectiveness of different Variance Reduction Techniques (VRTs). We consider both stand-alone and combined applications of two input techniques, Antithetic Variates (AV) and Latin Hypercube Sampling (LHS), and two output techniques, Control Variates (CV) and Poststratified Sampling (PS). Previous research in the area mainly focuses on asymptotic variance reduction. In this experimental study, we measure the performance of VRTs under finite simulation run lengths and analyze their effects. Our findings show that the asymptotic variance reduction results do not readily apply to finite-length simulations. We consider three different types of systems (M/M/1, serial production line and (s, S) inventory control systems) and compare the VRTs under various experimental conditions. We observe that a variance reduction cannot be guaranteed for every instance a VRT is applied. Our results also indicate that the output VRTs (CV, PS) are better than input VRTs (AV, LHS) on the average for the single systems considered in this study. More interestingly, the less-sophisticated techniques (AV, CV) often perform better than the relatively more-complex techniques (LHS, PS). A comprehensive bibliography is also provided.