Ensemble pruning is the process of selecting a subset of component classifiers from an ensemble which performs at least as well as the original ensemble while reducing storage and computational costs. Ensemble pruning in data streams is a largely unexplored area of research. It requires analysis of ensemble components as they are running on the stream and differentiation of useful classifiers from redundant ones. We present two on-the-fly ensemble pruning methods; Class-wise Component Ranking-based Pruner (CCRP) and Cover Coefficient-based Pruner (CCP). CCRP aims that the resulting pruned ensemble contains the best performing classifier for each target class and hence, reduces the effects of class imbalance. On the other hand, CCP aims to select components that make misclassification errors on different instances. The conducted experiments on real-world and synthetic data streams demonstrate that different types of ensembles that integrate pruners consume significantly less memory and perform significantly faster without hurting the predictive performance.