Browsing by Author "Mercan, Caner"
Now showing 1 - 5 of 5
Results Per Page
Sort Options
Item Open Access Deep feature representations and multi-instance multi-label learning of whole slide breast histopathology images(2019-03) Mercan, CanerThe examination of a tissue sample has traditionally involved a pathologist investigating the case under a microscope. Whole slide imaging technology has recently been utilized for the digitization of biopsy slides, replicating the microscopic examination procedure with the computer screen. This technology made it possible to scan the slides at very high resolutions, reaching up to 100; 000 100; 000 pixels. The advancements in the imaging technology has allowed the development of automated tools that could help reduce the workload of pathologists during the diagnostic process by performing analysis on the whole slide histopathology images. One of the challenges of whole slide image analysis is the ambiguity of the correspondence between the diagnostically relevant regions in a slide and the slide-level diagnostic labels in the pathology forms provided by the pathologists. Another challenge is the lack of feature representation methods for the variable number of variable-sized regions of interest (ROIs) in breast histopathology images as the state-of-the-art deep convolutional networks can only operate on fixed-sized small patches which may cause structural and contextual information loss. The last and arguably the most important challenge involves the clinical significance of breast histopathology, for the misdiagnosis or the missed diagnoses of a case may lead to unnecessary surgery, radiation or hormonal therapy. We address these challenges with the following contributions. The first contribution introduces the formulation of the whole slide breast histopathology image analysis problem as a multi-instance multi-label learning (MIMLL) task where a slide corresponds to a bag that is associated with the slide-level diagnoses provided by the pathologists, and the ROIs inside the slide correspond to the instances in the bag. The second contribution involves a novel feature representation method for the variable number of variable-sized ROIs using the activations of deep convolutional networks. Our final contribution includes a more advanced MIMLL formulation that can simultaneously perform multi-class slide-level classification and ROI-level inference. Through quantitative and qualitative experiments, we show that the proposed MIMLL methods are capable of learning from only slide-level information for the multi-class classification of whole slide breast histopathology images and the novel deep feature representations outperform the traditional features in fully supervised and weakly supervised settings.Item Open Access Deep feature representations for variable-sized regions of ınterest in breast histopathology(IEEE, 2021) Mercan, Caner; Aygüneş, Bulut; Aksoy, Selim; Mercan, Ezgi; Shapiro, L. G.; Weaver, D. L.; Elmore, J. G.Objective: Modeling variable-sized regions of interest (ROIs) in whole slide images using deep convolutional networks is a challenging task, as these networks typically require fixed-sized inputs that should contain sufficient structural and contextual information for classification. We propose a deep feature extraction framework that builds an ROI-level feature representation via weighted aggregation of the representations of variable numbers of fixed-sized patches sampled from nuclei-dense regions in breast histopathology images. Methods: First, the initial patch-level feature representations are extracted from both fully-connected layer activations and pixel-level convolutional layer activations of a deep network, and the weights are obtained from the class predictions of the same network trained on patch samples. Then, the final patch-level feature representations are computed by concatenation of weighted instances of the extracted feature activations. Finally, the ROI-level representation is obtained by fusion of the patch-level representations by average pooling. Results: Experiments using a well-characterized data set of 240 slides containing 437 ROIs marked by experienced pathologists with variable sizes and shapes result in an accuracy score of 72.65% in classifying ROIs into four diagnostic categories that cover the whole histologic spectrum. Conclusion: The results show that the proposed feature representations are superior to existing approaches and provide accuracies that are higher than the average accuracy of another set of pathologists. Significance: The proposed generic representation that can be extracted from any type of deep convolutional architecture combines the patch appearance information captured by the network activations and the diagnostic relevance predicted by the class-specific scoring of patches for effective modeling of variable-sized ROIs.Item Open Access From patch-level to ROI-level deep feature representations for breast histopathology classification(SPIE, 2019) Mercan, Caner; Aksoy, Selim; Mercan, E.; Shapiro, L. G.; Weaver, D. L.; Elmore, J. G.; Tomaszewski, J. E.; Ward, A. D.We propose a framework for learning feature representations for variable-sized regions of interest (ROIs) in breast histopathology images from the convolutional network properties at patch-level. The proposed method involves fine-tuning a pre-trained convolutional neural network (CNN) by using small fixed-sized patches sampled from the ROIs. The CNN is then used to extract a convolutional feature vector for each patch. The softmax probabilities of a patch, also obtained from the CNN, are used as weights that are separately applied to the feature vector of the patch. The final feature representation of a patch is the concatenation of the class-probability weighted convolutional feature vectors. Finally, the feature representation of the ROI is computed by average pooling of the feature representations of its associated patches. The feature representation of the ROI contains local information from the feature representations of its patches while encoding cues from the class distribution of the patch classification outputs. The experiments show the discriminative power of this representation in a 4-class ROI-level classification task on breast histopathology slides where our method achieved an accuracy of 66.8% on a data set containing 437 ROIs with different sizes.Item Open Access Iterative estimation of Robust Gaussian mixture models in heterogeneous data sets(2014-07) Mercan, CanerDensity estimation is the process of estimating the parameters of a probability density function from data. The Gaussian mixture model (GMM) is one of the most preferred density families. We study the estimation of a Gaussian mixture from a heterogeneous data set that is de ned as the set of points that contains interesting points that are sampled from a mixture of Gaussians as well as non-Gaussian distributed uninteresting ones. The traditional GMM estimation techniques such as the Expectation-Maximization algorithm cannot e ectively model the interesting points in a heterogeneous data set due to their sensitivity to the uninteresting points as outliers. Another potential problem is that the true number of components should often be known a priori for a good estimation. We propose a GMM estimation algorithm that iteratively estimates the number of interesting points, the number of Gaussians in the mixture, and the actual mixture parameters while being robust to the presence of uninteresting points in heterogeneous data. The procedure is designed so that one Gaussian component is estimated using a robust formulation at each iteration. The number of interesting points that belong to this component is also estimated using a multi-resolution search procedure among a set of candidates. If a hypothesis on the Gaussianity of these points is accepted, the estimated Gaussian is kept as a component in the mixture, the associated points are removed from the data set, and the iterations continue with the remaining points. Otherwise, the estimation process is terminated and the remaining points are labeled as uninteresting. Thus, the stopping criterion helps to identify the true number of components without any additional information. Comparative experiments on synthetic and real-world data sets show that our algorithm can identify the true number of components and can produce a better density estimate in terms of log-likelihood compared to two other algorithms.Item Open Access Multi-instance multi-label learning for whole slide breast histopathology(International Society for Optical Engineering SPIE, 2016-02-03) Mercan, Caner; Mercan, E.; Aksoy, Selim; Shapiro, L. G.; Weaver, D. L.; Elmore, J. G.Digitization of full biopsy slides using the whole slide imaging technology has provided new opportunities for understanding the diagnostic process of pathologists and developing more accurate computer aided diagnosis systems. However, the whole slide images also provide two new challenges to image analysis algorithms. The first one is the need for simultaneous localization and classification of malignant areas in these large images, as different parts of the image may have different levels of diagnostic relevance. The second challenge is the uncertainty regarding the correspondence between the particular image areas and the diagnostic labels typically provided by the pathologists at the slide level. In this paper, we exploit a data set that consists of recorded actions of pathologists while they were interpreting whole slide images of breast biopsies to find solutions to these challenges. First, we extract candidate regions of interest (ROI) from the logs of pathologists' image screenings based on different actions corresponding to zoom events, panning motions, and fixations. Then, we model these ROIs using color and texture features. Next, we represent each slide as a bag of instances corresponding to the collection of candidate ROIs and a set of slide-level labels extracted from the forms that the pathologists filled out according to what they saw during their screenings. Finally, we build classifiers using five different multi-instance multi-label learning algorithms, and evaluate their performances under different learning and validation scenarios involving various combinations of data from three expert pathologists. Experiments that compared the slide-level predictions of the classifiers with the reference data showed average precision values up to 62% when the training and validation data came from the same individual pathologist's viewing logs, and an average precision of 64% was obtained when the candidate ROIs and the labels from all pathologists were combined for each slide. © 2016 SPIE.