Browsing by Subject "Weakly supervised learning"

Now showing 1 - 6 of 6

Open Access
Deep feature representations for variable-sized regions of ınterest in breast histopathology
(IEEE, 2021) Mercan, Caner; Aygüneş, Bulut; Aksoy, Selim; Mercan, Ezgi; Shapiro, L. G.; Weaver, D. L.; Elmore, J. G.
Objective: Modeling variable-sized regions of interest (ROIs) in whole slide images using deep convolutional networks is a challenging task, as these networks typically require fixed-sized inputs that should contain sufficient structural and contextual information for classification. We propose a deep feature extraction framework that builds an ROI-level feature representation via weighted aggregation of the representations of variable numbers of fixed-sized patches sampled from nuclei-dense regions in breast histopathology images. Methods: First, the initial patch-level feature representations are extracted from both fully-connected layer activations and pixel-level convolutional layer activations of a deep network, and the weights are obtained from the class predictions of the same network trained on patch samples. Then, the final patch-level feature representations are computed by concatenation of weighted instances of the extracted feature activations. Finally, the ROI-level representation is obtained by fusion of the patch-level representations by average pooling. Results: Experiments using a well-characterized data set of 240 slides containing 437 ROIs marked by experienced pathologists with variable sizes and shapes result in an accuracy score of 72.65% in classifying ROIs into four diagnostic categories that cover the whole histologic spectrum. Conclusion: The results show that the proposed feature representations are superior to existing approaches and provide accuracies that are higher than the average accuracy of another set of pathologists. Significance: The proposed generic representation that can be extracted from any type of deep convolutional architecture combines the patch appearance information captured by the network activations and the diagnostic relevance predicted by the class-specific scoring of patches for effective modeling of variable-sized ROIs.
Open Access
Graph convolutional networks for region of interest classification in breast histopathology
(S P I E - International Society for Optical Engineering, 2021) Aygüneş, Bulut; Aksoy, Selim; Cinbiş, R.G.; Kösemehmetoğlu, K.; Önder, S.; Üner, A.
Deep learning-based approaches have shown highly successful performance in the categorization of digitized biopsy samples. The commonly used setting in these approaches is to employ convolutional neural networks for classification of data sets consisting of images all having the same size. However, the clinical practice in breast histopathology necessitates multi-class categorization of regions of interest (ROI) in biopsy samples where these regions can have arbitrary shapes and sizes. The typical solution to this problem is to aggregate the classification results of fixed-sized patches cropped from these images to obtain image-level classification scores. Another limitation of these approaches is the independent processing of individual patches where the rich contextual information in the complex tissue structures has not yet been sufficiently exploited. We propose a generic methodology to incorporate local inter-patch context through a graph convolution network (GCN) that admits a graph-based ROI representation. The proposed GCN model aims to propagate information over neighboring patches in a progressive manner towards classifying the whole ROI into a diagnostic class. The experiments using a challenging data set for a 4-class ROI-level classification task and comparisons with several baseline approaches show that the proposed model that incorporates the spatial context by using graph convolutional layers performs better than commonly used fusion rules.
Open Access
Weakly supervised approaches for image classification in remote sensing and medical image analysis
(2020-12) Aygüneş, Bulut
Weakly supervised learning (WSL) aims to utilize data with imprecise or noisy annotations to solve various learning problems. We study WSL approaches in two different domains: remote sensing and medical image analysis. For remote sensing, we focus on the multisource fine-grained object recognition problem that aims to classify an object into one of many similar subcategories. The task we work on involves images where an object with a given class label is present in the image without any knowledge of its exact location. We approach this problem from a WSL perspective and propose a method using a single-source deep instance attention model with parallel branches for joint localization and classification of objects. We then extend this model into a multisource setting where a reference source assumed to have no location uncertainty is used to aid the fusion of multiple sources. We show that all four proposed fusion strategies that operate at the probability level, logit level, feature level, and pixel level provide higher accuracies compared to the state-of-the-art. We also provide an in-depth comparison by evaluating each model at various parameter complexity settings, where the increased model capacity results in a further improvement over the default capacity setting. For medical image analysis, we study breast cancer classification on regions of interest (ROI) of arbitrary shapes and sizes from breast biopsy whole slides. The typical solution to this problem is to aggregate the classification results of fixed-sized patches cropped from ROIs to obtain image-level classification scores. We first propose a generic methodology to incorporate local inter-patch context through a graph convolution network (GCN) that aims to propagate information over neighboring patches in a progressive manner towards classifying the whole ROI. The experiments using a challenging data set for a 3-class ROI-level classification task and comparisons with several baseline approaches show that the proposed model that incorporates the spatial context performs better than commonly used fusion rules. Secondly, we revisit the WSL framework we use in our remote sensing experiments and apply it to a 4-class ROI classification problem. We propose a new training methodology tailored for this WSL task that combines the patches and labels from pairs of ROIs together to exploit the instance attention model’s capability to learn from samples with multiple labels, which results in superior performance over several baselines.
Open Access
Weakly supervised deep convolutional networks for fine-grained object recognition in multispectral images
(Institute of Electrical and Electronics Engineers Inc., 2019) Aygüneş, Bulut; Aksoy, Selim; Cinbiş, R. G.
The challenging task of training object detectors for fine-grained classification faces additional difficulties when there are registration errors between the image data and the ground truth. We propose a weakly supervised learning methodology for the classification of 40 types of trees by using fixed-sized multispectral images with a class label but with no exact knowledge of the object location. Our approach consists of an end-to-end trainable convolutional neural network with separate branches for learning class-specific and location-specific scoring of image regions. Comparative experiments show that the proposed method simultaneously learns to detect and classify the objects of interest with high accuracy.
Open Access
Weakly supervised instance attention for multisource fine-grained object recognition with an application to tree species classification
(Elsevier BV, 2021-06) Aygüneş, Bulut; Cinbiş, R. G.; Aksoy, Selim
Multisource image analysis that leverages complementary spectral, spatial, and structural information benefits fine-grained object recognition that aims to classify an object into one of many similar subcategories. However, for multisource tasks that involve relatively small objects, even the smallest registration errors can introduce high uncertainty in the classification process. We approach this problem from a weakly supervised learning perspective in which the input images correspond to larger neighborhoods around the expected object locations where an object with a given class label is present in the neighborhood without any knowledge of its exact location. The proposed method uses a single-source deep instance attention model with parallel branches for joint localization and classification of objects, and extends this model into a multisource setting where a refer- ence source that is assumed to have no location uncertainty is used to aid the fusion of multiple sources in four different levels: probability level, logit level, feature level, and pixel level. We show that all levels of fusion provide higher accuracies compared to the state-of-the-art, with the best performing method of feature-level fusion resulting in 53% accuracy for the recognition of 40 different types of trees, corresponding to an improvement of 5.7% over the best performing baseline when RGB, multispectral, and LiDAR data are used. We also provide an in-depth comparison by evaluating each model at various parameter complexity settings, where the increased model capacity results in a further improvement of 6.3% over the default capacity setting.
Open Access
Weakly supervised object localization with multi-fold multiple instance learning
(IEEE Computer Society, 2017) Cinbis, R. G.; Verbeek, J.; Schmid, C.
Object category localization is a challenging problem in computer vision. Standard supervised training requires bounding box annotations of object instances. This time-consuming annotation process is sidestepped in weakly supervised learning. In this case, the supervised information is restricted to binary labels that indicate the absence/presence of object instances in the image, without their locations. We follow a multiple-instance learning approach that iteratively trains the detector and infers the object locations in the positive training images. Our main contribution is a multi-fold multiple instance learning procedure, which prevents training from prematurely locking onto erroneous object locations. This procedure is particularly important when using high-dimensional representations, such as Fisher vectors and convolutional neural network features. We also propose a window refinement method, which improves the localization accuracy by incorporating an objectness prior. We present a detailed experimental evaluation using the PASCAL VOC 2007 dataset, which verifies the effectiveness of our approach. © 2016 IEEE.