Browsing by Subject "Region of interest classification"

Now showing 1 - 4 of 4

Open Access
Deep feature representations for variable-sized regions of ınterest in breast histopathology
(IEEE, 2021) Mercan, Caner; Aygüneş, Bulut; Aksoy, Selim; Mercan, Ezgi; Shapiro, L. G.; Weaver, D. L.; Elmore, J. G.
Objective: Modeling variable-sized regions of interest (ROIs) in whole slide images using deep convolutional networks is a challenging task, as these networks typically require fixed-sized inputs that should contain sufficient structural and contextual information for classification. We propose a deep feature extraction framework that builds an ROI-level feature representation via weighted aggregation of the representations of variable numbers of fixed-sized patches sampled from nuclei-dense regions in breast histopathology images. Methods: First, the initial patch-level feature representations are extracted from both fully-connected layer activations and pixel-level convolutional layer activations of a deep network, and the weights are obtained from the class predictions of the same network trained on patch samples. Then, the final patch-level feature representations are computed by concatenation of weighted instances of the extracted feature activations. Finally, the ROI-level representation is obtained by fusion of the patch-level representations by average pooling. Results: Experiments using a well-characterized data set of 240 slides containing 437 ROIs marked by experienced pathologists with variable sizes and shapes result in an accuracy score of 72.65% in classifying ROIs into four diagnostic categories that cover the whole histologic spectrum. Conclusion: The results show that the proposed feature representations are superior to existing approaches and provide accuracies that are higher than the average accuracy of another set of pathologists. Significance: The proposed generic representation that can be extracted from any type of deep convolutional architecture combines the patch appearance information captured by the network activations and the diagnostic relevance predicted by the class-specific scoring of patches for effective modeling of variable-sized ROIs.
Open Access
Graph convolutional networks for region of interest classification in breast histopathology
(S P I E - International Society for Optical Engineering, 2021) Aygüneş, Bulut; Aksoy, Selim; Cinbiş, R.G.; Kösemehmetoğlu, K.; Önder, S.; Üner, A.
Deep learning-based approaches have shown highly successful performance in the categorization of digitized biopsy samples. The commonly used setting in these approaches is to employ convolutional neural networks for classification of data sets consisting of images all having the same size. However, the clinical practice in breast histopathology necessitates multi-class categorization of regions of interest (ROI) in biopsy samples where these regions can have arbitrary shapes and sizes. The typical solution to this problem is to aggregate the classification results of fixed-sized patches cropped from these images to obtain image-level classification scores. Another limitation of these approaches is the independent processing of individual patches where the rich contextual information in the complex tissue structures has not yet been sufficiently exploited. We propose a generic methodology to incorporate local inter-patch context through a graph convolution network (GCN) that admits a graph-based ROI representation. The proposed GCN model aims to propagate information over neighboring patches in a progressive manner towards classifying the whole ROI into a diagnostic class. The experiments using a challenging data set for a 4-class ROI-level classification task and comparisons with several baseline approaches show that the proposed model that incorporates the spatial context by using graph convolutional layers performs better than commonly used fusion rules.
Open Access
Use of subgraph mining in histopathology image classification
(2022-09) Berdiyev, Bayram
Breast cancer is the most common cancer in women and has a high mortality rate. Computer vision techniques can be used to help experts to analyze the breast cancer biopsy samples better. Graph neural networks (GNN) have been widely used to solve the classification of breast cancer images. Images in this field have varying sizes and GNNs can be applied to varying sized inputs. Graphs can store relations between the vertices of the graph and this is another reason why GNNs are preferred as a solution. We study the use of subgraph mining in classification of regions of interest (ROI) on breast histopathology images. We represent ROI samples with graphs by using patches sampled on nuclei-rich regions as the vertices of the graph. Both micro and macro level information are essential when classifying histopathology images. The patches are used to model micro-level information. We apply subgraph mining to the resulting graphs to identify frequently occurring subgraphs. Each subgraph is composed of a small number of patches and their relations, which can be used to represent higher level information. We also extract ROI-level features by applying a sliding window mechanism with larger sized patches. The ROI-level features, subgraph features and a third representation obtained from graph convolutional networks are fused to model macro-level information about the ROIs. We also study embedding the subgraphs in the graph representation as additional vertices. The proposed models are evaluated on a challenging breast pathology dataset that includes four diagnostic categories from the full spectrum. The experiments show that embedding the subgraphs in the graph representation improves the classification accuracy and the fused feature representation performs better than the individual representations in an ablation study.
Open Access
Weakly supervised approaches for image classification in remote sensing and medical image analysis
(2020-12) Aygüneş, Bulut
Weakly supervised learning (WSL) aims to utilize data with imprecise or noisy annotations to solve various learning problems. We study WSL approaches in two different domains: remote sensing and medical image analysis. For remote sensing, we focus on the multisource fine-grained object recognition problem that aims to classify an object into one of many similar subcategories. The task we work on involves images where an object with a given class label is present in the image without any knowledge of its exact location. We approach this problem from a WSL perspective and propose a method using a single-source deep instance attention model with parallel branches for joint localization and classification of objects. We then extend this model into a multisource setting where a reference source assumed to have no location uncertainty is used to aid the fusion of multiple sources. We show that all four proposed fusion strategies that operate at the probability level, logit level, feature level, and pixel level provide higher accuracies compared to the state-of-the-art. We also provide an in-depth comparison by evaluating each model at various parameter complexity settings, where the increased model capacity results in a further improvement over the default capacity setting. For medical image analysis, we study breast cancer classification on regions of interest (ROI) of arbitrary shapes and sizes from breast biopsy whole slides. The typical solution to this problem is to aggregate the classification results of fixed-sized patches cropped from ROIs to obtain image-level classification scores. We first propose a generic methodology to incorporate local inter-patch context through a graph convolution network (GCN) that aims to propagate information over neighboring patches in a progressive manner towards classifying the whole ROI. The experiments using a challenging data set for a 3-class ROI-level classification task and comparisons with several baseline approaches show that the proposed model that incorporates the spatial context performs better than commonly used fusion rules. Secondly, we revisit the WSL framework we use in our remote sensing experiments and apply it to a 4-class ROI classification problem. We propose a new training methodology tailored for this WSL task that combines the patches and labels from pairs of ROIs together to exploit the instance attention model’s capability to learn from samples with multiple labels, which results in superior performance over several baselines.