Browsing by Subject "Graph neural networks"

Now showing 1 - 5 of 5

Open Access
Automatic code optimization using graph neural networks
(2023-01) Peker, Melih
Compilers provide hundreds of optimization options, and choosing a good optimization sequence is a complex and time-consuming task. It requires extensive effort and expert input to select a good set of optimization flags. Therefore, there is a lot of research focused on finding optimizations automatically. While most of this research considers using static, spatial, or dynamic features, some of the latest research directly applied deep neural networks on source code. We combined the static features, spatial features, and deep neural networks by rep-resenting source code as graphs and trained Graph Neural Network (GNN) for automatically finding suitable optimization flags. We chose eight binary optimization flags and two benchmark suites, Polybench and cBench. We created a dataset of 12000 graphs using 256 optimization flag combinations on 47 benchmarks. We trained and tested our model using these benchmarks, and our results show that we can achieve a maximum of 48.6%speed-up compared to the case where all optimization flags are enabled.
Open Access
Improving the performance of 1D vertex parallel GNN training on distributed memory systems
(2024-07) Taşcı, Kutay
Graph Neural Networks (GNNs) are pivotal for analyzing data within graphstructured domains such as social media, biological networks, and recommendation systems. Despite their advantages, scaling GNN training to large datasets in distributed settings poses significant challenges due to the complex task of managing computation and communication costs. The objective of this work is to scale 1D vertex-parallel GNN training on distributed memory systems via (i) twoconstraint partitioning formulation for better computational load balancing and (ii) overlapping communication with computation for reducing communication overhead. In the proposed two-constraint formulation, one constraint encodes the computational load balance during forward propagation, whereas the second constraint encodes the computational load balance during backward propagation. We propose three communication and computation overlapping methods that perform overlapping at three different levels. These methods were tested against traditional approaches using benchmark datasets, demonstrating improved training efficiency without altering the model structure. The outcomes indicate that multi-constraint graph partitioning and the integration of communication and computation overlapping schemes can significantly mitigate the challenges of distributed GNN training. The research concludes with recommendations for future work, including adapting these techniques to dynamic and more complex GNN architectures, promising further improvements in the efficiency and applicability of GNNs in real-world scenarios.
Open Access
Self-supervised learning with graph neural networks for region of interest retrieval in histopathology
(IEEE, 2021-05-05) Özen, Yiğit; Aksoy, Selim; Kösemehmetoğlu, Kemal; Önder, Sevgen; Üner, Ayşegül
Deep learning has achieved successful performance in representation learning and content-based retrieval of histopathology images. The commonly used setting in deep learning-based approaches is supervised training of deep neural networks for classification, and using the trained model to extract representations that are used for computing and ranking the distances between images. However, there are two remaining major challenges. First, supervised training of deep neural networks requires large amount of manually labeled data which is often limited in the medical field. Transfer learning has been used to overcome this challenge, but its success remained limited. Second, the clinical practice in histopathology necessitates working with regions of interest (ROI) of multiple diagnostic classes with arbitrary shapes and sizes. The typical solution to this problem is to aggregate the representations of fixed-sized patches cropped from these regions to obtain region-level representations. However, naive methods cannot sufficiently exploit the rich contextual information in the complex tissue structures. To tackle these two challenges, we propose a generic method that utilizes graph neural networks (GNN), combined with a self-supervised training method using a contrastive loss. GNN enables representing arbitrarily-shaped ROIs as graphs and encoding contextual information. Self-supervised contrastive learning improves quality of learned representations without requiring labeled data. The experiments using a challenging breast histopathology data set show that the proposed method achieves better performance than the state-of-the-art.
Open Access
Self-supervised representation learning with graph neural networks for region of interest analysis in breast histopathology
(2020-12) Özen, Yiğit
Deep learning has made a major contribution to histopathology image analysis with representation learning outperforming hand-crafted features. However, two notable challenges remain. The first is the lack of large histopathology datasets. The commonly used setting in deep learning-based approaches is supervised training of deep and wide models using large labeled datasets. Manually labeling histopathology images is a time-consuming operation. Assembling a large public dataset is also proven difficult due to privacy concerns. Second, the clinical practice in histopathology necessitates working with regions of interest of multiple diagnostic classes with arbitrary shapes and sizes. The typical solution to this problem is to aggregate the representations of fixed-sized patches cropped from these regions to obtain region-level representations. However, naive methods cannot sufficiently exploit the rich contextual information in the complex tissue structures. To tackle these two challenges, this thesis proposes a generic method that utilizes graph neural networks, combined with a self-supervised training method using a contrastive loss function. The regions of interest are modeled as graphs where vertices are fixed-sized patches cropped from the region. The proposed method has two stages. The first stage is patch-level representation learning using convolutional neural networks which concentrates on cell-level features. The second stage is region-level representation learning using graph neural networks using vertex dropout augmentation. The experiments using a challenging breast histopathology dataset show that the proposed method achieves better performance than the state-of-the-art in both classification and retrieval tasks. networks which can learn the tissue structure. Graph neural networks enable representing arbitrarily-shaped regions as graphs and encoding contextual information through message passing between neighboring patches. Self-supervised contrastive learning improves quality of learned representations without requiring labeled data. We propose using self-supervised learning to train graph neural networks using vertex dropout augmentation. The experiments using a challenging breast histopathology dataset show that the proposed method achieves better performance than the state-of-the-art in both classification and retrieval tasks.
Open Access
Towards deeply intelligent interfaces in relational databases
(2021-08) Usta, Arif
Relational databases is one of the most popular and broadly utilized infrastruc-tures to store data in a structured fashion. In order to retrieve data, users have to phrase their information need in Structured Query Language (SQL). SQL is a powerfully expressive and ﬂexible language, yet one has to know the schema underlying the database on which the query is issued and to be familiar with SQL syntax, which is not trivial for casual users. To this end, we propose two diﬀerent strategies to provide more intelligent user interfaces to relational databases by utilizing deep learning techniques. As the ﬁrst study, we propose a solution for keyword mapping in Natural Language Interfaces to Databases (NLIDB), which aims to translate Natural Language Queries (NLQs) to SQL. We deﬁne the key-word mapping problem as a sequence tagging problem, and propose a novel deep learning based supervised approach that utilizes part-of-speech (POS) tags of NLQs. Our proposed approach, called DBTagger (DataBase Tagger), is an end-to-end and schema independent solution. Query recommendation paradigm, a well-known strategy broadly utilized in Web search engines, is helpful to suggest queries of expert users to the casual users to help them with their information need. As the second study, we propose Conquer, a CONtextual QUEry Recom-mendation algorithm on relational databases exploiting deep learning. First, we train local embeddings of a database using Graph Convolutional Networks to ex-tract distributed representations of the tuples in latent space. We represent SQL queries with a semantic vector by averaging the embeddings of the tuples returned as a result of the query. We employ cosine similarity over the ﬁnal representations of the queries to generate recommendations, as a Witness-Based approach. Our results show that in classiﬁcation accuracy of database rows as an indicator for embedding quality, Conquer outperforms state-of-the-art techniques.