Hybrid Models: Combining GNNs and LLMs for Ransomware Detection

Recognizing the complementary strengths of GNNs for structural analysis and LLMs for semantic understanding, a promising new research direction involves creating hybrid models [1]. This area is still relatively new, particularly in malware detection.

Theoretical Foundations: Integration Patterns

The integration of GNNs and LLMs can be systematically categorized into distinct architectural patterns, as surveyed by Ren et al. [1]. Of particular relevance to this thesis is the LLMs as Prefix paradigm, where LLMs generate representations that enhance GNN training. Within this paradigm, two primary mechanisms exist:

• Embeddings from LLMs for GNNs: LLMs generate meaningful initial node embeddings, addressing the common issue of sparse or uninformative features [2][1]. For instance, TAPE queries LLMs using customized prompts to generate text explanations, which are then converted into node embeddings for GNNs [3]. This feature enhancement approach significantly improves node classification performance in graph learning models.

• Labels from LLMs for GNNs: The LLM generates supervision signals (labels, embeddings, or structural components) to optimize GNN training, particularly useful when training data is sparse [1]. For example, LLM-GNN utilizes LLMs as annotators to generate node category predictions with confidence scores [4].

Hybrid Models for Malware Detection

A recent study by Tran et al. [5] directly addresses the integration of LLMs with GNNs for Android malware classification. They propose a framework to enrich FCGs with contextual features including function metadata and code embeddings derived from LLMs. Their key finding is that this semantic enrichment significantly improves the model's robustness against distribution shift, which is a common problem where models fail to generalize to new, unseen malware variants.

More relevant, the MalHAPGNN framework has advanced this hybrid concept [6]. This framework first uses a BERT-based model to extract semantic features for nodes in a function call graph. It then processes this semantically enriched graph with a HAPGNN. The architecture allows the model to learn a comprehensive profile of the application by considering its semantic, syntactic, and structural dimensions simultaneously. The authors acknowledge that while their semantic enrichment shows promising results, there remains potential for further enhancement. They specifically suggest that providing more detailed insights into malware functions could improve detection performance, indicating that deeper semantic analysis of individual graph nodes represents a valuable research direction. This observation directly underscores the focus of this proposed thesis, which aims to advance semantic node representation for ransomware detection.

Within malware detection, other multi-view approaches have explored similar integration strategies. For instance, DeepCatra combines a GNN with a BiLSTM to analyze call traces, demonstrating the benefits of integrating sequence-based and graph-based features [7].

Cross-Domain Validation

The principle of combining structural and semantic information extends beyond malware detection. In the security domain, vulnerability detection has successfully employed similar approaches, with frameworks like LineVD using LLM-generated embeddings as node features for GNNs operating on code dependency graphs [8]. Beyond security, hybrid approaches have demonstrated value in domains such as fake news detection, where GNNs model social propagation networks while LLMs analyze textual content [9]. More broadly, research in general graph learning, such as the work by Shi et al. [2], focuses on improving node representations in TAGs by leveraging LLMs. These cross-domain successes validate the fundamental value of semantic enrichment for graph-based learning tasks.

---

References

[1] Ren et al., "A Survey of Large Language Models for Graphs," 2024

[2] Shi et al., "Deep Semantic Graph Learning via LLM based Node Enhancement," 2025

[3] He et al., "Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning," 2023

[4] Chen et al., "Label-free Node Classification on Graphs with Large Language Models (LLMs)," 2023

[5] Tran et al., "Mitigating Distribution Shift in Graph-Based Android Malware Classification via Function Metadata and LLM Embeddings," 2025

[6] Guo et al., "MalHAPGNN: An Enhanced Call Graph-Based Malware Detection Framework Using Hierarchical Attention Pooling Graph Neural Network," 2025

[7] Wu et al., "DeepCatra: Learning flow‐ and graph‐based behaviours for Android malware detection," 2022

[8] Hin et al., "LineVD: statement-level vulnerability detection using graph neural networks," 2022

[9] Gong et al., "Fake News Detection Through Graph-based Neural Networks: A Survey," 2023