Venue
IEEE Transactions on Knowledge and Data Engineering
Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the field of natural language processing and artificial intelligence, due to their emergent ability and generalizability.However, LLMs are black-box models, which often fall short of capturing and accessing factual knowledge.In contrast, Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge.KGs can enhance LLMs by providing external knowledge for inference and interpretability.Meanwhile, KGs are difficult to construct and evolve by nature, which challenges the existing methods in KGs to generate new facts and represent unseen knowledge.Therefore, it is complementary to unify LLMs and KGs together and simultaneously leverage their advantages.In this article, we present a forward-looking roadmap for the unification of LLMs and KGs.Our roadmap consists of three general frameworks, namely, 1) KG-enhanced LLMs, which incorporate KGs during the pre-training and inference phases of LLMs, or for the purpose of enhancing understanding of the knowledge learned by LLMs; 2) LLM-augmented KGs, that leverage LLMs for different KG tasks such as embedding, completion, construction, graph-to-text generation, and question answering; and 3) Synergized LLMs + KGs, in which LLMs and KGs play equal roles and work in a mutually beneficial way to enhance both LLMs and KGs for bidirectional reasoning driven by both data and knowledge.We review and summarize existing efforts within these three frameworks in our roadmap and pinpoint their future research directions.
The paper discusses a roadmap for unifying Large Language Models (LLMs) and Knowledge Graphs (KGs). It outlines three frameworks: KG-enhanced LLMs, LLM-augmented KGs, and Synergized LLMs + KGs. The paper describes the advantages and challenges associated with both LLMs and KGs, emphasizing how integrating KGs can enhance LLMs by providing factual knowledge and improving interpretability. The authors propose a detailed categorization of existing research under these frameworks and highlight future research directions to further this unification effort. The main contributions include a forward-looking roadmap, a categorization of existing research, emerging advanced techniques in both fields, and future challenges in the integration process.
This paper employs the following methods:
- KG-enhanced LLMs
- LLM-augmented KGs
- Synergized LLMs + KGs
- BERT
- RoBERTA
- T5
- ChatGPT
- GPT-4
The following datasets were used in this research:
- Wikidata
- ConceptNet
- YAGO
- NELL
- Knowledge Occean
- Unification of LLMs and KGs through three frameworks improves performance and interpretation of knowledge in NLP tasks.
The authors identified the following limitations:
- LLMs struggle with hallucinations and factual inaccuracies.
- Knowledge Graph construction is challenging and often incomplete.
- Number of GPUs: None specified
- GPU Type: None specified