Synergistic Integration of Knowledge Graphs and Large-scale Models: A New Paradigm
The symbiotic relationship between the natural world and the technological world is similar to the mutualistic relationship between knowledge graphs and large-scale models. In the natural world, examples of mutualistic relationships include the symbiosis between bees and flowers, as well as the interdependence between clownfish and sea anemones, which inspire us. Similarly, in the technological realm, mutualistic patterns such as the combination of photovoltaic and hydroelectric power demonstrate the advantages of complementarity and collaborative operation.
In the field of artificial intelligence, there is also a mutually beneficial relationship between large-scale models and knowledge graphs. Large-scale models improve their language understanding and generation capabilities through learning from massive amounts of textual data, while knowledge graphs provide structured domain knowledge and relationship graphs. They complement each other, as large-scale models can obtain more accurate knowledge representations from knowledge graphs, and knowledge graphs can be updated and expanded through the learning capabilities of large-scale models.
The concept of “graph-model complementarity” emphasizes the importance of the symbiotic pattern between knowledge graphs and large-scale models. By combining the strengths of knowledge graphs and large-scale models, we can achieve more powerful, intelligent, and interpretable systems. Knowledge graphs can provide controllability, interpretability, and factual evidence, while large-scale models can offer richer language understanding and generation capabilities.
In summary, the graph-model complementarity system of knowledge graphs and large-scale models is one of the best practices in the field of artificial intelligence. Their combination can help us build more powerful, intelligent, and interpretable systems, providing comprehensive knowledge and understanding capabilities for various application scenarios.
The combination and complementarity of knowledge graphs and large-scale models can address the respective challenges they face and achieve higher-level cognitive abilities and intelligent applications. Large-scale models are responsible for language understanding and generation, extracting entities and relationships from unstructured texts, and constructing or updating knowledge graphs. Knowledge graphs provide deterministic facts and knowledge with the ability for real-time updates. Through this symbiotic pattern, the illusions, lack of interpretability, and lack of control issues of large-scale models can be addressed, as well as the challenges of knowledge graph construction and weak language understanding and generation capabilities. Symbiotic patterns can achieve trustworthy, reliable, and human-centric general artificial intelligence, improving the overall performance and user experience of intelligent systems.
There are two approaches to the graph-model complementarity mechanism: model-centric and graph-centric.
Model-centric approach: Large pre-trained models serve as the main component, with knowledge graphs as auxiliary tools to enhance the model’s capabilities. By integrating knowledge from the knowledge graph, such as entities, relationships, and attributes, the model can better understand and utilize the information from the knowledge graph. This addresses the issues of illusions and fabrications that may arise when generating text, improving credibility and reliability.
Graph-centric approach: Knowledge graphs serve as the primary component, with large pre-trained models as auxiliary tools to enhance the capabilities of the knowledge graph. Large models provide additional reasoning and understanding abilities to assist knowledge graphs in supplementation and inference. By combining the language understanding and generation capabilities of large models, knowledge graphs can better answer questions, generate text, or perform inference.
Both approaches have their advantages and suitable scenarios. When choosing a method, application requirements and the complexity of the problem should be considered. They should not be seen as competitive but rather selected according to the situation to meet user needs.
Knowledge-enhanced large language models use the structure of knowledge graphs to enhance language models, capture factual knowledge from texts, and provide better results. Methods include ERNIE, SKILL, KLMo, and KEPLER, among others. Knowledge graphs can also enhance the reasoning capabilities of large models, such as LaMDA and KG-BART. Retrieval-augmented generation (RAG) utilizes document indexing to introduce external knowledge, addressing the issue of large models’ inability to update knowledge. Knowledge graphs also play an important role in the interpretation of large language models and specialized domain tasks, such as LMExplainer and CohortGPT. Knowledge-enhanced large language models improve effectiveness, reasoning abilities, interpretability, and application capabilities in specialized domains, expanding possibilities in natural language processing and other fields.
Using large models for knowledge graph construction and completion is promising. Large models can handle natural language processing tasks such as entity and relationship extraction, as demonstrated by ChatIE and ChatExtract. Manual sample curation and learning based on data generation can reduce reliance on manually annotated samples. Large models can extract knowledge from large-scale corpora for knowledge graph completion. Combining structural information and reasoning can improve the accuracy and effectiveness of knowledge graph completion. KoPA is a study exploring structure-aware reasoning. In conclusion, utilizing large models for knowledge graph construction and completion has the potential to improve the quality and completeness of knowledge graphs.
Using large language models for knowledge graph reasoning and question-answering tasks is promising. This approach can transform complex knowledge graph reasoning into a combination of contextual knowledge graph search and logical query inference, achieving higher performance reasoning. Additionally, the application of large language models in knowledge graph question-answering can provide a more natural and interactive user experience. Models like GPT-3 have been successfully used for knowledge graph question-answering tasks, where the model takes a natural language question as input and generates the corresponding answer based on the knowledge graph.
Overall, the symbiotic relationship between knowledge graphs and large-scale models in artificial intelligence is a promising approach that combines the structured knowledge representation and reasoning capabilities of knowledge graphs with the language understanding and generation capabilities of large-scale models. This combination can lead to more robust, intelligent, and interpretable systems, enabling advancements in various application domains.
