Jan 31, 2025
SurrealDB
GraphRAG: Enhancing Retrieval-Augmented Generation with SurrealDB, Gemini and DeepSeek
Imagine a detective investigating a complex case. They have a mountain of evidence: witness statements, forensic reports, surveillance footage. But sifting through it all, trying to connect the dots and uncover hidden relationships, proves overwhelming. Traditional RAG systems are like this detective, diligently analyzing individual pieces of information but often missing crucial connections that lie hidden within the data.
Now, imagine that same detective equipped with a powerful tool – a knowledge graph that visually maps out all the entities involved in the case and their intricate relationships. Suddenly, the detective can see the bigger picture, identify key connections, and even infer new information that was previously obscured. This is the power of GraphRAG: by combining the strengths of traditional RAG with the semantic richness of graph databases, we unlock a new level of understanding and insight.
In this blog post, we’ll explore how GraphRAG enhances Retrieval-Augmented Generation by incorporating graph databases in SurrealDB alongside traditional vector search. We’ll delve into the practical implementation, demonstrate tangible benefits through concrete examples and code, and compare the impact of different LLMs, from locally run open-source models like DeepSeek to massive hosted models like Gemini. Join us on this journey as we unlock the true potential of RAG with the power of graphs.
LLM and Hosting Options
Choosing the right LLM and hosting option can significantly impact your GraphRAG system’s performance and capabilities. Here’s a comparison of the three options explored in this post:
Feature | Gemini 1.5 Flash | DeepSeek R1 (Local) | DeepSeek R1 (TogetherAI) |
---|---|---|---|
Model Size | ~32 billion parameters | 7 billion parameters | 671 billion parameters |
Context Window | Massive | Large | Limited by api |
Chunking Required | No | No | Yes |
Hosting | Google Cloud | Local | TogetherAI Cloud |
Speed | Extremely Fast | Moderate | Moderate |
Control | Less | Full | Less |
Cost | Pay-as-you-go | Dependent on infrastructure | Pay-as-you-go |
Benefits | Powerful performance, handles complex queries, large context window | Local control, potentially lower latency, large context window | Access to full model, large context window |
Limitations | Less control, cost may vary | Requires local infrastructure, potential maintenance | Token limits require chunking, less control, cost may vary |
As you can see, each option offers a unique set of advantages and disadvantages.
DeepSeek R1 (DeepSeek hosted) the DeepSeek hosted version wasn’t tested due to cyber security concerns and an outage of registering API tokens at the time of this analysis.
Ultimately, the best choice depends on your specific needs and priorities. Consider factors such as the complexity of your queries, the size of your documents, your budget, and your desired level of control when making your decision.
Traditional RAG systems rely on vector search to find relevant text chunks based on semantic similarity. However, this approach can miss crucial connections and contextual information not explicitly present in the retrieved text.
By using a knowledge graph, we can explicitly represent entities and their relationships as nodes and edges. This allows for a more nuanced understanding of the context surrounding a user’s query.
Here’s how graph databases improve RAG systems:
Adding graph capabilities to vector search enables a RAG system to move beyond simple keyword matching and reason about the user’s query within a broader context.
The get_related_entities function uses SurrealDB’s vector search capabilities to find related entities based on text descriptions.
DEFINE FUNCTION OVERWRITE fn::get_related_entities($text: string) { LET $emb = fn::sentence_to_vector($text); RETURN SELECT id, description, ->RELATED_TO.{out, strength, description} FROM PERSON WHERE embedding <|5,COSINE|> $emb; };
Here’s a breakdown:
DEFINE FUNCTION OVERWRITE fn::get_related_entities($text: string) { ... };:
This defines a function named fn::get_related_entities that accepts a string argument $text. The OVERWRITE keyword ensures any existing function with the same name is replaced.LET $emb = fn::sentence_to_vector($text);:
This calculates the vector embedding of the input string $text using the fn::sentence_to_vector function
(a custom function likely using a pre-trained model like GloVe) and stores the result in the $emb
variable.RETURN SELECT id, description, ->RELATED_TO.{out, strength, description} ...:
This specifies the data to be returned, including the id, description, and related records (with out, strength, and description fields) linked through the RELATED_TO
edge.FROM PERSON:
This indicates the query should be executed against the PERSON
table.WHERE embedding <|5,COSINE|> $emb;:
This filters records in the PERSON
table based on the semantic similarity of their embeddings to the input, using a k-nearest neighbors search (KNN) with cosine distance to find the 5 most similar records.In essence, this function takes a text query, converts it into a vector embedding, and finds the 5 most semantically similar records in the PERSON
table, returning their details and related records.
The notebooks provides code examples demonstrating how to use these graph capabilities.
get_the_graph_context_surql = "fn::get_related_entities('{0}')" outcome = await db.query( get_the_graph_context_surql.format("Who is the main character?") )
This snippet calls the fn::get_related_entities
function to get context from the graph, executing it with the question “Who is the main character?” and transforming the result into a data structure used to provide context to the language model.
These functions create a message for the LLM by augmenting the input prompt with data from the database. Depending on the model we follow different methods for querying the LLM.
Gemini:
1)Upload the corpus document as an attachment
2)Query the database for the relevant graph data
3)Construct the prompt combining the graph and question
4)Send the prompt and reference to the API for processing
DeepSeek R1 (Local):
1)Query the database for the relevant graph data
2)Construct the prompt combining the corpus, graph and question
3)Send the prompt and reference to the API for processing
DeepSeek R1 (TogetherAI):
1)Query the database for the relevant graph data
3)Query the database for the relevant chunks from the corpus document
3)Construct the prompt combining the corpus chunks, graph and question
4)Send the prompt and reference to the API for processing
In all cases the function get_graph_RAG_response , while generate_graph_RAG_messages generates the semantically relevant knowledge graph using fn::get_related_entities.
For the chunking operation we leverage the function fn::get_related_chunks.
which is similar to the related entities function but just returns the semantically similar chunks without the need for a graph.
The standard RAG prompt makes reference to the source material and appends the input text as such:
Goal - You are a researcher who is providing answers to a question for you client based on documents provided you. Answer the questions to the best of your ability given the documents.
<question>
{question}
</question>
We extend that prompt to include the knowledge graph for comparison:
Goal-
You are a researcher who is providing answers to a question for your client based on documents provided to you.
You also have access to a knowledge graph based on the documents provided.
Answer the questions to the best of your ability given the documents and knowledge graph.
<knowledge graph>
{knowledge_graph}
</knowledge graph>
<question>
{question}
</question>
For the DeepSeek prompts we also incorporate a section to contain the document or chunks of the document in a section delimited by
While both standard RAG and Graph RAG aim to answer questions based on a given text, their approaches and results differ significantly. Let’s delve deeper into the specific examples from the notebook to highlight these differences:
1. Enhanced Detail and Context:
2. Inference and Relationship Awareness:
3. Consistent and Nuanced Table Generation:
4. Subjectivity vs. Grounded Responses:
In summary, these examples illustrate how Graph RAG consistently outperforms standard RAG in terms of detail, context, inference, and nuanced understanding. By incorporating a knowledge graph, Graph RAG moves beyond simple text matching and delves into the intricate web of relationships and meanings, leading to more comprehensive and insightful responses.
While both Gemini and DeepSeek showcased the advantages of graph-enhanced RAG, their outputs revealed some intriguing differences, highlighting the diverse capabilities of these language models.
Quality of Response:
Gemini consistently generated more comprehensive and nuanced responses, demonstrating a deeper understanding of the context and relationships within the knowledge graph. For instance, when asked about Alex Mercer’s motivations, Gemini provided a detailed analysis, incorporating information from both the text and the knowledge graph to paint a complete picture of his character. DeepSeek’s responses, while accurate, were sometimes overly simplistic and lacked the same level of depth.
Example:
Following Formatting Instructions:
Gemini consistently followed the given formatting instructions, producing well-structured markdown tables and lists as requested. DeepSeek, on the other hand, struggled with this aspect, sometimes failing to generate proper markdown tables or deviating from the specified format. This suggests that Gemini has a better grasp of formatting conventions and can more reliably produce outputs that adhere to specific requirements.
Example:
Speed:
Gemini exhibited significantly faster response times compared to both locally hosted DeepSeek and DeepSeek hosted on TogetherAI. This highlights Gemini’s impressive processing power and efficiency, making it a strong choice for applications where speed is critical. While DeepSeek’s local hosting offers potential advantages in terms of control and data privacy, it may come at the cost of slower response times.
DeepSeek’s “Think” Section:
One unique aspect of DeepSeek’s responses is the inclusion of a “think” section, which provides valuable insights into the model’s reasoning process. This section reveals how DeepSeek utilizes the knowledge graph to enhance its responses, offering a better understanding of the mechanisms at work. This transparency is invaluable for developers and researchers seeking to analyze and improve RAG systems.
In summary, both Gemini and DeepSeek demonstrated the benefits of graph-enhanced RAG, but their outputs revealed distinct differences in terms of response quality, formatting capabilities, and speed. Gemini’s power and efficiency make it a strong choice for complex tasks and large-scale applications, while DeepSeek’s local hosting and “think” section offer advantages in terms of control, transparency, and potential for customization.
The notebook includes several sample questions to highlight the differences in responses between standard RAG and graph-enhanced RAG.
1: “Who is the main character and what motivates them?”
------ STANDARD RAG -------------------------
While other characters like Dr. Jordan Hayes (driven by scientific curiosity) and Agent Taylor Cruz (motivated by ambition and control) also play pivotal roles, Mercer’s character arc, spanning from strict compliance to a more nuanced understanding of the mission, positions him as the central figure through which the overarching narrative unfolds.
----- GRAPH RAG -------------------------
Analysis: The graph RAG response provides a more definitive answer and includes additional motivations like mentorship and survival, gleaned from the knowledge graph.
2: “What places and people does Alex Mercer hold dearest? Format your response as a markdown list.”
----- STANDARD RAG -------------------------
Based solely on the provided text, there is no explicit mention of places or people Alex Mercer holds dearest. The text focuses on his professional life and his participation in Operation: Dulce. While he shows camaraderie and respect for his team members (Jordan Hayes, Taylor Cruz, and Sam Rivera), there’s no indication of these relationships extending beyond professional respect and shared mission objectives. The narrative doesn’t offer details about his personal life or attachments outside of his work with Paranormal Military Squad.
----- GRAPH RAG -------------------------
Based on the provided text and knowledge graph, here’s what we can infer about the places and people Alex Mercer holds dearest:
Analysis: The standard RAG system fails to provide any relevant information, while the graph RAG system infers the answer based on the knowledge graph.
3: “Create a table of places that Alex communicates with: In the first column indicate the person. In the 2nd column list the frequency he has scenes with them. In the 3rd column list the places they had scenes together ordered by In the 4th column indicate the strength of the bond between the people In the 5th column score the total weight of the interactions across the output the response in a table format using markdown language.”
----- STANDARD RAG -------------------------
Person | Communication Frequency | Places of Communication (Ordered by Importance) | Bond Strength | Interaction Weight Score |
---|---|---|---|---|
Jordan Hayes | Very High | 1. Dulce Base Command Center (multiple scenes, critical collaborations), 2. Dulce Base Corridors (shared concerns), 3. Elevator to Dulce (shared apprehension), 4. Dulce Base Lab (scientific collaboration) | Very Strong | 9 |
Taylor Cruz | High | 1. Dulce Base Command Center (conflicts and uneasy alliances), 2. Briefing Room (initial instructions, later collaboration), 3. Equipment Bay (philosophical disagreement) | Strong but strained | 7 |
Sam Rivera | High | 1. Dulce Base Command Center (collaborative work, shared excitement), 2. Sam’s Office (advice and encouragement), 3. Dulce Base Corridors (shared mission) | Strong | 7 |
Note: The “Interaction Weight Score” is a subjective assessment based on the length and significance of their interactions throughout the provided text. A higher score indicates more frequent and/or more impactful interactions. The ordering of places within each person’s row reflects the apparent importance of the setting to their relationship and the overall narrative.
----- GRAPH RAG -------------------------
Based on the provided text and knowledge graph, here’s a table summarizing Alex Mercer’s communication and interactions, keeping in mind that the provided knowledge graph lacks information on the frequency of scenes and requires inference from the narrative. The “Strength of Bond” and “Total Interaction Weight” columns are estimations based on the narrative and the knowledge graph’s strength scores. Precise quantification isn’t possible without more structured data.
Person | Frequency of Scenes | Places of Interaction (ordered by importance) | Strength of Bond | Total Interaction Weight (estimated) |
---|---|---|---|---|
Jordan Hayes | Very High | Dulce Base Command Center, Mainframe Room, Communications Hub, Underground Meeting Room | 9 | 90 |
Sam Rivera | High | Dulce Base Command Center, Mainframe Room, Server Room, Underground Meeting Room | 8 | 72 |
Taylor Cruz | High | Dulce Base Command Center, Mainframe Room, Underground Meeting Room, Briefing Room | 7 | 63 |
Notes:
The knowledge graph is limited. A more comprehensive graph that includes scene frequency and interaction details would allow for a more precise and detailed response.
Analysis: Both responses provide tables, but the graph RAG system incorporates specific data from the knowledge graph (like bond strength scores) for a more nuanced response.
4: “Create a table of places that Alex visits: In the first column indicate the place name. In the 2nd column indicate the frequency he visits them. In the 3rd column list the people who were there with him by strength In the 4th column indicate the importance of the place. output the response in a table format using markdown language.”
----- STANDARD RAG -------------------------
Place Name | Frequency | People Present (Bond Strength) | Importance | Location Description | Interaction Weight |
---|---|---|---|---|---|
Briefing Room | 3 | Cruz (Antagonistic), Hayes (Understanding), Rivera (Respectful), | High (Mission planning, strategy)b | Sterile, large room with projectors; used for mission briefings. | 7 |
Cramped Underground Room | 1 | Hayes, Cruz | High (Mission initiation, descent) | Small, dark room, leading to elevator to Dulce. | 3 |
Elevator Shaft | 1 | Hayes, Cruz, Rivera | High (Journey to Dulce) | Metal shaft, descending into Dulce base. Dark and claustrophobic. | 3 |
Dulce Base Corridors | 3 | Hayes (Strong), Cruz (Antagonistic), Rivera (Respectful) | High (Exploration, discoveries) | Long, dark corridors within Dulce base. Steel doors, eerie atmosphere. | 8 |
Dulce Base Mainframe Room | 2 | Hayes (Strong), Cruz (Antagonistic), Rivera (Respectful) | High (Central hub, communication) | Large room containing mainframe and communications equipment. | 7 |
Dulce Base Lab | 1 | Hayes (Strong) | Moderate (Alien tech analysis) | A laboratory within Dulce base; contains alien technology. | 2 |
Dulce Base Server Room | 1 | Rivera (Mentor-Mentee), | Moderate (Data analysis) | Small room with numerous servers; used for data analysis. | 2 |
Dulce Base Command Center | 6 | Hayes (Strong), Cruz (Antagonistic), Rivera (Respectful) | Highest (Alien communication, mission culmination) | Central hub of Dulce base, containing advanced communications equipment and consoles. | 15 |
Note: Bond strength is a subjective assessment based on the narrative. Interaction weight is an aggregate score reflecting the significance of interactions in each location across all chapters, with higher scores indicating greater importance and interaction. Frequency counts instances where Alex is explicitly present and actively participating.
----- GRAPH RAG -------------------------
Based on the provided text and knowledge graph, here’s a table summarizing the places Alex Mercer visits, along with associated details:
Place Name | Visit Frequency | People Present (by strength of bond) | Importance of Place | Location Description | Total Interaction Weight |
---|---|---|---|---|---|
Dulce Base | Multiple | Jordan Hayes (9), Taylor Cruz (8), Sam Rivera (7) | Very High | Underground military base; mysterious, potentially dangerous. | 24 |
Briefing Room | Multiple | Jordan Hayes, Taylor Cruz, Sam Rivera | High | Sterile room where missions are briefed. | 24 |
Cramped Office | At least once | Sam Rivera | Medium | Small office where Sam works with servers; cramped workspace. | 7 |
Deserted Corridor | At least once | Taylor Cruz | Low | A hallway inside Dulce Base. | 6 |
Secluded Equipment Bay | At least once | Taylor Cruz, Jordan Hayes | Low | Equipment storage area inside Dulce Base. | 11 |
Underground Meeting Room | At least once | Jordan Hayes, Taylor Cruz, Sam Rivera | Medium | Small underground room near elevator to Dulce Base. | 24 |
Cramped Underground Meeting Room | At least once | Jordan Hayes, Taylor Cruz, Sam Rivera | Medium | Small underground room near elevator to Dulce Base. | 24 |
Central Communications Hub | At least once | Jordan Hayes, Taylor Cruz, Sam Rivera | High | Main communications area within Dulce Base. | 24 |
Central Hub | At least once | Jordan Hayes, Taylor Cruz, Sam Rivera | High | Main access point to deeper areas within Dulce Base. | 24 |
Mainframe Room | At least once | Jordan Hayes, Taylor Cruz, Sam Rivera | High | Dulce Base’s mainframe room; contains important computers and servers. | 24 |
Lab | At least once | Jordan Hayes, Taylor Cruz | High | Area within Dulce Base where alien technology is studied. | 15 |
Server Room | At least once | Sam Rivera, Alex Mercer | Medium | Room containing multiple servers. | 7 |
Washington D.C. | Multiple | Multiple (unspecified) | High | Government headquarters; source of orders and information. | Unknown |
Note: The “Total Interaction Weight” is a rough estimate based on the strength of relationships mentioned in the knowledge graph and the frequency of interactions described in the text. The Washington D.C. entry has an unknown weight because the specific people Alex interacted with there aren’t explicitly stated. The frequency of visits to some locations is inferred from the narrative.
Analysis: Both responses provide a table, but the graph RAG output incorporates more information and shows a higher level of consistency and stronger analysis of relationships.
5: I delve into creative applications by exploring questions related to book cover design.
Analysis: Given the subjective nature of such tasks, it’s recommended to personally experiment and evaluate the impact of incorporating graph databases. In my own assessment, I’ve observed that graph-enhanced RAG systems tend to yield more comprehensive and focused responses. To gain further insights, I urge you to engage with the provided code and explore your own creative inquiries.
General Observations:
Specific Examples:
Overall, the “think” sections provide compelling evidence that graph RAG consistently outperforms standard RAG in terms of efficiency, accuracy, and depth of analysis. The integration of a knowledge graph empowers graph RAG to access and utilize relevant information more effectively, leading to more comprehensive and insightful responses.
The fusion of graph databases and Retrieval-Augmented Generation (RAG) unlocks a new level of sophistication in language model applications. By grounding responses in a structured knowledge graph, we empower these models to reason about relationships, infer missing information, and provide more accurate and contextually relevant answers. This approach not only enhances the factual grounding of generated text but also opens doors to more complex and nuanced queries.
Imagine a world where AI assistants can truly understand the intricate connections between concepts, effortlessly navigate through a web of knowledge, and provide insightful answers that go beyond simple information retrieval. This is the promise of GraphRAG. From enhancing customer support interactions to powering advanced research tools, the practical applications are vast and transformative.
As we’ve seen through the analysis of various sample questions, GraphRAG consistently outperforms standard RAG systems, demonstrating improved accuracy, reduced hallucination, and the ability to handle complex queries with ease. This technology has the potential to revolutionize how we interact with information and unlock new possibilities in fields like education, research, and customer service.
The journey of GraphRAG is just beginning, and its future is filled with exciting potential. As knowledge graphs become more comprehensive and language models grow even more powerful, we can expect even more impressive advancements in the years to come. This is an invitation to join the exploration, experiment with the possibilities, and contribute to the evolution of this groundbreaking technology.
This post has just scratched the surface of what’s possible with graph-enhanced RAG. Now it’s your turn to explore and experiment! Here are some exciting next steps:
The world of graph-enhanced RAG is full of possibilities. Start exploring today and unlock the full potential of your RAG applications!
On this page