While experimenting with GraphRAG (see my notes on getting it working here), I wanted an example that I was very familiar with, didn’t have too much text so I went in search of an eBook.
I settled on The Nursery Rhymes of England. Within that I extracted the story of The Three Little Pigs. There’s quite a lot packed into one story – there’s many characters, locations, events with varying degrees of linkage and it’s written with some wording you don’t associate with today ie. furze instead of sticks.
This the the darker version of the story where the first 2 pigs are eaten by the wolf and the 3rd pig ends up setting up a trap for the wolf and boils him and proceeds to eat him.
After I’d run my pipeline it was quite interesting to look through the artifacts.
Nodes
Characters such as Old Sow, Man, Wolf have been pulled out and identified as Person, with a fair description.
Places such as House, Shanklin, Merry-Garden as Geography
Actions/Events such as as Getting apples, The fair as Events
Groups such as Three little pigs as Organisations
and then other like First Little Pig left unknown.
Relationships
The relationships it has drawn between the elements e.g. Shanklin and Fair – Shanklin is the location of the Fair.
I took that and put it into a mermaid diagram with some colour coding: Blue for Person, Yellow for Geo, Pink for Event, Green for Organisation so it’s a bit easier to see how it’s visualised the story. Even with such a short story the linkages are pretty vast.
Queries
It’s then interesting to see how well the queries work. For my llm I used gpt35 turbo and embeddings text-embedding-ada-002 with all the defaults in the project.
Looking at the relationships I notice:
- Shanklin is related to Fair
- Little Pig is related to Butter Churn
- The Little Pig is related to The Fair
but no further hard linkages
When asked: “Where was the butter churn located?”
The result: “In the story of the Three Little Pigs, the butter churn was bought by Little Pig at a fair [Data: Entities (15)]. The location of the fair is mentioned as Shanklin [Data: Entities (20)], and it is also referred to as “the fair” [Data: Entities (23)]. Therefore, it can be inferred that the butter churn was obtained at a fair in Shanklin.”
Cool to see the inferences here to get from The little pig to the –> the butter churn –> the fair –> Shanklin.
Asking a slightly more abstract question about a character (the wolf) by not using it’s name.
When asked: “What happened to the antagonist?”
Answer: “In the story of the Three Little Pigs, the antagonist is the wolf. The wolf attempts to enter the houses of the little pigs and ultimately eats one of them. However, in the end, the wolf is boiled and eaten by the third little pig, who he is frightened of [Data: Entities (11, 12); Relationships (2, 3, 12)].”
Here it correctly identifies I was referring to the wolf. I did find the reference to the wolf eating one of them to be incorrect and interesting. Which lead me to query about the welfare of each pig
When asked: “What happened to each of the pigs?”
Answer: “In the story of the Three Little Pigs, each pig builds a house out of different materials and has a different encounter with the wolf. The first little pig builds a house out of straw and is eaten by the wolf after he blows the house down [Data: Entities (8), Relationships (2)]. The second little pig builds a house out of furze and is also eaten by the wolf after he blows the house down [Data: Entities (9), Relationships (3)]. The third little pig builds a house out of bricks and is able to outsmart the wolf by boiling him in a pot and eating him for supper [Data: Entities (12), Relationships (11)].”
Here it has correctly navigated what happened to all three of the children but hasn’t inferred the mother would also be a pig.
I was curious what it thought of “the man” in the story. As it’s a bit ambiguous as to whether the “the man” supplying each of the pigs with their building materials is the same man of different, unrelated men
When asked: “How many men are in the story?”
Answer: “There is only one man mentioned in the story of the Three Little Pigs [Data: Entities (2)]. The man provides the first little pig with straw and the second little pig with furze to build their houses [Data: Sources (0)].”
Comparing to vector index
I was curious what the same embedding and model would do with just the out of the box vector index.
When asked: “Where was the butter churn located?”
The result: “The story does not mention where the butter churn was located. The little pig bought the butter churn at the fair and was going home with it when he saw the wolf coming. He then got into the churn to hide, and by so doing turned it round, and it rolled down the hill with the pig in it, which frightened the wolf so much, that he ran home without going to the fair.”
Here it’s unable to identify Shanklin as the location of the fair where the butter churn was purchased.
Winner: Graph
When asked: “What happened to the antagonist?”
Answer: “In the story of the Three Little Pigs, the antagonist, who is the wolf, was frightened when the butter churn with the little pig inside rolled down the hill past him, and he ran home without going to the fair 1 . Later on, the wolf tried to eat up the little pig but failed, and the little pig ended up boiling him and eating him for supper. After that, the little pig lived happily ever after.”
Correctly identifies the wolf but seems to completely forget there are another 2 pigs that are eaten.
Winner: Technically a tie but Graph gave a more accurate summary
When asked: “What happened to each of the pigs?”
Answer: “In the story of the Three Little Pigs, the first little pig built a house with straw but it was blown down by the wolf who then ate up the pig 1 . The second little pig built a house with furze, but it was also blown down by the wolf who then ate up the pig 1 . The third little pig built a house with bricks which the wolf couldn’t blow down. When the wolf found out he couldn’t blow the house down, he tried to trick the little pig by convincing him to go with him to get some turnips and apples. However, the little pig managed to outsmart the wolf and eventually boiled him and ate him up for supper”
Winner: Tie
When asked: “How many men are in the story?”
Answer:”There is no mention of the number of men in the story of the Three Little Pigs 1 2 . The story is about three little pigs who leave their mother’s house to seek their fortune and a wolf who tries to eat them.”
Winner: Graph. Correctly identifies at least 1 man.
Final thoughts
While a simple example, I think this quickly shows the power of using a knowledge graph when items in your input have relationships.
What would you ask about The Three Little Pigs?