The demo that took 2 seconds instead of 12
Vector search is genuinely good at one particular thing. It finds chunks of text that sound similar to the question you've just asked it. That works fine until the answer depends on how several separate facts connect to each other.
Recently, I had the pleasure of speaking at Devfest Wales, it was such a fun time with fun folks (except the 3hr drive... I hate long drives but it was worth it 👌🏽).
The audience and I watched the AI agent demo I put together on grounding agents in company data, and one comparison really stuck. One query finished in about 2 seconds, not 12. It also made just 2 requests to get its answer, where the other approach needed 8.
A drop from 8 requests to 2 is hard to explain away by sloppy arithmetic.
Vector search is genuinely good at one particular thing. It finds chunks of text that sound similar to the question you've just asked it. That works fine until the answer depends on how several separate facts connect to each other.
A knowledge graph goes about it in another way. It follows the relationships between things, moving from one record to the next, hop by hop. That's why a graph pipeline can easily pull multi-hop references that similarity search tends to skip.
This is exactly what regular agents were poor at. Without the graph, the agent in the demo needed 8 requests to piece the answer together. Each of those requests added time and gave the agent another chance to miss a link.
Research on graph retrieval points the same way, too. It says traditional RAG neglects the kinds of relationships that similarity alone simply can't capture. Citation links between papers are one example, since two papers can connect without sounding much alike.
None of this means I'd drop vector search entirely. Plenty of questions really are about finding similar text, and that approach handles them just fine. The graph sits alongside it and covers the relationships that similarity on its own can't see.
Splitting the work between the two seems simple enough. If a question needs multi-hop referencing, I'd take it through the knowledge graph pipeline. If it only needs similar passages, basic retrieval on its own may well be good enough.
Knowledge graphs don't replace vector search, they complement it. Similarity finds what sounds right, while the graph follows what's actually connected. They pull the multi-hop relationships basic RAG may miss, right where regular agents were poor.
Have you built a rag system lately? I'd love to learn about it via any of my socials - @chukaofili