LLM Wiki vs. RAG Knowledge Base

LLM Wiki vs. RAG

In early April 2026, Andrej Karpathy introduced a new design pattern — LLM Wiki [1]. It is a framework for building personal knowledge bases using language models. The idea looks promising as it attempts to address significant weaknesses of LLM+RAG systems. It introduces new mechanisms and supplements the system with a knowledge base whose expansion and maintenance can be fully autonomous (managed by the LLM), while remaining human-auditable. It solves the problem of knowledge non-accumulation and improves the ability to synthesize information from multiple sources.

“The idea here is different. Instead of just retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources.”

LLM Wiki is a “Wiki-First” type of RAG system. It uses a RAG mechanism to search a knowledge base that is continuously created and updated by the model itself. As a result, the system “learns” and builds an increasingly broader context over time, rather than starting from scratch with every new query.

This concept represents a paradigm shift:

A move from retrieval (RAG) to the compilation of raw documents into a coherent, constantly updated knowledge base.

It treats knowledge not as a static library to be searched, but as “living source code” that AI actively rebuilds and maintains.

Key Differences in a Nutshell

Since LLM Wiki is still a novelty, I focused this comparison primarily on the conceptual level. Of course, not all RAGs are created equal, and LLM Wiki itself will evolve over time. However, certain fundamental differences are already plain to see. Only when we recognize them and understand their real impact and limitations will we be able to build top-tier solutions with full awareness.

Here is an interesting comparison of the differences between LLM Wiki and RAG, based on article [2] by Emily Winks. According to the author:

“The sharpest differences appear at three axes: scale, infrastructure, and governance. LLM wikis win on simplicity and token efficiency below the 50,000-100,000 token threshold. RAG wins on scale, dynamism, and multi-user access. Neither wins on enterprise data governance - that requires a separate layer entirely.”

Summary of the most important points:

• Architecture: LLM Wiki loads the entire, organized index into the context. RAG works differently – at the moment a question is asked, it dynamically “pulls” specific fragments of knowledge from a vector database.
• Scale: LLM Wiki performs great within the 50–100 thousand token limit. RAG has no “ceiling” – (when building a vector database) it can handle millions of documents.
• Infrastructure: LLM Wiki is a nearly maintenance-free solution – you don’t need additional backend infrastructure. RAG requires a whole machinery: a vector database, embedding mechanisms, and a retrieval layer.
• Data Management: Neither approach solves corporate data security on its own. A separate system is required, such as a professional data catalog with advanced access control.

The RAG Paradox: Why Are We Reinventing the Wheel?

Currently, RAG (Retrieval-Augmented Generation) is the standard. You feed in documents, ask a question, and the AI searches the database. Sounds good? Only on the surface. RAG comes with dozens of challenges – it requires a solid engineering approach and strong architectural foundations. With every query, the system searches the entire collection of documents (or rather, their fragments), selects the most important information, and starts all synthesis work from scratch. Of course, identical queries can be handled by the cache. However, if you ask something very similar tomorrow, the basic RAG will once again pull the same document fragments (often a fairly large stack), rearrange them, and will have to laboriously piece together the meaning from this jumble all over again.

To make matters worse, processing happens on a limited number of text fragments, meaning the vast majority of them will not be analyzed when constructing the answer. While this is somewhat desirable – we don't want to consider content unrelated to the question – it has side effects. Very often, loosely related fragments that are still important in a given context are simply omitted.

LLM Wiki reverses this process. Instead of searching for information only at query-time, we compile knowledge at ingest-time. AI becomes your personal editor, rewriting and connecting facts into a coherent whole in real-time. This way, we don’t have to search all documents from scratch with every query. It’s enough to insert the already compiled knowledge into the context (if it fits the context window) or a selected, organized part of it (when the base is too extensive or LLM limits are too restrictive). In the LLM Wiki approach, the knowledge base is significantly more stable. Even if you change the query, the AI operates on the complete set of information every time, ensuring significantly higher response consistency.

LLM Wiki: A Revolution or Just Andrej Karpathy's "Hype"?

Opinions are divided. Skeptics criticize it as nothing groundbreaking, arguing the solution has no future and does not scale. Optimists praise the elements that function better and paint joyful visions of the future of LLM Wiki. I think it is still too early to decide. Instead, I suggest we look at the pros and cons of this solution.

Looking at this issue systematically, I see great value in separating the knowledge base (or knowledge graph) from the LLM and the RAG mechanism themselves. However, I have doubts whether a Wiki, as a data organization model, will hold up at a large scale and whether we might need a different abstraction here. Indeed, the Wiki format is extremely useful when a human audits the content or when we transfer knowledge into the prompt context.

However, the gathered knowledge could reside in dedicated structures, while a separate tool layer handles its auditing and feeding into the model.

Such tools could simply extract the data and convert it into a readable Markdown format only when needed.

Although the topic is relatively fresh, there have already been attempts to use LLM Wiki in combination with a knowledge graph — see, for example, video [3] by Nodus Labs.