Definition
Embeddings are dense vector representations of text that capture semantic meaning. They are central to RAG systems, where documents are converted to vectors and compared by cosine similarity. In the LLM Wiki pattern, embeddings play a secondary role — the primary mechanism is structured markdown with explicit wikilinks rather than vector similarity.
Key Points
- In traditional RAG, embeddings are the core retrieval mechanism: documents are chunked, embedded, and searched by similarity
- In the LLM Wiki pattern, knowledge is organized through explicit wikilinks and structured pages — not vector proximity
- Embeddings can still be useful in LLM Wiki for: routing queries to the right wiki, discovering potential new links, and initial clustering of sources
- The Level Up Coding deep dive mentions embeddings as part of step 4 of the ingest pipeline (“Embed — updated page is re-embedded”)
Related Concepts
- rag — RAG relies on embeddings for retrieval; LLM Wiki does not
- retrieval-augmented-generation — Embeddings are the foundation of RAG retrieval
- llm-wiki — LLM Wiki uses structured markdown instead of vector embeddings
Sources
^[raw/articles/karpathy-llm-wiki-gist.md]