Definition
Fine tuning is the process of adapting a pre-trained language model to perform better on specific tasks by training it on curated datasets. In the context of LLM Wiki, fine tuning represents an alternative approach to improving model performance — instead of modifying the model, the wiki modifies the knowledge it has access to.
Key Points
- Fine tuning changes the model’s weights; LLM Wiki changes the model’s context
- Fine tuning is expensive and requires expertise; LLM Wiki only requires adding source files
- Fine tuning improves model capability; LLM Wiki improves knowledge organization
- The two approaches are complementary: a fine-tuned model can still benefit from an LLM Wiki
- The RAG vs fine-tuning decision is a common architectural choice that the LLM Wiki pattern offers a third option to
Related Concepts
- rag-vs-fine-tuning — Direct comparison between the two approaches
- rag — Retrieval-based alternative to fine tuning
- llm-wiki — Offers a compile-time alternative to both RAG and fine tuning