Academic Databases
Explore Our Semantic Databases
STEM & Tech
Math, Physics, CS & Engineering...
Economics
Macro, Micro, Econometrics...
Business
Management, Marketing, Finance...
Health & Life Sciences
Biology, Medicine, Neuroscience...
Humanities & Social Sciences
Psychology, Law, Philosophy, History...
General Studies
General & interdisciplinary studies...
Explore Topics
Features

Fine-Tune LLMs for Ollama: Step-by-Step Guide in Python

Added: 2026-08-06

Deep Dive

Basic proficiency in Python programming and familiarity with deep learning frameworks like PyTorch or the Hugging Face Transformers library.

Hugging Face 101: Your First Steps Coding with Transformers

Hugging Face 101: Your First Steps Coding with Transformers

17.5K

This video tutorial guides beginners through setting up a Python environment with virtual environments, installing necessary libraries like pip, PyTorch, and the Hugging Face Transformers library, and demonstrates how to download and run pre-trained AI models using code, including handling common errors like missing dependencies and device configuration issues.

Click to watch →

Wprowadzenie do PyTorch – od podstaw do sieci neuronowych [WARSZTAT]

Wprowadzenie do PyTorch – od podstaw do sieci neuronowych [WARSZTAT]

1.2K

This comprehensive segment covers deep learning fundamentals including automatic feature extraction from unstructured data like text and images. The curriculum introduces PyTorch as an open-source framework supporting CPU and GPU computation with automatic gradient calculation. Key topics include tensor operations, broadcasting, matrix multiplication, and GPU acceleration providing 30,000x faster matrix multiplication than CPUs. The MNIST dataset serves as a standard introduction to supervised learning with 28x28 pixel handwritten digits. Hugging Face Transformers provides pre-trained models for computer vision, NLP, and audio tasks including BERT, GPT2, and T4. The segment demonstrates practical applications including sentiment analysis, text generation, and translation models.

Click to watch →

Coding Tutorial - PyTorch, Hugging Face Transformers, Re-Implementing and Modifying BERT

Coding Tutorial - PyTorch, Hugging Face Transformers, Re-Implementing and Modifying BERT

322

This tutorial demonstrates how to build and customize BERT using PyTorch and Hugging Face Transformers, covering PyTorch fundamentals including tensors with GPU support and automatic differentiation, then implementing BERT's architecture consisting of an embedding layer and stacked transformer layers with multi-head attention and feed-forward networks, and finally showing how to modify BERT by adding a custom pooling operation to transform token-level embeddings into a single sequence-level embedding for tasks like sequence classification.

Click to watch →

Sentiment Analysis with BERT Neural Network and Python

Sentiment Analysis with BERT Neural Network and Python

152.7K

This tutorial demonstrates how to perform sentiment analysis using a pre-trained BERT neural network via the Hugging Face Transformers library in Python, covering the complete workflow from installing dependencies (PyTorch, Transformers, Requests, BeautifulSoup, Pandas, Numpy) to instantiating the model, encoding text for sentiment scoring, and applying the pipeline to scrape and analyze Yelp reviews at scale.

Click to watch →

Finetune LLMs to teach them ANYTHING with Huggingface and Pytorch | Step-by-step tutorial

Finetune LLMs to teach them ANYTHING with Huggingface and Pytorch | Step-by-step tutorial

133.4K

The Hugging Face Transformers library provides APIs for downloading and training state-of-the-art pre-trained models. Users specify model IDs to load tokenizers and models. Causal language models predict next words based on previous context, unlike masked models. Tokenization converts text to integer tokens, requiring padding for batch processing. Attention masks filter out padding tokens during computation. Chat templates format prompts with system/user/assistant roles for instruction tuning.

Click to watch →

Deep Dive

Fundamental understanding of Large Language Models (LLMs), including the transformer architecture, tokenization, and the difference between base and instruct models.

🔴 LLAMA 3 - ¡META Vuelve a REVOLUCIONAR el mundo de la IA!

🔴 LLAMA 3 - ¡META Vuelve a REVOLUCIONAR el mundo de la IA!

156.8K

Large language models come in two primary categories: base models and instruct models. Base models are fundamental language understanding systems designed to predict the next token in a sequence, learning to continue phrases and anticipate words. Instruct models are base models that have been fine-tuned specifically for chatbot and assistant tasks, enabling them to understand and respond to user queries in conversational contexts. Most practical applications involve instruct models rather than raw base models.

Click to watch →

Large Language Models (LLMs) - Everything You NEED To Know

Large Language Models (LLMs) - Everything You NEED To Know

435K

Large Language Models (LLMs) are neural networks trained on massive text data to understand and generate natural language, differing fundamentally from traditional programming by learning patterns from examples rather than following explicit instructions; they work through tokenization (splitting text into units), embeddings (converting tokens to numerical vectors representing semantic meaning), and transformer architectures with attention mechanisms that enable context-aware predictions, though they face challenges including bias, hallucinations, high computational costs, and limitations in mathematical reasoning despite their remarkable capabilities in tasks like translation, summarization, and creative writing.

Click to watch →

Low Level Technicals of LLMs: Daniel Han

Low Level Technicals of LLMs: Daniel Han

53.6K

This section covers Transformer architecture fundamentals and tokenization. The speaker explains that the Transformer is the foundation behind all modern language models (Gemma, Llama, Mistral, etc.), designed for sequence modeling and applicable to any sequential data. The architecture consists of stacked layers with attention mechanisms and feed-forward networks. The speaker introduces Transformer Plus+, an enhanced version with RoPE embeddings, SwiGLU activation, RMSNorm, and untied weights, considered the best current Transformer architecture. The section details Transformer layer components: LayerNorm, RoPE embeddings, attention mechanism, residual connections, and another LayerNorm, repeated L times (e.g., 32 layers in Llama 3). The speaker then covers tokenization, explaining that computers only understand numbers, so text must be converted to numerical IDs. The speaker demonstrates creating a simple tokenizer by combining punctuation with words and ignoring spaces, then discusses vocabulary size, word stemming, and lowercasing as techniques to reduce vocabulary size.

Click to watch →

KIMI just BROKE the AI Industry...

KIMI just BROKE the AI Industry...

107.7K

Large language models are typically released in two versions: base foundation models and instruct models. Base models provide the foundational capabilities and are intended for researchers and developers who will fine-tune them. Instruct models are chatbot-style versions designed for direct user interaction with conversational understanding.

Click to watch →

Por dentro das LLMs como ChatGPT - O Curso Completo

Por dentro das LLMs como ChatGPT - O Curso Completo

215K

Large Language Models (LLMs) are AI systems that can converse about any topic, execute tasks, and create documents. They belong to Natural Language Processing (NLP), which studies human language. The Transformer architecture, proposed by Google Brain in 2017, revolutionized NLP by outperforming previous Recurrent Neural Networks (RNNs). Training requires massive datasets from CommonCrawl, a non-profit that crawls the entire internet. The data cleaning process involves: (1) RL filtering to remove sensitive content; (2) Text extraction to remove HTML tags; (3) Language filtering; (4) Privacy removal. Tokenization transforms text into tokens through UTF encoding, byte grouping, and Byte Pair Encoding (BPE), making processing more efficient. Transformers measure statistical relationships between token sequences to generate text by predicting the most likely next token at each step.

Click to watch →

Deep Dive

Conceptual knowledge of Parameter-Efficient Fine-Tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA).

LoRA & QLoRA Fine-tuning Explained In-Depth

LoRA & QLoRA Fine-tuning Explained In-Depth

140.5K

Large language model development progresses through pre-training, instruct tuning, safety tuning, and domain fine-tuning. Full parameter fine-tuning requires storing and updating billions of weights, demanding massive GPU resources. LoRA addresses this by tracking changes in two smaller matrices that multiply to form full-size updates, reducing trainable parameters dramatically. QLoRA extends this by applying quantization that compresses 16-bit parameters to 4-bit representations while preserving original precision. Together, these methods enable efficient fine-tuning on consumer-grade hardware, democratizing access to large language model customization.

Click to watch →

LLM Fine Tuning Crash Course | LLM Fine Tuning Tutorial

LLM Fine Tuning Crash Course | LLM Fine Tuning Tutorial

44K

LoRA addresses computational challenges of full fine-tuning by keeping original model weights frozen and adding small trainable delta W matrices. The critical innovation lies in matrix decomposition: delta W (d×d) is split into A (r×d) and B (d×r) matrices, reducing trainable parameters from O(d²) to O(rd). With rank r=8 and d=512, this reduces parameters from 262,144 to just 8,192. Common rank values range from 4-16, selected based on dataset characteristics. This parameter-efficient approach maintains effectiveness while dramatically reducing computational overhead.

Click to watch →

AI Infrastructure | Part 5 | AI Model Fine-tuning

AI Infrastructure | Part 5 | AI Model Fine-tuning

126

LoRA (Low-Rank Adaptation) freezes all pre-trained weights and introduces two small trainable matrices (A and B) with dimensions between 4-64, updating only 1-4% of parameters. QLoRA extends this by loading base models in lower precision (4-bit or 8-bit) while keeping adapters in full precision, achieving 60-70% memory reduction. QLoRA uses page optimizer to handle memory spikes during gradient accumulation. These methods enable efficient fine-tuning of large models (70B+ parameters) that would otherwise require excessive memory, making domain adaptation feasible on standard hardware.

Click to watch →

How Fine-Tuning Works: LoRA & QLoRA Explained Simply

How Fine-Tuning Works: LoRA & QLoRA Explained Simply

Parameter-efficient fine-tuning (PEFT) techniques like LoRA (Low Rank Adaptation) and QLoRA (Quantized LoRA) enable efficient retraining of large neural networks by freezing pre-trained model parameters and training only low-rank matrices (e.g., 6x1 and 1x6 instead of 6x6) to capture domain-specific changes, reducing computational cost from millions to thousands of trainable parameters while maintaining model performance.

Click to watch →

Тихомиров М.М. - Обучение больших языковых моделей - 7. Методы Fine-tuning и LoRA

Тихомиров М.М. - Обучение больших языковых моделей - 7. Методы Fine-tuning и LoRA

132

This section introduces LoRA (Low-Rank Adaptation), a breakthrough PEFT method. LoRA decomposes weight updates into low-rank matrices (B × A), reducing trainable parameters by orders of magnitude. The rank R controls the trade-off between model capacity and memory usage. LoRA can achieve results comparable to full fine-tuning while requiring only 0.3% of trainable parameters. The alpha parameter controls the scaling of the update and can be tuned differently for different layers.

Click to watch →

Deep Dive

Familiarity with command-line interfaces (CLI) and basic environment setup for running local AI models.

Cómo ejecutar Inteligencia Artificial en tu PC con C# y sin pagar nada

Cómo ejecutar Inteligencia Artificial en tu PC con C# y sin pagar nada

3.2K

To run local AI models, download CLI binaries from GitHub releases (selecting appropriate OS and architecture like Windows X64 or ARM) and pre-trained models from Hugging Face (such as TinyLlama). Copy the DLL files and CLI executable to a folder, then run the executable with the model name and prompt as arguments. Alternatively, Ollama provides a more convenient interface for local AI execution. After installing Ollama from their website, run models by specifying the model name (e.g., 'ollama run llama3'). If the model isn't already downloaded, Ollama will automatically download it. You can then interact with the model by sending commands through the command line.

Click to watch →

Taller de IA Local desde cero: sin nube, privada y gratis

Taller de IA Local desde cero: sin nube, privada y gratis

7.7K

Ollama is a command-line tool for running local AI models, offering a lighter alternative to LM Studio. Installation is straightforward via terminal commands or pre-built executables. Key features include: (1) Simple model management - list, download, and switch between models; (2) Support for reasoning models and tooling-enabled models; (3) Configuration options for context window size (128-256K tokens); (4) Hardware-based model recommendations; (5) Documentation for API integration and programmatic access. Ollama is particularly useful for developers who want to integrate AI into their workflows programmatically.

Click to watch →

สอนติดตั้ง AI บนเครื่องคอมเราเอง ไม่ต้องเสียเงิน!

สอนติดตั้ง AI บนเครื่องคอมเราเอง ไม่ต้องเสียเงิน!

10K

To run AI models locally, users must first install the Ollama software, then open a terminal window (accessed by typing 'cmd' in Windows). Users can execute commands to download and run specific models by specifying the model name and parameters. The system automatically downloads the model files to the local machine for processing.

Click to watch →

SO nutzt du Claude Code GRATIS! (Gemma 4 + Claude Code)

SO nutzt du Claude Code GRATIS! (Gemma 4 + Claude Code)

931

To run local AI models, install Ollama via terminal command or download button. Select appropriate model size based on hardware: 31B, 26B, 4B, or 2B parameter versions. Smaller models suit weaker computers. Download models using terminal commands (9.6GB for Gemma 4 latest). Verify installation by checking progress bars reach 100%. Test models by sending messages and observing thinking processes. For Cloud Code integration, use Visual Studio Code as the IDE, install Cloud Code via Extensions panel, and activate through terminal commands or sidebar buttons.

Click to watch →

Uruchom lokalne modele AI na swoim komputerze

Uruchom lokalne modele AI na swoim komputerze

566

Ollama is open-source software that enables running large language models locally without cloud servers. It functions as a model manager and execution environment, simplifying model downloading, configuration, and application integration. Installation is done via command line or desktop application. The command-line interface provides simple commands: 'ollama' to start the interface, 'ollama list' to show installed models, 'ollama run <model-name>' to launch a specific model, and 'ollama help' for available options. LM Studio is a graphical application offering a more user-friendly interface with chat functionality, developer tools for API integration, and a model library. The interface shows which models are compatible with your hardware (green indicators for compatible models). Models can be configured with system prompts defining their behavior and character. Some models like Gemma 4 include vision capabilities for analyzing images and documents.

Click to watch →

Fine-tuning Basics

Data Preparation

Notebook Setup

Model Training

Training Execution

Inference Testing

Model Export

Ollama Integration

Final Results

Fine-tuning Basics

0:00

Playing Section

1
Fine-tuning adapts pre-trained models to specific tasks with less data.
2
Distinct from parameter tuning; improves domain performance but may reduce generality.

Deep Dive

Integrating your locally deployed Ollama model into Retrieval-Augmented Generation (RAG) systems using frameworks like LangChain or LlamaIndex.

Ollama meets LangChain

Ollama meets LangChain

59.6K

This tutorial demonstrates how to run large language models locally using Ollama and integrate them with LangChain for various tasks. The video covers three main examples: loading Ollama models (like LLaMA-2) using LangChain's pre-made LLM wrapper with streaming callbacks, creating basic chains with prompt templates and parameters like temperature and max tokens, and building a RAG (Retrieval Augmented Generation) system that scrapes web pages, splits content, stores it in ChromaDB vector database, and uses the LLM to answer questions based on the retrieved information. This enables developers to perform tasks like extracting headlines from websites without writing custom parsing code, as the language model handles the information extraction automatically.

Click to watch →

Python AI Web Scraper Tutorial - Use AI To Scrape ANYTHING

Python AI Web Scraper Tutorial - Use AI To Scrape ANYTHING

346.3K

Ollama is a tool that allows running open-source large language models locally without API tokens. LangChain provides integration between Python code and various LLMs. To use them together: (1) install Ollama and pull a model (e.g., llama3), (2) import from langchain_community.llms import OllamaLLM, (3) create a model instance specifying the model name, (4) define a prompt template containing variables for Dom content and parse description, (5) create a chain by piping the prompt to the model, then invoke the chain with specific content and description inputs.

Click to watch →

Gemma 2 - Local RAG with Ollama and LangChain

Gemma 2 - Local RAG with Ollama and LangChain

24.1K

A fully local RAG (Retrieval-Augmented Generation) system can be built using Gemma 2 9B model via Ollama, Nomic embeddings, and ChromaDB, where the indexer processes documents through text splitting and embedding, the retriever fetches relevant context using similarity search, and the LLM generates responses based on the retrieved context, enabling offline AI applications without cloud dependencies.

Click to watch →

Python Advanced AI Agent Tutorial - LlamaIndex, Ollama and Multi-LLM!

Python Advanced AI Agent Tutorial - LlamaIndex, Ollama and Multi-LLM!

170.1K

This tutorial demonstrates how to build an advanced AI agent that leverages multiple large language models (LLMs) locally using the LlamaIndex framework and Ollama, enabling Retrieval-Augmented Generation (RAG) capabilities where the agent can read code files and documentation, generate new code, and save outputs to files through a multi-step pipeline involving query engines, code readers, output parsers, and retry handling mechanisms.

Click to watch →

Run DeepSeek R1 Locally With Ollama | Build a Local Gradio App for RAG

Run DeepSeek R1 Locally With Ollama | Build a Local Gradio App for RAG

17.6K

This tutorial demonstrates how to securely run the DeepSeek R1 large language model locally on your machine using Ollama, then build a Retrieval-Augmented Generation (RAG) system that processes PDF documents through text splitting, embedding generation with vector databases, and combines retrieved context with model queries to answer questions more accurately while reducing hallucinations.

Click to watch →

Deep Dive

Exploring advanced alignment techniques such as Direct Preference Optimization (DPO) or Reinforcement Learning from Human Feedback (RLHF).

Reinforcement Learning, RLHF, & DPO Explained

Reinforcement Learning, RLHF, & DPO Explained

15.4K

This video explains three optimization techniques for aligning AI language models with human preferences: (1) RLHF uses reinforcement learning where a reward model scores outputs and an optimizer updates the policy model through trial and error, but faces challenges with instability and expensive human labeling; (2) DPO directly optimizes the language model on preference data without explicit reward modeling, achieving equivalent or better results than RLHF while being simpler, faster, and more robust to temperature changes; (3) KTO applies prospect theory from behavioral economics to maximize utility by considering human risk aversion, using a sigmoid value function and handling noisy/inconsistent data better than other methods, with tunable hyperparameters for balancing risk aversion between desirable and undesirable examples.

Click to watch →

Aligning LLMs with Direct Preference Optimization

Aligning LLMs with Direct Preference Optimization

33.9K

Direct Preference Optimization (DPO) is a simplified alignment technique that trains language models directly on human-preference data without requiring a separate reward model, using a differentiable loss function that maximizes the difference between log probabilities of preferred versus dispreferred responses; this approach achieves comparable or better results than traditional reinforcement learning from human feedback (RLHF) while being more computationally efficient and easier to implement, as demonstrated by its successful application in training Zephyr, a state-of-the-art 7B parameter chat model.

Click to watch →

CS 285: Eric Mitchell: Reinforcement Learning from Human Feedback: Algorithms & Applications

CS 285: Eric Mitchell: Reinforcement Learning from Human Feedback: Algorithms & Applications

8.5K

Reinforcement Learning from Human Feedback (RLHF) is a four-step pipeline for training large language models: (1) unsupervised pre-training on internet text, (2) supervised fine-tuning on human demonstrations, (3) fitting a reward model using preference data collected from human comparisons of model responses, and (4) reinforcement learning with the reward model and KL-divergence constraint to optimize the policy. Direct Preference Optimization (DPO) simplifies this process by deriving a closed-form optimal policy directly from preference data, eliminating the need for separate reward model training and RL optimization loops. DPO achieves this by parameterizing the reward function as the log probability ratio between the current policy and a reference model, transforming the RLHF objective into a tractable classification loss that can be optimized directly without rollouts.

Click to watch →

Direct Preference Optimization: Forget RLHF (PPO)

Direct Preference Optimization: Forget RLHF (PPO)

16.1K

Direct Preference Optimization (DPO) is a new training methodology developed by Stanford University that bypasses the complex two-step process of Reinforcement Learning from Human Feedback (RLHF) by directly optimizing language models using a single maximum likelihood objective based on human preference data, achieving similar or better performance than traditional RLHF methods while being substantially simpler to implement and train.

Click to watch →

Direct Preference Optimization (DPO) - How to fine-tune LLMs directly without reinforcement learning

Direct Preference Optimization (DPO) - How to fine-tune LLMs directly without reinforcement learning

29.4K

Direct Preference Optimization (DPO) is a method for fine-tuning Large Language Models using human feedback without requiring reinforcement learning. Unlike Reinforcement Learning with Human Feedback (RLHF), which uses two separate neural networks (a policy network and a reward model), DPO trains only the Transformer model directly by embedding the reward function within its loss function. The Bradley-Terry model converts human reward scores into probabilities using the sigmoid function, while KL Divergence prevents the model from changing too drastically from its original behavior. This approach is more efficient than RLHF as it eliminates the need to train a separate reward model.

Click to watch →

Deep Dive

Understanding model quantization formats (like GGUF) and how to manually convert and package custom model weights for Ollama.

Ollama: How To Create Custom Models From HuggingFace ( GGUF )

Ollama: How To Create Custom Models From HuggingFace ( GGUF )

21.9K

This section covers the foundational knowledge for working with Ollama: (1) Installation verification through terminal commands and localhost:11434 browser check, (2) Understanding model file components including 'from' lines, templates with system/user/assistant roles, and stop parameters, (3) GGUF file format as the latest unified format containing all model weights and information, (4) Key considerations when selecting GGUF models including quantization method, bits, RAM requirements, and use cases. The balanced quantization provides optimal trade-off between performance and resource usage.

Click to watch →

Ollama - Loading Custom Models

Ollama - Loading Custom Models

54.7K

To install custom quantized models in Ollama, download the GGUF format model files (quantized versions of models like Mistral 7B), create a model configuration file specifying the weights path and system prompt template, then use the Ollama CLI to build and run the model, enabling access to fine-tuned or specialized models not available in the default Ollama model library.

Click to watch →

Local AI just leveled up... Llama.cpp vs Ollama

Local AI just leveled up... Llama.cpp vs Ollama

162.9K

GGUF is the model format supported by Llama CPP, different from the standard SafeTensor format used by most models. Quantization reduces model size by lowering precision: Q8 (8-bit) halves the size compared to F16 (float16), while Q4KM (4-bit) reduces it to approximately one-quarter of the original F16 size. When selecting models from Hugging Face, look for GGUF in the filename to ensure compatibility with Llama CPP. The model identifier typically includes organization, model name, and quantization suffix (e.g., quen34B-Q8-GGUF).

Click to watch →

Adding Custom Models to Ollama

Adding Custom Models to Ollama

41.4K

This tutorial demonstrates how to convert and quantize a pre-trained AI model from Hugging Face for use with Ollama, covering the complete workflow including downloading the model using HF Downloader, converting it to GGUF format with Docker, applying 4-bit quantization for efficiency, creating the model configuration with appropriate templates, and finally publishing it to Ollama for sharing with others.

Click to watch →

How to Convert/Quantize Hugging Face Models to GGUF Format | Step-by-Step Guide

How to Convert/Quantize Hugging Face Models to GGUF Format | Step-by-Step Guide

11.4K

This tutorial demonstrates how to convert Hugging Face models to GGUF format for use with applications like LM Studio and Olama, covering the complete workflow: downloading models from Hugging Face, locating cached model files, cloning the llama.cpp repository, setting up a Python virtual environment, converting models to GGUF float16 (f16) format using the convert.py script, and re-quantizing to lower precision types like Q4_K_M for optimized memory usage while maintaining model performance.

Click to watch →

Deep Dive

Implementing systematic evaluation frameworks (e.g., MT-Bench, AlpacaEval) to quantitatively measure the performance improvement of your fine-tuned model.

Ajustando LLMs com Parameter efficient fine-tunning (PEFT)

Ajustando LLMs com Parameter efficient fine-tunning (PEFT)

743

Quantitative model evaluation involves systematically comparing model outputs against reference text using automated metrics. The process includes: loading evaluation libraries (like ROUGE), preparing reference texts (human-generated summaries) and generated texts (model outputs), computing metrics (ROUGE-1, ROUGE-2, ROUGE-L), and comparing results across different model versions. This approach provides objective, numerical evidence of model performance improvements or degradation.

Click to watch →

Build an LLM from Scratch 7: Instruction Finetuning

Build an LLM from Scratch 7: Instruction Finetuning

35.7K

Systematic model evaluation enables quantitative comparison of fine-tuning effectiveness. The process involves extracting and saving model responses for all test examples by iterating through the dataset, formatting examples using training templates, generating responses, and cleaning outputs. Saving the fine-tuned model with appropriate naming conventions preserves progress for future use. Automated evaluation using another LLM (such as Llama 3.2 via Ollama) scores responses against correct answers, typically on scales from 0 to 100. Results vary by evaluation model size—smaller models may produce inconsistent scores. Best practices include using improved prompt templates with structured scoring rubrics and employing larger evaluation models when feasible. This infrastructure supports iterative experimentation, enabling practitioners to systematically compare model performance across different configurations and hyperparameter settings.

Click to watch →

Top Technologies to Learn in 2025–26 | Which One Should You Pick?

Top Technologies to Learn in 2025–26 | Which One Should You Pick?

Writing evaluations (tests/benchmarks) quantifies whether fine-tuned models outperform base models. This involves creating test suites that measure specific capabilities and comparing results against baselines. Open source evaluations exist that companies optimize for, providing standardized benchmarks for comparing different model versions and fine-tuning approaches.

Click to watch →

Developing an LLM: Building, Training, Finetuning

Developing an LLM: Building, Training, Finetuning

129.2K

Preference fine-tuning refines model behavior by incorporating human preference signals when multiple valid responses exist, particularly valuable for safety and helpfulness improvements. Evaluation requires multiple complementary metrics: MMLU measures multiple-choice question answering ability (0-100 score), AlpacaEval compares responses against GPT-4 using win-rate statistics, and Chatbot Arena provides crowdsourced pairwise comparisons. No single metric captures all model capabilities. Practical development follows rules: continued pre-training instills new knowledge effectively, fine-tuning excels at specialized tasks, and preference fine-tuning enhances conversational agent behavior. The Meta Code Llama project exemplifies combining these stages.

Click to watch →

Day 72/75 Hugging Face LLM Evaluation Framework [Explained] | BLEU ROUGE HELM | Python GenAI

Day 72/75 Hugging Face LLM Evaluation Framework [Explained] | BLEU ROUGE HELM | Python GenAI

1.5K

The Hugging Face evaluate library provides a comprehensive solution for building custom evaluation frameworks for large language models. It supports multiple evaluation types including metric-type evaluation (comparing predictions against ground truth), comparison-type evaluation (comparing multiple models against a single reference), and measurement-type evaluation (analyzing dataset properties and model characteristics). Key metrics include accuracy, precision, recall, F1 score, and word length. The library enables combining multiple metrics simultaneously, saving results to JSON files, and provides bootstrapping for confidence interval estimation to assess statistical significance of performance differences.

Click to watch →

Fine-tuning Basics

Data Preparation

Notebook Setup

Model Training

Training Execution

Inference Testing

Model Export

Ollama Integration

Final Results

Fine-tuning Basics

0:00

Playing Section

1
Fine-tuning adapts pre-trained models to specific tasks with less data.
2
Distinct from parameter tuning; improves domain performance but may reduce generality.

245.9K views6.8K22:02@TechWithTimOriginal Release: 2025-06-27

Fine-tuning is the process of taking a pre-trained language model and adapting it to perform better at specific tasks by training it on domain-specific data, which requires significantly less data and computational resources than training from scratch but may reduce general task performance; the process involves gathering relevant training data, using libraries like Unsloth for efficient training, and exporting the model in formats compatible with platforms like Ollama for local deployment.

[00:00:00]Today, you'll learn how to fine-tune LLMs in Python for use in O Lama. I'll walk you through it step by step, give you all the code, and show you how to test it out. So, let's go ahead and get started. Now, first, I want to discuss what fine-tuning is and when you should actually do it. So, fine-tuning is taking a pre-trained language model and teaching it to be better at your specific task. So, think of this like hiring an experienced chef and training them on your restaurant's particular recipes rather than teaching someone to cook completely from scratch. So, here's how it works. Instead of training a model from zero, you start with something like GBT or Claude that already understands human language. Then you feed these examples of your specific use cases. So maybe customer service conversations, legal documents, or medical records. The model then adjusts its existing knowledge to excel at your specific domain using those examples.

[00:00:52]Now, this is completely different from parameter tuning, which is just adjusting settings like temperature or top K to change how the model performs.

[00:00:59]Parameter tuning is like adjusting your car's radio, whereas fine-tuning is like teaching your car to drive in a completely different neighborhood. Now, when should you fine-tune? Well, there's three main scenarios. First is when you need consistent formatting or style that prompting alone can't achieve. So something like you want to output a particular JSON model or something or you want the model to write in a very particular format. Second, you have a lot of domain specific data that the model just hasn't seen before. So things like advanced medical records or customer service logs or information that it just wouldn't naturally know because it's very specific to what it is that you're doing. And then third, when you need to reduce costs by using a smaller specialized model instead of a really massive larger one. Now, the key advantage to fine-tuning rather than training from scratch is that you need way less data and compute power to do this. Instead of millions of examples and months of training, you might need thousands or even hundreds of examples and maybe minutes to train or hours to train, not weeks. But here's the catch.

[00:02:00]When you do fine-tune, you can make these models worse at general tasks, but you will make them better at yours. So, if you do fine-tune one of these models, keep in mind that it will get worse at general tasks, at least typically, but be much better at what you're fine-tuning it to do. Anyways, with that information out of the way, I want to show you how to do this fine-tuning. So, let's go over to the computer and I'm going to show you how to set this up in Python with some example data that I have. So, I'm on the computer now. And the first step that we need to complete if we want to fine-tune a model is we need to gather the data that we're going to fine-tune that model with. Now, this is the most important step. If you have bad data, you're going to have a poorly fine-tuned model. So, make sure that you take your time here and gather the correct data. Now, for this video, I've just used AI to generate a pretty simple data set which demonstrates HTML extraction. So, you can see that I have some HTML like a div, an H2 tag, a price tag, whatever, etc. Okay, there's a bunch of information kind of inside of here. And what I'm expecting the model to give me is a nicely formatted output that tells me the name, price, category, and manufacturer of this particular data. Now, this is very specific because I just want to extract data from these types of tags. However, you could do anything as advanced as you want here.

[00:03:13]And you can see that I have 500 examples in this JSON file where I have some sample input and some sample output.

[00:03:20]Now, this can be literally anything that you want because all you're doing to fine-tune the model is you're giving it example prompts and then example outputs of what the result or the answer from the LLM should be based on that prompt.

[00:03:33]So, you can use customer data again, medical reports, files, whatever. In this case, just simple HTML extraction.

[00:03:39]So, you can see kind of how it works and I can compare it easily in this video.

[00:03:42]If you want this specific data set, I'll leave a link to download it in the description. Okay. Now, for this video, we're going to use something called Unsloth for doing our fine training.

[00:03:51]This is open source. It's free to use, and it's just extremely good and very fast at fine-tuning these LLM models.

[00:03:58]All right. Now, what I'm going to do is I'm going to go over to this fine training notebook that I've already set up. For the purpose of this video, I'm not going to write all of the code from scratch. I'm just going to leave a link to this notebook in the description that you can download or that you can connect to in Google Collaboratory. And then you can use your own data and start fine-tuning your own models. But before we get into that, here's a quick idea that's worth stealing. If you're already fine-tuning LLMs or building tools with Olama, chances are you're solving real problems for real users. Maybe it's a niche SAS, an AI agent, or a quick side hustle for earlystage businesses. Now, imagine giving those users a fully functional website, a storefront, booking system, portfolio, whatever it is, without ever leaving your app. No templates, no drag and drop builder to maintain just one API call. Well, that's what Tenweb's AI website builder API does. It's not a template engine or a static generator. It's a real API that spins up fully edible WordPress-based websites, structure, content, design, images, e-commerce, etc., all in under 60 seconds. With one API call, you can generate a complete website. Your users never leave your dashboard. You get full white label control, so your branding, your domain, your customer relationship.

[00:05:13]And Tenweb handles the hosting, WordPress management, and technical infrastructure while you focus on your core product. This is how Tenweb powers websites with over 2 million users. And now you can do the same thing without building anything. Whether your user is launching a restaurant, coaching business, or online store, the site comes complete with hosting, security, mobile optimization, and full e-commerce capabilities. All powered by WordPress, the platform that runs 62% of the web.

[00:05:42]Now, if you're ready to turn your next big idea into reality, then visit my link in the description and book a strategy call with the 10 web team to see how they can help transform your business. Okay, so how does the fine-tuning work? Well, you can fine-tune this locally on your own computer by running all of the code that you see right here and that you can download from the link in the description. But unless you have a really powerful GPU, something like a 4080, 4090, something even more powerful than that, then this is going to take a very long time. So for most of you, I suggest that you use something called Google Collaboratory. This is a free online code editor environment provided by Google that allows you to connect and use super high-end GPUs. So you can do all of your training in Google Collaboratory. We can then take the model that we train here and we can download it to our own computer and run it in Olava, which is exactly what I'm going to show you how to do. So, first step here is open up this notebook.

[00:06:32]Again, I'll leave a link in the description if you're doing this on Google Collab, and connect to a runtime.

[00:06:37]So, you should see a button here that says connect. We're going to press that, and we should connect to a T4 instance runtime, which is a T4 GPU. Now, if you're not sure, you can press on this right here, and it's going to show you what you have available. So, system RAM, GPU, RAM, and disk. And it will show you the runtime you're connected to. If that didn't work, you can go here, and you can go to change runtime type. And then you can make sure that you select T4 GPU and that you're running in Python 3.

[00:07:02]Okay. So now that we're connected to this runtime, we can start running the various cells that I have inside of this notebook. Now, because we're going to train this on our own custom data, we need to bring in this file. So what I've done is I have this JSON file and I'm going to open it up here. But in order to open it in Google Collaboratory, I need to upload it to Google Collaboratory. So, I'm going to press on this file button here on the left hand side and I'm going to go to this little upload button where it says upload and I'm going to select this file. Again, you can download this from the link in the description. Okay. So, once this file is uploaded, you should see it here. And what I'm doing is I'm simply opening this file and loading it as a JSON file by providing the name JSON extraction data set 500. Now, you can use any type of data that you want.

[00:07:44]Again, just make sure you get it in this format where you have some input and then some output. Okay. And the output should actually be a string. I'm going to convert this to a string in 1 second because right now it's a JSON object.

[00:07:56]All right. So now that we have that, I'm going to run this. And you can see that I just printed out the first example.

[00:08:00]And I'm successfully loading in this file that I've uploaded. So I'll go ahead and close that window. Next, we need to install the various dependencies. In my case, I'm in Google Collab. So I'm going to run exclamation mark pip install. For now, I'll remove the uninstall command. I just had that in case I had something installed beforehand. Okay. This is going to take probably a minute or two to download and install. Once it's done, I'll be right back and we'll continue. All right, so that install has finished here and now we're going to move on to the next step here, which is related to the GPU check.

[00:08:27]However, as it says here, the following packages were previously imported in this runtime. You must restart the runtime in order to use the newly installed versions. So, I'm just going to press on the restart session button here just to get this reloaded. And once this restarting thing is done, okay, we're good to go. We can move on to the next cell. Now, if you are running this locally, the install might look a little bit different and you are going to need to have CUDA installed on your system and be using an Nvidia GPU in order for this to work or at least for it to work quickly. That's why I like using Google Collab because everything is already set up for us. So, I'm going to go ahead and press run here. I'm just going to run this cell and check if I have a GPU available and if CUDA is installed. I should get two TRS here or sorry, a true and then a GPU value. So CUDA is available and I'm using the Tesla T4 GPU in this Google Collab instance or runtime which is exactly what I want.

[00:09:16]Okay, so now we can move on and we can start actually setting up our fine-tuning training. Now here we need to pick the model that we want to fine-tune. Now I'm going to pick a really small model because I want to do this in a small amount of time. I don't want this to take days or weeks to train. And the model that I'm using here is the Fi3 mini model. Okay, you can look up this model if you want to see more information about it, but you can fine-tune any model that you want that's open source. So, for the model name here, you can put any Unsloth model. You can go to the Unsloth documentation to find all of the models that are available, but pretty much any open source model you can do here, like Llama 3.1, for example, Mistl, Mixel, etc. You can put them all here. Then, we're going to set the sequence length. Now, in this case, I'm just going with 48 tokens.

[00:09:57]This is the maximum number of tokens that the model can handle as input for the DT type or the data type. Don't worry about that. We'll leave it as none. And this just means that we're going to autodetect what the data type is. Now, what we need to do here is just load the model that we're trying to fine-tune. So that's why I put the model name here. And I'm getting the model and the tokenizer from the fast language model from Unsloth. And then I'm loading this pre-trained model, which is fi3.

[00:10:19]Giving the maximum sequence length load in 4bit is equal to true because this is a 4bit model. Okay. So I'm going to go ahead and press on run. That's going to load the model for us. It will need to download it if it's not already here. So that can take a second. And you might see some stuff like this where Unsloth is patching our computer. Don't worry, that is totally normal. And once that is finished, I'll be right back and we'll move on to the next step. All right, so the model is now downloaded here. Again, I picked a pretty small one just for this video. If you pick a larger one, this can take a much longer amount of time. And next, what we're going to do is we're just going to pre-process our data. So the day that I have, remember, comes in this format where we have some input and then we have some output. Now, what we need to do is we need to put this in just one single string that we can send to the model when we're fine-tuning it. So, what we're going to do is we're just going to use this format prompt function that I wrote here where we're going to have an input. That input is going to be equal to whatever the input is from our data set. So, you'll likely have to change this function to work with whatever type of data it is that you have. Then I have a new line character. Then I have the output. And what I've done here for the output is I've gone and I've grabbed this JSON object. I've converted it into text. So that's what JSON.dumps is doing. I'm taking a JSON object and converting this to a string. So I'm doing that for my output. Okay. And then I put this end of text tag here so that the model knows, okay, this is the end of the text. That's it. There's many different ways that you can format the prompt, but this is what I want the prompt to look like where we have this input and then this is the expected output. Okay. I then have my formatted data where I essentially just run through all of the items that I have in my file and call this format prompt function on them. and then I convert this into a data set. Now, if you try to run this, you might get an error. That's just because this first line up here, we may need to rerun where we're loading in the file because we reset the runtime.

[00:12:04]So, if we scroll down here now after rerunning that we can re-execute this cell and we should be good to go and we now generate this data set. That's just because we need this data set object when we're running this trainer. Okay.

[00:12:14]So, now what we're going to do is we're going to use something called the Laura adapters. Now, I'm not going to go into too much depth on exactly what this is doing, but this line right here is essentially going to add the kind of layers that we need to our LLM in order to actually do this fine-tuning. So, again, this can get very complicated.

[00:12:30]You don't need to understand all of the parameters that are here. And if you do want to mess with them, you are going to have to look these up or maybe use an LLM to explain them to you because there's a lot going on and there's not enough time for me to explain this in this video if I want to make it short.

[00:12:42]So, in fact, if we want more information on this, so we're in Collab. I'll just highlight this, right click on it, press explain code. This is going to use Gemini to explain the code to us. And then we can actually just see what the explanation is. And that's going to be more accurate than me walking through everything. So it says the selected code applies the Laura low rank adaption method to the language model using this.

[00:13:00]Here's a breakdown of the parameters.

[00:13:02]And then it tells you exactly what these are. Okay. So use Gemini within here if you want to know about all the parameters. Either way, we're going to go ahead and run this. So, we're going to create this new model now by kind of adding these adapters to it, which then will actually allow us to perform the fine-tuning, which we'll do now. So, you can see it patched 32 layers with 32 QKV layers. Don't worry too much about that if it means nothing to you. Okay, so now that we've loaded in the model, we're going to set up the trainer. Now, the trainer is actually the thing that's going to perform the fine-tuning for us.

[00:13:28]People much smarter than all of us have written all of this code. So, all we have to do is simply use it. So, again, you can read through all the parameters if you want, but what I'm using here is the SFT trainer. Importantly, I pass my model. I pass the tokenizer for the model. This is actually going to convert our string into the token so it can be understood by the model. I pass my data set which I created earlier. The uh field for my data set is text. That's important because if we go look at the data set, you can see that where is it here? We created the data set with this dictionary where it says text and then it has all of the values. Okay. And then we have a few other things in here that you can look at. For example, the maximum sequence length that needs to match with what we had before. And then all of these training arguments. Again, I'm just going to leave these kind of all default, but you can adjust them if you like. All right, so now we're going to run this. We're going to initialize the trainer. And now that that is created, what we're going to do is actually train the model. Now, this step is going to take a different amount of time based on how many examples you have and the different settings that you configure here. The more examples you have, the longer this will take, but the better performance you will get. The larger the base model is that you're using then again the longer this will take. In my case I'm using a very small model with a very small amount of examples. So my performance won't be great but this should train in maybe a few minutes not hours or weeks. All right. So we are training now and you can watch this window and you'll kind of see the progress. So once this is finished I'll be right back and then I'll show you how we can actually test this model, make sure that it's working and then download it and use it with Olama. All right. So the training step is finished here. Just as an FYI, that took about 10 minutes to run. So the next step is we're going to set up the model for inference just so we can test it and make sure it works before we download it. So in order to do that, pretty much all you have to do here is modify these messages to be the messages that you want to test. So you can see I have some RO, which is the user, and then I have content, which is the prompt I want to test. You can place multiple messages here, and you can see that this message is just similar to one of the examples where I'm doing this kind of HTML text extraction. You don't need to worry too much about these lines right here. We're just kind of putting the input and output into a format where we're able to actually send it to this model at least in the inference mode.

[00:15:37]And then we can go ahead and press on run. That will just take a second. It will run the inference. Then you can see that we get the output. So we have user, right, which said extract the information. And then if we go over here, you can see we have the assistant and then we get the output in the format that we specified. Okay. So we can see this is indeed working. And we could test this with multiple other messages to make sure it is working to our liking before we go ahead and download it.

[00:16:01]Okay, so now that we've assumed, all right, this is good, it's working, we've tested a few times in Google Collab, what we want to do is download this model so we can actually load it into a llama and then start using it more permanently on our own computer or our local machine. Now, this step does take a while, so bear with me here. What we're going to do is run this where we say model.save pre-train gguf. GGUF is the model format that O Lama understands. So that's why we're downloading it in this format. So I'm going to go ahead and press run. And it's now going to essentially create this output file for us. Download it to Google Collab. And then after that, we need to manually download this ourself so we can save it to our computer. Now this can take a long time. It can take 10 minutes, 15 minutes, 20 minutes. So just bear with it. Be patient. And once it's done, you can run this cell. This cell will then download it to your own computer, then we're going to move on to the next step where I show you how to actually load this into O Lama on your local machine. So once this is done executing and you've ran that final cell, you should have a file downloaded to your downloads folder that's named something like this. Unsloth.q4 km gguf. You're just looking for some file that ends in gguf. This is going to be a pretty large file and it will take a long time to download. In my case, both those steps took about 25 minutes to run. So, just keep that in mind. This will take a while, especially if you have a slow internet connection when you're downloading the file. Okay, now that we have the file, what we need to do is add this to O Lama. So, we need to create something called a model file.

[00:17:34]So, what I'm going to do is I'm going to open up my terminal. I'm just going to do this all from terminal commands to make our life a little bit easier. And I'm going to open this up. Okay, I'm going to zoom in a little bit and we're just going to test and make sure that Olama is installed on the system. So run the lama command. Make sure you've got that downloaded and installed for the next steps to work. Okay. From here, I'm going to go to my downloads folder. So cd into downloads. And from the downloads folder, I know that I have this unsloth kind of Q4_K_M file. So what I'm going to do is I'm just going to make a new directory. So I'm going to say mkdur lama- test. Okay.

[00:18:10]If we go to downloads now, we can see this folder is created. I'm just going to drag this file into the folder here.

[00:18:15]So now it's inside of O Lama test. Okay, I'm now going to go inside of O Lama Test and I'm just doing this to kind of organize things, but you don't need to follow this whole setup here. And what we're going to do now is we're going to make something called a model file. Now a model file defines a custom configuration for a model that you want to run in O Lama. So we need to make one of those. Now to do that, we can make a new file. So we can say something like touch and then we're going to call this model file. Okay, just exactly like this with the capital M. That's going to make a new file for us. If we ls, we can see it here. Now to edit this, we can do nano model file like that. And then I'm going to paste in what the model file should look like. Okay. So it's going to say from do slash and then this needs to be the name of your file or the name of your model. So the local file that we had is called this. So I'm just going to copy the name by going to rename copy and then pasting it here. So we're going to say from/unsloth.q4_k_m.gguf.

[00:19:11]Again, this is kind of the base model.

[00:19:13]So we're saying from this file right here which is in our local directory where this model file exists. Then we can specify some different parameters.

[00:19:21]So we can do the top p the temperature the parameter stop. So user end of text because that's the way that our model works. And then we have a template. So the template is going to be user prompt and then assistant. And then we just have a system message. So if we want to tell the model to do something specific, we can. In this case, we just say you are a helpful AI assistant. So this is a very simple model file. This is how we're going to be able to load in this particular model and use it in a llama.

[00:19:47]So now we are going to save. So I'm going to hit Ctrl X and then press Y to save the model file. Go ahead and press enter. And now you can see if we open up the model file again. So nano model file. It looks like this. Again we can get out of that. So now we have a model file and we have the model downloaded.

[00:20:05]What we need to do is add this to a llama. So in order to add this to a llama, we're going to type lama and then create and then we're going to give the model a name. So I'm going to say this is an HTML model. Okay. And we're going to do -f and then model file. What this is going to do is add a new model configuration to lama for us using the model file. So let's go ahead and press enter. That's going to take a second.

[00:20:27]And now if we want to see if this is here, we can type lama list. And we should see that we get the HTML model that was just created. So now if we want to run the model, we can just type lama run and then the name of this which is the HTML model. It's going to take a second to load up and then we can paste any message inside of here and we should see it working. So let's go into our data set. Let's just copy one of the ones that we have just to keep things simple. So let's copy this right here.

[00:20:54]Go back and paste it. Okay. And I just cleared the screen here so it's a bit easier to see. Okay. And then you can see after I put the prompt here and press enter, this is the result that we got. Now this isn't always going to work. So I can put this a few times and you'll see now look like it gives me a different output. That's because obviously we had a small data set. We're using you know a pretty poor model not one of the massive models but generally it is working in the way that we wanted it to work and more examples pass to this will give us a better output.

[00:21:19]Either way this is a now custom fine-tuned model loaded in a llama that we can run locally on our own computer.

[00:21:26]We could connect to it from Python or something and use it in our application.

[00:21:29]And that is going to wrap up this video.

[00:21:31]Now, I know this is not going to be perfect. You do need to experiment with this, modify some of the parameters, but I wanted to show you how to do fine-tuning, talk about what that actually is, and give you a step-by-step guide. So, hopefully this can get you up and running as fast as possible. All of the assets for this video will be linked in the description in case you need them. And with that said, I look forward to seeing you in another video.

[00:21:55][Music]

Up Next

5 Themes of Geography Explained: Location, Place, Region, Movement, and Human-Environment Interaction

@jasonjackson6872

16.7K views•2012-09-12

IFS Therapy Demonstration: Complete Session with Unburdening

@IFSCA

95.9K views•2021-01-13

Python Pandas Tutorial: Data Analysis Fundamentals in 30 Minutes

@TechWithTim

155.1K views•2025-08-06

Game of Thrones Opening Credits: A Cinematic Analysis

@gameofthrones

46.3M views•2011-04-18

Related Study Plans & Knowledge Roadmaps

Structured learning paths in General & Interdisciplinary Studies

Browse General & Interdisciplinary Study Plans

general-studies

Containerization: Evolution, Logistics & Trade

01Ancient Trade & Bulk Cargo: Amphorae to Barrels

02The Industrial Revolution & Cardboard Packaging

03The Containerization Revolution: Malcom McLean

+ 2 more learning modules

Explore Roadmap

general-studies

Textile Tech: Evolution, Labor & Global Impact

01Pre-Industrial Textiles: Handloom & Cottage Industry

02Mechanization: The Spinning Jenny to the Power Loom

03The Factory System & Global Labor Transformation

+ 2 more learning modules

Explore Roadmap

general-studies

Pharmacology: Botany, Patents & Biopiracy

01Foundations of Ethnobotany & Traditional Medicine

02The Birth of Synthetic Chemistry: Aspirin & Penicillin

03The Battle Against Malaria: Quinine to Artemisinin

+ 3 more learning modules

Explore Roadmap

general-studies

Writing Surfaces: Papyrus, Parchment & Paper

01Ancient Foundations: Clay, Stone, and Egyptian Papyrus

02Parchment, Vellum, and the Rise of the Codex

03The Chinese Rag Paper Revolution and the Silk Road

+ 3 more learning modules

Explore Roadmap