Academic Databases
Explore Our Semantic Databases
STEM & Tech
Math, Physics, CS & Engineering...
Economics
Macro, Micro, Econometrics...
Business
Management, Marketing, Finance...
Health & Life Sciences
Biology, Medicine, Neuroscience...
Humanities & Social Sciences
Psychology, Law, Philosophy, History...
General Studies
General & interdisciplinary studies...
Explore Topics
Features

How Fine-Tuning Works: LoRA & QLoRA Explained Simply

Added: 2026-08-06

Deep Dive

Basic understanding of neural network architecture, specifically how weights, biases, and backpropagation work during training.

“Backpropagation” – Fundamentos de Deep Learning – (Parte 7)

“Backpropagation” – Fundamentos de Deep Learning – (Parte 7)

9.6K

Backpropagation computes gradients for weights and biases using the chain rule. For weights: ∂L/∂w = (∂L/∂z) × (∂z/∂w), where z = w×x + b. For biases: ∂L/∂b = (∂L/∂z) × (∂z/∂b) = ∂L/∂z (since ∂z/∂b = 1). These gradients are then used to update parameters via gradient descent: w = w - α × (∂L/∂w) and b = b - α × (∂L/∂b). The process propagates gradients layer by layer from output to input.

Click to watch →

Neural network tutorial: The back-propagation algorithm (Part 1)

Neural network tutorial: The back-propagation algorithm (Part 1)

307.9K

Fully connected neural networks consist of input, hidden, and output layers, with each node receiving weighted inputs from all nodes in the preceding layer. Input layer nodes act as buffers holding raw data without transformation. The backpropagation algorithm uses systematic notation: X_j for inputs, W_ij for weights, θ_j for biases, o_j for outputs, and T_k for target values. The error function E = ½Σ(o_k - T_k)² quantifies prediction accuracy across output nodes. Training involves computing gradients of this error with respect to all weights, enabling gradient descent optimization. Weights in hidden layers differ from output layer weights in their relationship to target values, requiring distinct treatment during error propagation. This framework enables automated learning where networks adjust parameters to minimize prediction errors.

Click to watch →

Самое простое объяснение нейросети

Самое простое объяснение нейросети

197.9K

Neural network training involves optimizing weights and biases to achieve desired outputs. Initially, weights and biases are randomly initialized. For each training example, forward propagation calculates the output, which is then compared to the desired output to determine the error. Backpropagation propagates this error backward through the network, calculating each neuron's contribution to the total error. Using gradient descent, weights are corrected in the direction opposite to the error gradient, gradually approaching the minimum error. Training continues until the model achieves desired accuracy or a stopping criterion is met. An epoch represents one complete pass through all training examples. Mini-batch training divides the dataset into small groups (e.g., 125 examples) to improve computational efficiency while maintaining learning effectiveness.

Click to watch →

1: Introduction to Neural Networks and Deep Learning; Training Deep NNs

1: Introduction to Neural Networks and Deep Learning; Training Deep NNs

100.7K

In neural network terminology, the coefficients next to variables (previously called 'coefficients' in regression) are called 'weights,' and the intercept term is called a 'bias.' Together, these are sometimes collectively referred to as 'weights.' This terminology difference from traditional statistics is important for understanding neural network documentation and discussions.

Click to watch →

The Essential Main Ideas of Neural Networks

The Essential Main Ideas of Neural Networks

1.3M

The parameters that are multiplied during processing are called weights, and the parameters that are added are called biases. A neural network starts with identical activation functions, but the weights and biases on the connections slice, flip, stretch, and reshape these functions into new shapes. These transformed shapes are then added together to create entirely new curves, which are shifted to fit the data.

Click to watch →

Deep Dive

Familiarity with the Transformer model architecture, including self-attention and feed-forward layers.

Stanford CS231N | Spring 2025 | Lecture 8: Attention and Transformers

Stanford CS231N | Spring 2025 | Lecture 8: Attention and Transformers

26.9K

The transformer architecture combines self-attention with multi-layer perceptrons (FFNs/MLPs). The transformer block consists of: (1) Self-attention layer where all vectors interact globally; (2) Residual connection around the self-attention; (3) Layer normalization; (4) Feed-forward network applied independently to each vector; (5) Another residual connection and layer normalization. Multiple transformer blocks are stacked to form the complete transformer model.

Click to watch →

Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 14 – Transformers and Self-Attention

Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 14 – Transformers and Self-Attention

160.1K

The transformer architecture uses attention primarily for computing representations of input data. For translation tasks: (1) Encoder processes input using self-attention layers followed by feed-forward layers with residual connections; (2) Decoder uses causal self-attention (masked to prevent future peeking) followed by encoder-decoder attention and feed-forward layers; (3) Positional information is added using sinusoidal functions or learned representations since attention is permutation-invariant; (4) Residual connections carry position information through all layers, enabling the model to maintain order awareness throughout processing.

Click to watch →

Прикладное машинное обучение 4. Self-Attention. Transformer overview

Прикладное машинное обучение 4. Self-Attention. Transformer overview

33.9K

The complete Transformer architecture consists of: (1) Positional encoding for both input and output sequences; (2) Encoder layers with multi-head self-attention and feed-forward networks; (3) Decoder layers with self-attention, cross-attention, and feed-forward networks; (4) Layer normalization throughout; (5) Residual connections between layers. The decoder generates output tokens sequentially, one at a time, using self-attention on its own outputs and cross-attention on encoder outputs. A causal mask prevents the model from attending to future tokens during training. The architecture enables parallel training while capturing complex sequence relationships, becoming the foundation for modern NLP models including BERT, GPT, and many others.

Click to watch →

ЮФУ -- 2026.04.09 -- Самовнимание и архитектура трансформера

ЮФУ -- 2026.04.09 -- Самовнимание и архитектура трансформера

180

Each Transformer layer contains three main components: (1) Self-attention mechanism that allows tokens to attend to each other, (2) Layer normalization that normalizes the vectors to improve gradient flow, and (3) A feedforward network (MLP) that mixes features within each vector. Each component is surrounded by residual connections that add the input to the output, preventing vanishing gradients. The feedforward network applies the same transformation to all tokens independently.

Click to watch →

Lec 04 | Transformer: Architecture and Training

Lec 04 | Transformer: Architecture and Training

766

Self-attention is mathematically a linear operation: Attention = Softmax(QK^T) × WV × H. Since it doesn't add nonlinearity, a position-wise feed forward network is added after self-attention. This network consists of two linear transformations with ReLU nonlinearity in between. The input is projected to 2D, passed through ReLU, then projected back to D. Parameters are shared across all tokens, enabling the model to learn complex nonlinear relationships.

Click to watch →

Deep Dive

The concept of standard (full-parameter) fine-tuning of Large Language Models and its associated computational and VRAM limitations.

QLoRA: Fine-Tune LLMs on a Single GPU with Python Code

QLoRA: Fine-Tune LLMs on a Single GPU with Python Code

Fine-tuning large language models faces severe computational barriers due to memory requirements. A 10 billion parameter model using fp16 format requires 20GB just for parameters, plus another 20GB for gradients, and approximately 160GB total when including optimizer states (which use fp32 format and add ~12x multiplier). This makes standard fine-tuning impractical on consumer hardware, requiring specialized equipment costing around $50,000 or more. The core challenge is that large language models are computationally expensive to fine-tune in standard ways.

Click to watch →

Learn Fine Tuning LLMs in 2 hours | RAGs vs Fine Tuning | Quantization | PEFT Techniques

Learn Fine Tuning LLMs in 2 hours | RAGs vs Fine Tuning | Quantization | PEFT Techniques

12.3K

Full parameter fine-tuning updates all model weights during training, creating severe computational bottlenecks. A 1.1 billion parameter model requires updating 1.1 billion weights per epoch, scaling to 5.5 billion updates for five epochs. For 70 billion parameter models, this reaches 350 billion weight updates—requiring substantial GPU memory and time. Practical constraints often prevent full fine-tuning on standard hardware, motivating the development of parameter-efficient alternatives that maintain adaptation capability while dramatically reducing computational demands.

Click to watch →

Ajustando LLMs com Parameter efficient fine-tunning (PEFT)

Ajustando LLMs com Parameter efficient fine-tunning (PEFT)

743

Full fine-tuning of large language models requires adjusting all model parameters, which can involve hundreds of millions or billions of parameters. This process demands not just memory to store the model, but also additional resources for training components, often requiring 12-20 times the original model size. This creates significant challenges for individual computers or single GPUs, necessitating distributed computing resources.

Click to watch →

LLMs | Parameter Efficient Fine-Tuning (PEFT) | Lec 14.1

LLMs | Parameter Efficient Fine-Tuning (PEFT) | Lec 14.1

4.8K

Full fine-tuning of large language models presents insurmountable practical barriers: (1) Hardware constraints - training requires 12-20% more memory than trainable parameters due to optimizer states, gradients, and activations, exceeding what most organizations can provide; (2) Storage burdens - model checkpoints can reach hundreds of gigabytes (e.g., 350GB), making it impractical to maintain multiple specialized models; (3) Data requirements - large models (175B parameters) given only 1,000-2,000 examples will memorize training data, defeating transfer learning's purpose; (4) Catastrophic forgetting - learning new tasks causes the model to forget previously acquired world knowledge. These barriers motivated the development of parameter-efficient alternatives.

Click to watch →

LoRA & QLoRA Fine-tuning Explained In-Depth

LoRA & QLoRA Fine-tuning Explained In-Depth

140.5K

Large language model development progresses through pre-training, instruct tuning, safety tuning, and domain fine-tuning. Full parameter fine-tuning requires storing and updating billions of weights, demanding massive GPU resources. LoRA addresses this by tracking changes in two smaller matrices that multiply to form full-size updates, reducing trainable parameters dramatically. QLoRA extends this by applying quantization that compresses 16-bit parameters to 4-bit representations while preserving original precision. Together, these methods enable efficient fine-tuning on consumer-grade hardware, democratizing access to large language model customization.

Click to watch →

Deep Dive

Fundamental concepts of linear algebra, particularly matrix multiplication and the mathematical concept of matrix rank.

#34 MCQ OF MATRICES | OBJECTIVE QUESTION OF MATRICES | IIT JAM 2022 | GATE MATHS | CSIR NET

#34 MCQ OF MATRICES | OBJECTIVE QUESTION OF MATRICES | IIT JAM 2022 | GATE MATHS | CSIR NET

31.4K

Matrix rank is defined as the maximum order of non-zero minors. If a matrix has at least one non-zero minor of order r and all minors of order r+1 are zero, then rank equals r. For a matrix of order m×n, rank ≤ min(m,n). These fundamental concepts establish the basis for understanding matrix properties and solving linear algebra problems.

Click to watch →

1.2 Ранг матрицы

1.2 Ранг матрицы

7.6K

Matrix rank is the maximum number of linearly independent rows or columns. Row rank equals column rank (fundamental theorem). To compute rank: if all zeros, rank=0; if one non-zero row with others proportional, rank=1; otherwise, find maximum linearly independent system. Any linearly independent row system can be extended to a basis of the row space. The rank equals the dimension of the row space (linear hull of rows).

Click to watch →

Introducción a las matrices | Evau 2 bachillerato

Introducción a las matrices | Evau 2 bachillerato

28.8K

Matrix multiplication requires the number of columns in the first matrix to equal the number of rows in the second, with the result having dimensions from the first's rows and second's columns. Each element is computed by multiplying corresponding row and column elements and summing. Matrix multiplication is associative and distributive but not commutative. Matrix power (A^n) means multiplying A by itself n times, not raising elements to power n. Matrix rank is the number of linearly independent rows or columns. To find rank using Gaussian method, transform to row echelon form (first non-zero element of each row to the right of previous row's first non-zero element). Rank equals number of non-zero rows. Operations allowed: interchange rows/columns, multiply by scalar, add/subtract rows.

Click to watch →

Лекция №13. Линейная зависимость. Ранг.

Лекция №13. Линейная зависимость. Ранг.

3.8K

If a vector b belongs to the linear span of a system A, then adding b to A does not change the rank. For a matrix of size m×n, the column rank is the rank of its column vectors, and the row rank is the rank of its row vectors. A fundamental theorem states that column rank equals row rank for any matrix. The rank of the set of all column vectors of height n is exactly n, meaning no more than n linearly independent vectors can exist in n-dimensional space. If A' is a submatrix obtained from A by deleting rows and/or columns, then the row rank of A' does not exceed the row rank of A, and the column rank of A' does not exceed the column rank of A. For matrices A and B of the same size, rank(A+B) ≤ rank(A) + rank(B). Matrix multiplication is defined for an m×n matrix A and n×p matrix B as an m×p matrix where the (i,j) element is the sum over k of aᵢₖbₖⱼ. This operation satisfies key properties: associativity (AB)C = A(BC), distributivity over addition, compatibility with scalar multiplication, and the transpose property (AB)ᵀ = BᵀAᵀ. The double summation theorem (ΣᵢΣⱼ aᵢⱼ = ΣⱼΣᵢ aᵢⱼ) is essential for proving these properties.

Click to watch →

$Bsc 1st year Maths paper 1 Chapter 3 unit 1 Rank of Matrices \ L-2$

Bsc 1st year Maths paper 1 Chapter 3 unit 1 Rank of Matrices \ L-2

26.4K

The rank of a matrix is the maximum number of linearly independent rows or columns. A matrix has rank r if and only if there exists at least one non-zero minor of order r, and all minors of order r+1 are zero. If the determinant of a square matrix is non-zero, the rank equals the matrix order. If the determinant is zero, the rank is less than the order. The rank is denoted by 'r' and is a fundamental property in linear algebra.

Click to watch →

Deep Dive

Basic knowledge of computer arithmetic representation (FP32, FP16, and INT8/INT4 precision) used in deep learning.

ESWeek 2022 Workshop: CODAI

ESWeek 2022 Workshop: CODAI

442

The AI hardware industry has evolved from FP32 arithmetic to FP16 for training systems and INT8 for inference systems. FP16-based AI hardware is approximately 4-8x more efficient than FP32, while INT8 quantization is approximately 16x more efficient. The choice between formats depends on whether models were trained in floating point (allowing direct deployment) or require additional quantization. This evolution represents a critical step in the precision scaling roadmap, enabling more efficient AI deployment while maintaining model accuracy.

Click to watch →

Lecture 15 | Efficient Methods and Hardware for Deep Learning

Lecture 15 | Efficient Methods and Hardware for Deep Learning

188.9K

Floating point numbers are discrete representations, not continuous real numbers. FP32 uses 32 bits with three components: sign bit, exponent bit, and mantissa. The value is calculated as (-1)^s × 1.M × 2^exponent. FP16 uses 16 bits instead of 32, reducing both energy consumption and chip area by approximately four times. Int8 represents fixed-point numbers using integer bits followed by a radix point and fractional bits, commonly used in TPUs for efficient computation.

Click to watch →

LLM Fine-Tuning 12: LLM Quantization Explained( PART 1) | PTQ, QAT, GPTQ, AWQ, GGUF, GGML, llama.cpp

LLM Fine-Tuning 12: LLM Quantization Explained( PART 1) | PTQ, QAT, GPTQ, AWQ, GGUF, GGML, llama.cpp

7.1K

Different precision levels have different trade-offs: (1) FP32 (32-bit full precision) - highest accuracy, highest memory consumption, slowest speed, used for training, (2) FP16/BF16 (16-bit mixed precision) - medium accuracy, medium memory, faster than FP32, used for mixed precision training, (3) INT8 (8-bit integer) - lower accuracy, low memory, fast, used for inference, (4) INT4 (4-bit integer) - lowest accuracy, lowest memory, fastest, used for extreme compression. The choice of precision depends on the use case: training requires high precision for learning, while inference can use lower precision for speed and memory efficiency.

Click to watch →

Я построил суперкомпьютер с искусственным интеллектом из 5 компьютеров Mac Studio (RUS)

Я построил суперкомпьютер с искусственным интеллектом из 5 компьютеров Mac Studio (RUS)

35.8K

Quantization reduces model size by compressing numerical precision. FP32 maintains full accuracy, FP16 halves size with minimal loss (0-2%), and INT8 reduces size by 4x with ~13% accuracy loss. INT4 further reduces size to 1/8th with 10-30% accuracy loss. These techniques enable running larger models on consumer hardware by trading precision for accessibility, though complex tasks like reasoning suffer more from quantization.

Click to watch →

LLMs | Quantization, Pruning & Distillation | Lec 14.2

LLMs | Quantization, Pruning & Distillation | Lec 14.2

3.5K

Quantization reduces bit-width for parameters and activations while preserving network structure. FP32 uses 1 sign, 8 exponent, 23 fraction bits; FP16 reduces both, improving speed. The basic process identifies max absolute value (alpha), maps to ±127, computes scale factor, and converts values. Post-training quantization applies after training using calibration data, while quantization-aware training incorporates quantization during fine-tuning. Symmetric quantization centers at zero, while asymmetric formats allow different ranges. Each vector requires its own scale factor, and activation ranges must be calibrated separately.

Click to watch →

Basics Recap

Storage Challenge

PEFT Intro

Layer Addition

LoRA Concept

Rank Tradeoff

Quantization

QLoRA Reveal

Basics Recap

0:00

Playing Section

1
Recap neural networks, weights, biases, and parameters.
2
Introduces fine-tuning for domain specialization.

Deep Dive

Practical hands-on implementation of LoRA and QLoRA using libraries like Hugging Face PEFT, BitsAndBytes, and TRL.

LoRA explained (and a bit about precision and quantization)

LoRA explained (and a bit about precision and quantization)

118.5K

The Hugging Face library provides easy implementation of LoRA through the PEFT (Parameter Efficient Fine-Tuning) repository. Instead of manually applying rank decompositions to every layer, developers use get_peft_model function. Configurations allow specifying target modules (like QKV matrices in Transformers) and hyperparameters alpha and rank. An example shows only 0.19% of original model weights are trained, enabling training of huge models on single GPUs.

Click to watch →

A fistful of dollars: fine-tune LLaMA 2 7B with QLoRA

A fistful of dollars: fine-tune LLaMA 2 7B with QLoRA

3.6K

Hugging Face provides seamless QLoRA implementation through integrated libraries. The bitsandbytes quantization library is built into Transformers, requiring only one parameter to quantize models when loading from the Hub. The TRL (Transformer Reinforcement Learning) library offers sample scripts for supervised fine-tuning using QLoRA. Users can fine-tune Llama 2 7B on prompt datasets with 10,000+ examples using parameter-efficient fine-tuning plus 4-bit quantization, running on GPUs as small as T4.

Click to watch →

Tutorial 11

Tutorial 11

334

QLoRa implementation requires BitsAndBytes library (CUDA wrapper for quantization), Transformers, PEFT, and HuggingFace. The BitsAndBytes library provides 8-bit optimizer, matrix multiplication, and 4-bit/8-bit quantization capabilities. The Alpaca dataset (52K instruction-following examples from Stanford) is commonly used for fine-tuning. LoRA configuration involves: task type ('causal LM'), rank (typically 64), alpha (stability parameter), dropout, and target modules (Q_proj and V_proj in Llama). Only target modules receive LoRA weights while other layers retain full finiteny. The training process involves: loading model with BitsAndBytes configuration, tokenizing dataset, applying LoRA configuration, loading checkpoint shards, setting training arguments (output_dir, evaluation_strategy, learning_rate, batch_size, epochs, weight_decay, save_strategy), initializing Trainer, and training with TensorBoard monitoring for loss functions, metrics, and gradient norms.

Click to watch →

Fine Tuning LLM with own dataset |Euron Live class

Fine Tuning LLM with own dataset |Euron Live class

700

LoRA implementation: (1) Load base model, (2) Create LoRA configuration with rank and alpha parameters, (3) Set target modules (Q, K, V, O projections), (4) Set dropout ratio, (5) Get PEFT model, (6) Calculate trainable parameters. QLoRA combines LoRA with quantization (4-bit NF4) for further efficiency. Both approaches reduce trainable parameters by 99%, enabling training on lower hardware while maintaining model performance.

Click to watch →

🐐Llama 2 Fine-Tune with QLoRA [Free Colab 👇🏽]

🐐Llama 2 Fine-Tune with QLoRA [Free Colab 👇🏽]

52.3K

This comprehensive tutorial covers the end-to-end process of fine-tuning Llama 2 using QLoRA (Quantized Low-Rank Adaptation) on free Google Colab. The workflow includes: (1) Understanding QLoRA fundamentals combining quantization (reducing precision for memory efficiency) and LoRA (fine-tuning only model portions), (2) Setting up essential libraries (Accelerate, PEFT, Transformers, Datasets, BitsAndBytes), (3) Preparing training data in human-assistant format, (4) Loading sharded models for memory efficiency, (5) Configuring LoRA parameters and training settings, (6) Executing training with loss monitoring showing convergence, (7) Saving fine-tuned adapters as JSON and binary files, (8) Deploying to Hugging Face Model Hub via authentication. The demonstration shows fine-tuning Llama 2 on French quotes to enable French text generation instead of default English responses.

Click to watch →

Deep Dive

Exploring advanced Parameter-Efficient Fine-Tuning (PEFT) alternatives and variants, such as DoRA (Weight-Decomposed Low-Rank Adaptation) and AdaLoRA.

Lec 17 | Parameter-Efficient Fine-Tuning (PEFT)

Lec 17 | Parameter-Efficient Fine-Tuning (PEFT)

731

Parameter-Efficient Fine-Tuning (PEFT) enables adapting pre-trained large language models to specific tasks by updating only a small fraction of parameters rather than the entire model, significantly reducing computational cost, energy consumption, and resource requirements while maintaining performance comparable to full fine-tuning; PEFT methods are categorized into three main approaches: additive methods (like adapters and prefix/prompt tuning that add new parameter modules), selective methods (like BitFit and FishMask that identify and update specific existing parameters), and reparameterization techniques (like LoRA, AdaLoRA, and DoRA that mathematically decompose weight matrices to enable efficient adaptation).

Click to watch →

DoRA: Weight-Decomposed Low-Rank Adaptation

DoRA: Weight-Decomposed Low-Rank Adaptation

2.8K

DORA (Weight-Decomposed Low-Rank Adaptation) is an improvement over LoRA that enhances fine-tuning accuracy by decomposing weight matrices into magnitude and direction components, then training them independently—magnitude through standard fine-tuning and direction through LoRA—thereby achieving better performance across text, image, and video benchmarks while maintaining memory efficiency.

Click to watch →

LLMs | Parameter Efficient Fine-Tuning (PEFT) | Lec 14.1

LLMs | Parameter Efficient Fine-Tuning (PEFT) | Lec 14.1

4.8K

This extensive section presents LoRA (Low-Rank Adaptation) as the most influential parameter-efficient fine-tuning technique, grounded in intrinsic dimensionality theory. The theory identifies the minimum number of dimensions (d << D) needed to achieve 90% of full fine-tuning performance, with dimension requirements depending on both model capability and task complexity. Structure-aware intrinsic dimensionality further reduces memory and training time by processing each Transformer layer separately with scaling factors. LoRA implements this theory through reparameterization: weight updates are expressed as AB where A and B are trainable matrices, with only query, key, value, and output projection matrices modified. No non-linearity is used because the goal is adding projections to existing weights. Critical to success is proper initialization: B starts at zero (preserving original behavior) while A uses Gaussian with small variance. Research shows optimal rank varies by task and model capability - complex tasks with weaker bases require higher ranks, while simple tasks with strong bases can use rank 1. LoRA achieves state-of-the-art efficiency, often outperforming other PEFT methods with even fewer parameters, and has spawned numerous variants including K-LoRA, LoRA+, and DORA.

Click to watch →

Dự án Fine-tune LLMs | step by step

Dự án Fine-tune LLMs | step by step

14.4K

This section explains Parameter-Efficient Fine-Tuning (PEFT) and its variants. PEFT enables fine-tuning large models without updating all parameters. LoRA (Low-Rank Adaptation) creates small adapter matrices to modify model behavior, reducing memory from 32GB to manageable levels. QLoRA combines LoRA with 4-bit quantization, further reducing memory to ~5.6GB and enabling fine-tuning on consumer GPUs like T4. The section covers key hyperparameters: rank (r) controls adapter dimensions, alpha (α) scales adapter output, and target modules determine where adapters are applied. These techniques democratize access to large model fine-tuning.

Click to watch →

The Tiny Idea That Lets Anyone Fine-Tune AI

The Tiny Idea That Lets Anyone Fine-Tune AI

479

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that allows adapting large pre-trained AI models by freezing the base model and learning a low-rank update matrix (delta W) through two smaller matrices A and B, where delta W = B × A. This approach reduces trainable parameters from billions to millions (less than 1% of the original model), making fine-tuning computationally feasible. The matrices A and B can be interpreted as a key-value memory system where A acts as feature detectors and B specifies corrections. Key variants include LoRA+ (different learning rates for A and B), QLoRA (quantizing the frozen backbone for memory efficiency), VeRA (sharing frozen random basis matrices across layers), and DoRA (separating magnitude and direction updates for stronger adaptation).

Click to watch →

Deep Dive

Learning the process of merging adapter weights back into the base LLM model for optimized, zero-latency inference.

Fine-tune LLMs with Unsloth: LoRA, QLoRA & GRPO

Fine-tune LLMs with Unsloth: LoRA, QLoRA & GRPO

58.4K

After training LoRA adapters, they can be merged with the base model by summing the adapter weights with the original base model weights. This creates a unified model that can be saved as a single GGUF file. Merging is useful when deploying models that don't require swapping different adapters at inference time. The merged model retains all learned behaviors while being a self-contained file suitable for sharing and deployment.

Click to watch →

Vision-Based Fine-tuning Gemma 3 LLM with Unsloth on Google Colab

Vision-Based Fine-tuning Gemma 3 LLM with Unsloth on Google Colab

1.2K

Merging fine-tuned adapters into the base model combines adapter weights with base model weights using save_pretrained_merge(). This process is memory-intensive and may require runtime restart. The merged model can be saved in GGUF format using save_pretrained_gguf() for optimized inference on consumer hardware. GGUF files are compatible with Ollama and other inference engines, enabling deployment on local machines with limited resources.

Click to watch →

Low-rank Adaption of Large Language Models: Explaining the Key Concepts Behind LoRA

Low-rank Adaption of Large Language Models: Explaining the Key Concepts Behind LoRA

130.6K

After training with LoRA, the small decomposed matrices can be merged back into the pre-trained weights, resulting in zero additional inference latency. This merging process combines the original frozen weights with the learned low-rank updates, producing a complete weight matrix that can be used directly without any overhead during inference. This makes LoRA practical for deployment scenarios where inference speed is critical.

Click to watch →

Тихомиров М.М. - Обучение больших языковых моделей - 7. Методы Fine-tuning и LoRA

Тихомиров М.М. - Обучение больших языковых моделей - 7. Методы Fine-tuning и LoRA

132

This section covers LoRA weight merging and quantization techniques. After training, LoRA adapters can be merged back into the base model by adding them to original weights. The alpha parameter controls the merge ratio. Combining LoRA with quantization (e.g., 4-bit precision) enables training on consumer GPUs for models that would otherwise require server-grade hardware. The lecture demonstrates the trade-offs between memory efficiency and training quality.

Click to watch →

LoRA: Low-Rank Adaptation of LLMs Explained

LoRA: Low-Rank Adaptation of LLMs Explained

11.7K

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique for large language models that freezes the original model weights and adds small trainable adapter matrices (A and B) to linear layers, where the output is computed as Y = WX + ABX. This approach dramatically reduces trainable parameters from O(D²) to O(D×R) where R << D (typically R=1-4), making fine-tuning more memory-efficient and computationally faster while maintaining model performance comparable to full fine-tuning. The adapter matrices can be compressed back into a single weight matrix (W_new = W + AB) for inference, allowing the original model to be recovered by subtracting the adapter weights.

Click to watch →

Deep Dive

Applying PEFT adapters within preference alignment techniques like Direct Preference Optimization (DPO) or Reinforcement Learning from Human Feedback (RLHF).

LLM Fine-Tuning 16: Preference Alignment & Preference Training in LLMs with RLHF, RLAIF, DPO, LoRA

LLM Fine-Tuning 16: Preference Alignment & Preference Training in LLMs with RLHF, RLAIF, DPO, LoRA

1.7K

Preference alignment is a technique used to train large language models to better align with human expectations by using preference data (chosen vs. rejected responses) and applying methods like Direct Preference Optimization (DPO), which uses a supervised learning loss function to optimize the model's preference for chosen responses over rejected ones, thereby improving safety, helpfulness, and ethical behavior in model outputs.

Click to watch →

Direct Preference Optimization: Forget RLHF (PPO)

Direct Preference Optimization: Forget RLHF (PPO)

16.1K

Direct Preference Optimization (DPO) is a new training methodology developed by Stanford University that bypasses the complex two-step process of Reinforcement Learning from Human Feedback (RLHF) by directly optimizing language models using a single maximum likelihood objective based on human preference data, achieving similar or better performance than traditional RLHF methods while being substantially simpler to implement and train.

Click to watch →

George Hotz | Reading & Talking | let's read ML papers | from $1499 buy comma.ai/shop/comma-three

George Hotz | Reading & Talking | let's read ML papers | from $1499 buy comma.ai/shop/comma-three

80.9K

RLHF (Reinforcement Learning from Human Feedback) is a technique for aligning language models with human preferences. It involves training a reward model that takes text sequences and returns a scalar reward representing human preference. Companies like Midjourney use this approach where users select preferred outputs, and this preference data is used to optimize models. This allows gathering human preference data without encoding it in large datasets.

Click to watch →

LLMs Fine-Tuning تطبيق عملي على ال

LLMs Fine-Tuning تطبيق عملي على ال

26.2K

LLM fine-tuning is a technique that involves training a pre-trained language model on specific task data to improve its performance for particular applications, using methods like parameter-efficient fine-tuning (PEFT) and adapters to reduce computational costs while maintaining model effectiveness.

Click to watch →

Aligning LLMs with Direct Preference Optimization

Aligning LLMs with Direct Preference Optimization

33.9K

Direct Preference Optimization (DPO) is a simplified alignment technique that trains language models directly on human-preference data without requiring a separate reward model, using a differentiable loss function that maximizes the difference between log probabilities of preferred versus dispreferred responses; this approach achieves comparable or better results than traditional reinforcement learning from human feedback (RLHF) while being more computationally efficient and easier to implement, as demonstrated by its successful application in training Zephyr, a state-of-the-art 7B parameter chat model.

Click to watch →

Deep Dive

Strategies for serving and deploying adapter-based models efficiently in production environments using frameworks like vLLM or Ollama.

Secure your AI agents for production workloads

Secure your AI agents for production workloads

5K

vLLM is a high-performance framework using paged attention to manage memory efficiently, allowing maximum GPU utilization and higher throughput. Unlike Ollama, the model weights are stored in a cloud storage bucket and downloaded on startup. Cloud Storage FUSE mounts the bucket as a virtual drive inside the container, decoupling serving code from model weights. This enables fast model updates without disrupting the automated pipeline. The container remains lightweight and fast to build. For production environments with continuous traffic, hosting on a dedicated cluster keeps pods warm, eliminating cold start penalties while still allowing weight swapping on the fly.

Click to watch →

LLMs Fine-Tuning تطبيق عملي على ال

LLMs Fine-Tuning تطبيق عملي على ال

26.2K

vLLM is a framework for efficient LLM inference that implements continuous batching for higher throughput, optimizes memory usage through efficient tensor management, and supports multiple model architectures. vLLM enables serving LLMs at scale with lower latency and higher throughput compared to traditional inference frameworks. Configuring vLLM involves setting model paths, configuring GPU resources, and tuning parameters for optimal performance. Multi-adapter system design involves creating adapters for different tasks, implementing adapter switching logic, and managing adapter storage and versioning. Multi-adapter systems enable a single base model to serve multiple tasks efficiently, reducing resource requirements compared to separate models for each task.

Click to watch →

Optimising Open Source LLM Deployment on Cloud Run

Optimising Open Source LLM Deployment on Cloud Run

1K

Ollama deployment with cached model achieves fastest startup (4-6 seconds) but only 5 concurrent requests per container. vLLM with GCS Fuse mounting enables flexible model swapping but suffers from 3.5-minute startup time and 512-token limit. Despite excellent concurrency (30-60 requests), vLLM's limitations in token size and startup time make it less ideal for production. Both require 8 CPUs, 32GB RAM, and NVL4 GPU (24GB).

Click to watch →

How to Choose LLM Infrastructure when Self Hosting (ollama, vLLM, paperspace, deepseek, gemma)

How to Choose LLM Infrastructure when Self Hosting (ollama, vLLM, paperspace, deepseek, gemma)

1.5K

vLLM deployment requires Hugging Face account credentials and supports different optimization formats including AWQ for production workloads. Unlike Ollama, vLLM takes longer to initialize but provides superior runtime performance. The Docker image is named for OpenAI API compatibility. With a 5,000 token context window, even 70B AWQ models consume nearly full VRAM capacity. vLLM requires more complex configuration than Ollama but offers better performance for production deployments. Understanding these trade-offs helps match deployment frameworks to specific use cases.

Click to watch →

LLM 서빙 프레임워크로 프라이빗 AI구축하기 feat. Ollama, vLLM, SGLang [세미남589@토크아이티, 윤성열 대표 / 드림플로우]

LLM 서빙 프레임워크로 프라이빗 AI구축하기 feat. Ollama, vLLM, SGLang [세미남589@토크아이티, 윤성열 대표 / 드림플로우]

7.8K

LLM Serving is the process of deploying Large Language Models for service use after model selection and quantization. Ollama serves as an accessible open-source framework for CPU and Apple Silicon environments, ideal for individuals and small teams. For enterprise batch and page processing needs, vLLM and SGLang offer superior performance. Quantization reduces model bit depth for budget-friendly deployment: GGUF provides universal CPU/GPU support for CPU-only environments, while AWQ optimizes GPU performance for production serving scenarios. The choice of framework and quantization method directly impacts deployment flexibility, performance, and resource utilization.

Click to watch →

Basics Recap

Storage Challenge

PEFT Intro

Layer Addition

LoRA Concept

Rank Tradeoff

Quantization

QLoRA Reveal

Basics Recap

0:00

Playing Section

1
Recap neural networks, weights, biases, and parameters.
2
Introduces fine-tuning for domain specialization.

10.5K views35717:13@KeertiPurswaniOriginal Release: 2025-11-30

Parameter-efficient fine-tuning (PEFT) techniques like LoRA (Low Rank Adaptation) and QLoRA (Quantized LoRA) enable efficient retraining of large neural networks by freezing pre-trained model parameters and training only low-rank matrices (e.g., 6x1 and 1x6 instead of 6x6) to capture domain-specific changes, reducing computational cost from millions to thousands of trainable parameters while maintaining model performance.

[00:00:00]Hello Baba.

[00:00:00]>> Hello Bach.

[00:00:02]>> New topic.

[00:00:02]>> Always welcome.

[00:00:03]>> Ready. So we are going to understand a lot of basics today. Going to understand things in details. I [music] will tell you don't worry.

[00:00:11]>> Okay.

[00:00:11]>> So you remember I taught you a topic which is like the basics of everything like how chat chippity works.

[00:00:17]>> Which is >> which is [music] what >> like what topic did I teach you?

[00:00:21]>> You taught me so many topics neural networks.

[00:00:24]>> So that is what I was talking about neural networks. Okay. So I taught you basics of neural networks, right? Right.

[00:00:31]So you remember that there were nodes in layers like layer 1, layer two, layer three.

[00:00:36]>> Yeah, I remember.

[00:00:36]>> And then it had like all the nodes were connected to each other >> with some different weightages and all.

[00:00:42]>> Correct. So how important is a particular input? We understood from the weight >> and then there were biases, right? So these weights and biases are called parameters of a model.

[00:00:52]>> Okay.

[00:00:53]>> Okay. So you must have heard that a model is like 12 billion parameter model or 8 billion parameter model.

[00:01:00]>> So what happens inside is that in the end there are 12 billion weights and biases that we are trying to learn.

[00:01:05]>> My god 12 billion.

[00:01:06]>> Yes. Okay. So this is the basics.

[00:01:09]>> Okay.

[00:01:10]>> Now we also talked about one more topic which is called finetuning.

[00:01:14]>> Yeah I remember.

[00:01:15]>> Yeah. So why do we need fine tuning?

[00:01:17]>> Basically to retrain the model with the latest uh inputs available.

[00:01:21]>> Correct. Right. So like for example if you want a model to be specialized in a particular domain like medical or like law because our usual models are they are generalized models. So like software developers are also using them lawyers are also using them medical folks are also using them but now we want our models to become specialized in a particular >> particular field. Okay. Particular topic. Okay.

[00:01:44]>> So then we do retraining of the model or fine-tuning of the model. Yes.

[00:01:48]>> Right. So today we will talk about if the model is so huge, how does fine-tuning happen internally?

[00:01:55]>> Okay, interesting.

[00:01:57]>> So how does fine-tuning happen internally is the topic of this video.

[00:02:00]>> Okay, [music] >> sounds good.

[00:02:02]>> Very good.

[00:02:02]>> Let's get started.

[00:02:03]>> Definitely.

[00:02:04]>> Before we move it, I would just like to take a minute over here to tell you how you can learn live with [music] me at ED courses. At courses, we have various courses for interview prep. I have high level system design, low-level system design, DSA. for project based learning.

[00:02:17]If you want to create really interesting scalable projects, not simple clone ones, check out HLD. It is MER plus DevOps plus HLD. We have hands-on generative AI course where I have not just covered the application side but also the internals of LLM in a lot of detail with white paper discussions, transformer architecture and so much more. 10 hands-on projects have been covered over there. Please check out the curriculum, the testimonials, the LinkedIn profiles of our all our students is tagged. Plus, if you enroll for [clears throat] any of our courses, you get lifetime access to all the patches, the past patches, the future patches. So, you'll get access to the zoom recordings of the live class. So, you actually get a feeling that it is live itself. This is a feedback that I have gotten from all our students that it really feels like it is live only even if it is not live. But if you want to attend live only, you will get access to all our future patches as well. So, [music] at least check it out. I would love to be part of your further learning journey. And if you still have any questions, feel free to reach out to us at supporttheredcourses.com or just drop a WhatsApp or call us at this number. [music] And now let's continue. For starters, let's get started with the 10 billion parameter model.

[00:03:19]>> 10 billion. Okay.

[00:03:20]>> Okay. So now there are trillion parameter models also but we are taking one standard 10 billion parameter model.

[00:03:26]Okay.

[00:03:26]>> Okay. So now in the end these parameters are what? Weights and biases are what?

[00:03:31]>> Numbers basically >> it's all numbers right?

[00:03:33]>> Now it is basically decimal number.

[00:03:36]>> Okay. Okay. And in computer everything is represented in bits by >> binaries.

[00:03:41]>> Correct.

[00:03:42]>> Right. So every number can be like a 32 bit number which is like four bytes.

[00:03:48]>> Yeah.

[00:03:49]>> Or it can be a 16 bit number.

[00:03:52]>> Let's take 326 which is like two bytes.

[00:03:55]>> Okay.

[00:03:56]>> Okay. Now when we are talking about 10 billion parameters.

[00:04:00]>> Yeah.

[00:04:00]>> Right. So that will be how much space?

[00:04:03]Let's just talk about storing 10 billion parameters. So every parameter let's take the smallest 16 bit for now.

[00:04:10]>> So 10 billion into 16 bits or you can say [music] two bytes >> 116.

[00:04:16]>> So that is 20 GB actually.

[00:04:18]>> Yeah.

[00:04:19]>> So 20 GB just to store the 10 billion parameters. Then another 20 GB because you remember for weights and biases what is the goal of the neural network that we have to do the training and we have to optimize and find the correct values.

[00:04:32]>> Correct values. Right? Now to find the correct values we were doing differentiation and we were finding gradients. So weight and bias with respect to you remember we have to minimize the cost and all of that right.

[00:04:44]So uh to store the gradients for these again we need another 20 GB and then for optimization because there are algorithms that are running for optimization. So it has like if you see like a normal optimization algorithm like a dam is a very famous algorithm that takes 100 GB for a 10 billion parameter. So if I take 20 GB to store the just the parameters >> 20 GB to store the gradients and the 100 GB for optimization related things that is 140 GB >> just for a 10 billion parameter training.

[00:05:16]>> Oh god.

[00:05:17]>> Okay. So do you think it is very easy to train a 10 billion parameter model? Uh I don't think >> yeah [laughter] >> definitely >> right so retraining a model is not very easy >> easy yeah >> correct >> certainly >> but we also want ways to be able to retrain and make our model very good >> understood >> correct so that is the problem right now >> and make it easier also >> yeah yeah [laughter] right so this 10 billion parameters if I decide that I want to train all the parameters again that is called full parameter finetuning >> full parameter fining >> that I'm going to train all the parameters again >> okay we discussed finetuning now it full parameter tuning all the parameters will be retrained >> correct I'm guessing now you can imagine that we don't want to retrain all the parameters >> certainly because the time computation and all of that right it I'm talking in terms of storage but also the computation the time everything right >> certainly >> so there comes parameter efficient finetuning >> okay or parameter you can call it optimization also >> yes this is called pet parameter efficient fining Okay, >> parameter efficient finetuning.

[00:06:24]>> So basically what I maybe it's a guess work the important parameters you'll pick up and fine-tune them.

[00:06:30]>> So we'll talk about that how to achieve PFT.

[00:06:33]>> Okay.

[00:06:33]>> Okay.

[00:06:34]>> Am I correct?

[00:06:34]>> Yes.

[00:06:35]>> Okay.

[00:06:35]>> To an extent. Yes.

[00:06:37]>> Okay.

[00:06:37]>> So I just drew this diagram. You remember this is neural network.

[00:06:40]>> So I have now connected only two layers like this. All the nodes are connected to connect different. Yes. So every connection is going to have a weight and then there are biases. We know all >> definitely.

[00:06:51]>> So now there were different methods that were explored for parameter efficient finetuning. Okay. One way is that what if after the entire thing we put one extra layer that we are going to use to train using the new data.

[00:07:04]>> Okay.

[00:07:05]>> Okay. But now see in every model now there are two phases. One is the training phase. Now after we finish the training there is the inference phase or when we are going to use the model.

[00:07:14]>> Okay.

[00:07:15]>> Correct?

[00:07:15]>> Okay.

[00:07:16]>> So what happens during the training we learn the weights and the biases. But when we use the model what happens is that we actually do the calculation. You remember y equal to m express. So once you have learned all the things all the parameters then you are just going to compute the output values for a input value. Right?

[00:07:32]>> Definitely.

[00:07:32]>> So that is called inference.

[00:07:34]>> Inference basically dering deriving out of the >> the input. Right. So after training happens inference when you give an input and you'll get the output. Right? Now the thing is that if we add one extra layer okay we will be able to train and we are not going to change the rest of the parameters. So we are going to do training and learn only these extra parameters.

[00:07:53]>> Extra parameters. Okay.

[00:07:54]>> But now what happens is that inference time also increases because now there are so many extra parameters. Right. So that was not considered a very good solution.

[00:08:03]>> So what better solution can be there.

[00:08:05]Right. So this is one solution. So like this there are many solutions.

[00:08:08]>> One solution is adding another layer.

[00:08:10]another layer that we are not going to learn all the parameters. These will be added layer we are going to so this added layer will learn the extra things that we need to learn for this particular domain >> like that right so it will have its own weights and biases but what happens is because of this the inference also becomes slow >> slow >> right so this is one way that you can add one layer at any of these points you can add it in the end you can add somewhere so this is one way of achieving parameter efficient finetuning >> okay >> but now there is some maths involved which is very very interesting and I want to tell you that so that you understand what other ways are there the better way okay so at any particular layer there are going to be lots of weights >> you remember when I had taught you a bit about self attention I had told you that everything is represented in terms of matrix [music] >> what is the impact of every word on every other word and all that yes >> so here also when we have so many weights no they are represented in terms of matrices >> okay >> okay so it is like one layer node how is it connected to the next layer nodes so suppose m node nodes here, M nodes over here. The weight matrix will become M into M >> M into suddenly >> how how is every node connected to every other node. So every node is going to have a weight [music] >> correct.

[00:09:24]>> Now this weight matrix it can be of 10 tens of thousands right? It can be like 1,024x 10,24 10,000 by 10,000.

[00:09:33]>> For simplicity to just give an example, I'm taking a 6x6 weight matrix. Very simple example. Simple simple. Okay. So if we take a 6x6 matrix, how many elements are going to be there inside that?

[00:09:44]>> 36.

[00:09:45]>> 36, right? So 1 2 3.

[00:09:51]>> Okay. Now there are 36 elements. Okay.

[00:09:54]Now if there are 36 weights what we have to do is we have to see that we have to change these weights somehow right we have to when we do training what are we doing we are finding the new values of weights and biases >> yes >> correct so one way would have been that I would have updated all the 36 weights right >> but the better way will be which has more significance >> that is true >> but think about it so if I have another >> matrix >> matrix which I can call as a delta w matrix Okay, >> this is like how much do I have to change the weights for the new domain right for the specialization >> basically change in the weights >> correct change in the weights so then now the goal becomes that I don't have to touch the already weights and biases that is a pre-trained model so now weights and biases are fixed >> but the goal becomes to find this delta w >> delta w okay >> right so delta w will also be 6x6 >> 6x6 >> right now the goal is to find these 36 elements >> okay >> right Now okay now think about it. So now instead of this 6x6 matrix >> okay >> what if I take two matrices of 6x1 and 1x 6 size.

[00:11:02]>> Okay.

[00:11:03]>> So when I multiply these two matrices what happens >> again 36 >> 36 elements right? But if we have these six into one so how does it look like?

[00:11:12]This is 1x 6 and this is 6x 1.

[00:11:15]>> Yeah.

[00:11:15]>> So how many elements are there now? Six elements over here. Six 12 elements over here.

[00:11:20]>> But when we multiply these two how many elements do we get?

[00:11:23]>> 36. 36. Yes.

[00:11:25]>> So what this is the concept of matrix decomposition >> that instead of training to find 36 elements >> we are finding only 12 elements.

[00:11:33]>> 12 elements.

[00:11:34]>> Okay. Making the task easier.

[00:11:35]>> Correct.

[00:11:36]>> Simpler.

[00:11:36]>> So now see right now we have taken the example of 6x6. But it can be 1,000 by,000.

[00:11:41]>> Definitely.

[00:11:41]>> Right. So if it was 1,000 by,000 we would have to learn 10^ 6 elements. But now we would have to learn only 2,000 elements.

[00:11:49]>> 2,000. Understanding.

[00:11:50]>> Understood. Understood. This is called low rank matrices because this is like rank is one >> one. Okay.

[00:11:56]>> Okay. So this is called lora.

[00:11:59]>> That is low rank matrices.

[00:12:01]>> Low rank adaptation.

[00:12:02]>> This is called low lower rank.

[00:12:04]>> This is called lora. This is parameter efficient finetuning that instead of finding 36 new elements [music] now we'll be finding 12 new elements.

[00:12:14]>> Okay. Okay.

[00:12:15]>> Yeah.

[00:12:15]>> Okay.

[00:12:16]>> Yes. So this is called low rank adaptation.

[00:12:18]>> Adaptation.

[00:12:19]>> Just one more thing over here. Just think about it.

[00:12:21]>> Sure.

[00:12:21]>> Now see if I am taking 12 elements instead of 36 elements obviously efficiency somewhere will go for compromised.

[00:12:29]>> Yeah. So it is a tradeoff right. But if we consider computation right because we are saving so much of computation like think about 10^ 6 and 2,000.

[00:12:39]>> This trade-off is considered okay.

[00:12:41]>> Okay. Okay.

[00:12:42]>> But if you still want to adjust it what you could do is you could take a 6x2 and a 2x6 matrix.

[00:12:49]>> Yeah. you are adding to it.

[00:12:51]>> Yes. So then it would look like two rows and this and then two and this >> basically 24 elements.

[00:12:56]>> Yes. 24 elements. So if I take 6x2 and 2x6 [music] the comput efficiency will become more but computation will also become more.

[00:13:06]>> Yeah. It will take some time but efficiency will improve.

[00:13:08]>> Yes. So that you can adjust using that how many rows and columns you want. This number is called rank.

[00:13:15]>> Okay.

[00:13:16]>> That is why low rank adaptation. [music] So if you take only one row, if you take six by one, one by six, that is rank one.

[00:13:22]>> Low rank adaptation. This is also low rank.

[00:13:24]>> This is also with rank two.

[00:13:25]>> This is with rank two. Like this you can take with rank three. You you might now understand the rank will obviously be lesser than the size of the >> definitely. Definitely. Right. So [clears throat] it can be rank one, rank two, rank three, four, five, six. But as you increase your rank, the computation that you require will also be more but efficiency will also increase. Yeah.

[00:13:44][snorts] >> So this is called lower rank adaptation.

[00:13:46]Okay, great.

[00:13:47]>> So what is happening in parameter efficient finetuning that [music] in these weights and biases are frozen >> frozen >> for few layers for few important layers what we are doing is we are going to retrain for these 12 parameters or for the low rank >> 12 or 24 whatever we decide >> so low rank matrices >> low rank matrices [music] >> interesting >> definitely >> so now the last topic is that how can we make this even more efficient >> even after low rank you want to make it further efficient okay without increasing the computation.

[00:14:18]>> Yeah. [music] Yeah. Interesting.

[00:14:20]>> So you remember we started with that every parameter, every weight of bias in the end is a decimal number of 16 bytes or 32 bytes. 32, right?

[00:14:29]>> Now what if I can have the same understanding or the same precision by a four bit instead of a 16 bit or instead of a 32bit.

[00:14:40]>> Okay. So if that was possible then [music] we would just have the uh saving of by four right instead of 32 starting we are using four bit so the saving is directly by 8 times 32 and 4.

[00:14:53]>> Oh yeah >> right 16x4 >> 1/8.

[00:14:56]>> Yes. Okay.

[00:14:57]>> Right. So you are going to save on lot of computation.

[00:15:00]>> Lot of computation.

[00:15:00]>> So this is called quantization.

[00:15:02]>> Quantization.

[00:15:03]>> That every parameter you are going to quantize it to four bits.

[00:15:08]>> Okay. Right. from 16 bits or 32 bits you are quantizing it to four bits.

[00:15:13]>> This is called Q Lora. Quantized low rank adaptation.

[00:15:17]>> Okay.

[00:15:18]>> Quantization is a generic concept but when you apply it with Lora over here it's called Qura.

[00:15:23]>> Okay.

[00:15:23]>> Okay. Now one more interesting point over here like normally people would think that when you represent the same thing in four bits obviously the precision is going to go for a toss like people do think like that right >> but there are white papers that have proven that no that is not true >> so what happens in quantization I'll tell you one very interesting thing >> sure >> have you heard have you seen this curve somewhere >> certainly >> this is a quashian curve or a bell curve like it is very common >> it's commonly used in normal theorems probability >> very good in probability and all right so what usually it says is central limit theorem if you have heard definitely right so what it says that usually the parameters like land up in this particular curve >> 99.7% will come >> correct correct correct right so the same weights and biases actually same thing applies over here and every parameter can be represented using four numbers on this curve >> okay >> so that is what four bits is used for Oh great.

[00:16:23]>> So now instead of representing everything in 32 bits or 16 bits you represent using only four bits that are there on this curve >> on this curve.

[00:16:32]>> Yes.

[00:16:32]>> Okay. Great.

[00:16:33]>> So that is how quantization is.

[00:16:34]>> Very nice. Very interesting.

[00:16:36]>> So this is Qura. So this is Lora Qura.

[00:16:38]This is how fine tuning happens in here.

[00:16:40]>> Low rank adaptation and quantization.

[00:16:42]>> Quantized low rank adaptation.

[00:16:44]Interesting.

[00:16:45]>> Very interesting. So there are libraries and there are companies that are specially like you know working on this that how can we apply algorithms for Laura how can we make >> very good very interesting thank you >> did you enjoy >> certainly >> what topic should you learn next >> that you can only tell me should tell >> ah of course your audience can tell better but if you have found this interesting do like share and subscribe >> thank you >> thank you >> thumbs up for papa okay bye >> thank

Up Next

Hugging Face Transformers Library: A Beginner's Tutorial

@AssemblyAI

616.7K views•2022-04-03

Building Real-Time ML Pipelines with Feature Stores and MLOps Frameworks

@ODSCAI

5.1K views•2022-02-20

Bypassing Tor Censorship: Bridges and Pluggable Transport Guide

@Coding_ForEveryone

397 views•2024-06-11

Neural Networks Explained: Math, Layers, and Learning Fundamentals

@3blue1brown

21.9M views•2017-10-05

Related Study Plans & Knowledge Roadmaps

Structured learning paths in Artificial Intelligence

View All Artificial Intelligence Topics

artificial-intelligence

Building Recommenders: Two-Tower & Vector DBs

01Foundations of Python & Machine Learning Basics

02Introduction to Recommender Systems

03Deep Learning & Representation Learning

+ 3 more learning modules

Explore Roadmap

artificial-intelligence

Latent Diffusion: PyTorch, UNet & CLIP

01Deep Learning & PyTorch Foundations

02Convolutional Networks & Autoencoders

03The Math and Mechanics of Diffusion Models

+ 3 more learning modules

Explore Roadmap

artificial-intelligence

NeRFs: 3D Reconstruction & Instant-NGP

01Math & Python Foundations for 3D Vision

02Photogrammetry & Camera Pose Estimation

03Neural Radiance Fields (NeRF) Core Concepts

+ 2 more learning modules

Explore Roadmap

artificial-intelligence

TFT Forecasting: Energy Demand & Deep Learning

01Programming & Data Science Foundations

02Introduction to Machine Learning & Time Series

03Deep Learning & Sequential Models

+ 3 more learning modules

Explore Roadmap