RAG Systems: Embeddings, LLMs & Production

Learning Goal: Build and optimize production-ready Retrieval-Augmented Generation (RAG) systems to ground large language models in custom proprietary data.

Prerequisites

  • Python Programming: Comfort with Python (variables, functions, data structures, and asynchronous requests). Note: General Python syntax instruction is not covered in this advanced path.
  • Command Line & Environment Management: Familiarity with pip/conda, dotenv, and Docker.
  • Basic Machine Learning Concepts: Understanding vectors, matrices, and cosine similarity.

Estimated Study Time

  • Total Estimated Study Time: 38 hours (includes lectures, hands-on programming labs, debugging, and system evaluation).

Module 1: LLM Foundations, APIs, and Tokenization

This module covers the core mechanics of Large Language Models (LLMs) from a practical developer perspective. You will explore how weights, parameters, context windows, and tokenization models operate under the hood, and how to interface with commercial and open-source models via API endpoints.

Gap Alert (Self-Study Recommendation): Due to the specialized nature of the video pool, tokenization mechanics (like Byte-Pair Encoding) and deep transformer attention animations are not explicitly detailed in these videos. We highly recommend reading the Hugging Face Tokenizers guide or watching Andrej Karpathy's "Let's build the GPT Tokenizer" as supplementary material.

Recommended Videos

A Hackers' Guide to Language Models
A Hackers' Guide to Language Models
@howardjeremyp02:26567901
A Hackers' Guide to Language Models

RAG combines information retrieval with language generation to improve answer quality. The process involves: (1) retrieving relevant documents from a database or corpus that may contain useful information; (2) providing these retrieved documents as context to the language model; (3) having the model generate answers based on the combined information. This approach leverages external knowledge sources to supplement the model's existing training data.

@howardjeremyp · 02:26

Why this video is valuable: Jeremy Howard provides an excellent, pragmatic baseline for language models. He strips away the hype to focus on the reality of model weights, predictive outputs, and how context injection directly affects response accuracy.

Beyond the Hype: A Realistic Look at Large Language Models • Jodie Burchell • GOTO 2024
Beyond the Hype: A Realistic Look at Large Language Models • Jodie Burchell • GOTO 2024
@GOTO-03:06149404
Beyond the Hype: A Realistic Look at Large Language Models • Jodie Burchell • GOTO 2024

LangChain is an open-source package for extending LLM functionality beyond simple input-output systems, officially supported in Python and JavaScript but ported to many languages through community projects. Building a RAG application involves: (1) Choosing an LLM; (2) Loading the documentation to search over; (3) Splitting documents into chunks based on characters; (4) Converting chunks into document embeddings using an embedding model; (5) Storing embeddings in a vector database (like Chroma);

@GOTO- · 03:06

Why this video is valuable: This video contextualizes the engineering limitations of vanilla LLMs. It covers structural constraints like token context ceilings and knowledge cutoffs, establishing why retrieval architectures (RAG) are mathematically necessary for grounding models in external enterprise knowledge.

Module 1 Knowledge Checkpoint

  • Understand the difference between parametric knowledge (model weights) and source knowledge (context-injected data).
  • Explain how context limits (e.g., 4k vs. 128k tokens) restrict the amount of raw text that can be directly passed to a model.
  • Articulate why fine-tuning is used for tone, style, and domain specialization, whereas RAG is used for factual grounding and data access.

Module 2: LLM APIs & Vector Embeddings

In this module, you will learn to represent unstructured text mathematically. You will study how text is transformed into dense vector representations (embeddings), how models like Sentence-BERT capture semantic intent, and how to programmatically make similarity searches using vector spaces.

Recommended Videos

Vector Search with LLMs - Computerphile
Vector Search with LLMs - Computerphile
@Computerphile04:0962232
Vector Search with LLMs - Computerphile

A complete Retrieval-Augmented Generation system integrates multiple components: document chunking (splitting large texts into manageable pieces), vector embedding, database storage (using systems like Chroma), and similarity search. The pipeline processes queries by: (1) embedding the query, (2) searching the database for nearest neighbors, (3) retrieving relevant passages, (4) feeding these to a language model for answer generation. This architecture enables automated question-answering ground

@Computerphile · 04:09

Why this video is valuable: This video provides a brilliant, visual explanation of vector embeddings. You will learn how words and phrases are mapped into a multi-dimensional coordinate space where mathematical proximity correlates directly to semantic meaning.

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)
Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)
@umarjamilai49:2484904
Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Retrieval Augmented Generation (RAG) is a technique that enhances language models by integrating external knowledge retrieval through a pipeline: documents are split into sentences, converted to embedding vectors using models like Sentence BERT, stored in a vector database (often using HNSW algorithm for efficient similarity search), and queried by converting the question to an embedding; the top matching documents are retrieved, combined with the query in a prompt template, and fed to a languag

@umarjamilai · 49:24

Why this video is valuable: This is a comprehensive mathematical and conceptual deep dive. Umar Jamil explains the architecture of embedding pipelines, the mechanics of Sentence-BERT, and how high-dimensional indices like Hierarchical Navigable Small World (HNSW) enable rapid approximate nearest neighbor (ANN) lookups.

Building a RAG application from scratch using Python, LangChain, and the OpenAI API
Building a RAG application from scratch using Python, LangChain, and the OpenAI API
@underfitted10:52103438
Building a RAG application from scratch using Python, LangChain, and the OpenAI API

Retrieval Augmented Generation (RAG) combines information retrieval with large language models to enhance answer quality. The core architecture retrieves relevant documents from a database and uses them as context for generative models. This addresses the fundamental limitation of traditional LLMs: they cannot process very long documents due to token context limits. The video introduces LangChain as the primary framework for building RAG systems, demonstrating how to initialize models (using Cha

@underfitted · 10:52

Why this video is valuable: This hands-on implementation guide walks through the mechanics of initializing API calls to OpenAI, transforming strings to vectors, and building a foundational vector lookup routine using raw Python.

Module 2 Knowledge Checkpoint

  • Define vector embeddings and how dimensions (e.g., 1536 for text-embedding-3-small) represent semantic coordinate values.
  • Explain the difference between dot product, cosine similarity, and Euclidean distance metric choices for vector search.
  • Describe how Sentence-BERT transforms bidirectional encoder outputs into unified dense text vectors.

Module 3: Introduction to RAG Architecture

This module introduces the classic two-stage architecture of RAG: Ingestion (Offline Pipeline) and Retrieval-Generation (Online Pipeline). You will explore the tooling ecosystem, comparing the capabilities of LangChain and LlamaIndex, and code your first complete orchestration loop.

Recommended Videos

RAG Explained
RAG Explained
@IBMTechnology08:03237737
RAG Explained

Retrieval Augmented Generation (RAG) is a technique where large language models combine their general knowledge with real-time data retrieval from vector databases to provide accurate, up-to-date answers; this approach addresses the limitation of static LLM training data by dynamically incorporating relevant information from external sources, making it particularly valuable for business applications requiring current and specific data.

@IBMTechnology · 08:03

Why this video is valuable: An exceptional whiteboard-style breakdown of the RAG conceptual pipeline. It clearly demarcates the split between how data is prepared, stored, searched, and formatted as a prompt template for LLM inference.

LangChain vs. LlamaIndex - What Framework to use for RAG?
LangChain vs. LlamaIndex - What Framework to use for RAG?
@codingcrashcourses853316:5023114
LangChain vs. LlamaIndex - What Framework to use for RAG?

Both LangChain and LlamaIndex implement Retrieval Augmented Generation (RAG) through similar core workflows: loading data, chunking documents, creating embeddings, indexing in vector stores, and retrieving relevant documents for LLM generation. LangChain provides a more low-level, customizable approach with explicit chain construction using the LangChain Expression Language, while LlamaIndex offers a higher-level abstraction with a more streamlined query engine interface. The choice between them

@codingcrashcourses8533 · 16:50

Why this video is valuable: This guide explains the core architectural differences between LangChain and LlamaIndex. It clarifies how LangChain serves as a general-purpose, component-chaining library, whereas LlamaIndex focuses deeply on index structure, query tasks, and data ingestion over deep document structures.

Complete RAG Crash Course With Langchain In 2 Hours
Complete RAG Crash Course With Langchain In 2 Hours
@krishnaik0602:08:09159051
Complete RAG Crash Course With Langchain In 2 Hours

Retrieval-Augmented Generation (RAG) is a technique that optimizes large language model outputs by referencing an authoritative external knowledge base outside the model's training data, thereby preventing hallucinations and enabling integration of organizational data without expensive model retraining; the complete RAG pipeline consists of two main stages: the data injection pipeline (which involves document parsing, chunking, and embedding into a vector database) and the retrieval pipeline (wh

@krishnaik06 · 02:08:09

Why this video is valuable: A massive, project-based video that ties together local embedding creation, data staging, and system integration. You will construct a complete local/cloud RAG application using LangChain, giving you direct exposure to basic orchestration code.

Module 3 Knowledge Checkpoint

  • Draw the sequence of a typical RAG query operation, beginning with a user query and ending with a streamed response.
  • Contrast LangChain’s LCEL (LangChain Expression Language) with LlamaIndex’s node-and-query engine abstraction.
  • Implement a basic prompt wrapper that takes {retrieved_context} and {user_question} parameters and passes them to an LLM.

Module 4: Data Ingestion & Vector Databases

Ingesting raw, real-world data requires robust pipelines. In this module, you will master parsing complex document schemas (such as multi-page PDFs with tables), advanced chunking algorithms (recursive character splits vs. semantic boundaries), and hybrid database retrieval strategies.

Gap Alert (Self-Study Recommendation): While the selected videos cover layout parsing and chunking extremely well, production parsing of complex scanned tables and OCR can be finicky. We recommend building a local pipeline using the PyMuPDF, Unstructured, or LlamaParse libraries to supplement this module.

Recommended Videos

The BEST Way to Chunk Text for RAG
The BEST Way to Chunk Text for RAG
@AdamLucek16:5347088
The BEST Way to Chunk Text for RAG

This extensive section explores advanced semantic chunking approaches and their rigorous evaluation. It begins with Greg Kamradt's original semantic chunker, which uses embedding models and cosine similarity to identify natural topic transitions through sliding windows of local context. Chroma DB modified this by adding binary search to maintain target chunk sizes, addressing unpredictably large chunks in the original implementation. The section then advances to cluster semantic chunking, which

@AdamLucek · 16:53

Why this video is valuable: This is a crucial overview of text segmentation. It progresses from static token-length boundaries to Greg Kamradt's semantic chunking technique, showing how to calculate cosine distance variance between adjacent sentences to find natural semantic break points.

Semantic Chunking for RAG with #langchain
Semantic Chunking for RAG with #langchain
@AI-Makerspace01:02:517839
Semantic Chunking for RAG with #langchain

Semantic chunking is an advanced text splitting method for Retrieval-Augmented Generation (RAG) systems that uses embedding models to identify meaningful semantic boundaries between sentences, grouping similar consecutive sentences into larger chunks while separating dissimilar ones, which has been shown to produce more relevant answers compared to traditional recursive character text splitting methods.

@AI-Makerspace · 01:02:51

Why this video is valuable: An exhaustive code-along session that demonstrates the programmatic setup of semantic chunkers within LangChain. It explains how embedding models are used mid-ingestion to establish boundary cuts.

Stop Losing Context! How Late Chunking Can Enhance Your Retrieval Systems
Stop Losing Context! How Late Chunking Can Enhance Your Retrieval Systems
@engineerprompt10:2931323
Stop Losing Context! How Late Chunking Can Enhance Your Retrieval Systems

This comprehensive section covers the evolution of retrieval techniques in RAG systems. It begins with embedding model fundamentals—max tokens and embedding dimension—as critical selection criteria. The video explains how standard RAG pipelines compress information through chunk-level embedding, causing context loss. Late chunking emerges as the superior solution by reversing the pipeline: first embedding entire documents to capture global context, then performing chunking and mean pooling. This

@engineerprompt · 10:29

Why this video is valuable: Introduces "Late Chunking"—a breakthrough method for retaining global context in embeddings. Instead of tokenizing and embedding isolated segments, late chunking embeds the entire document first using a transformer model to capture global attention before chunk boundary pooling is applied.

RAG for Complex PDFs with #llamaindex
RAG for Complex PDFs with #llamaindex
@AI-Makerspace01:00:0833044
RAG for Complex PDFs with #llamaindex

This video demonstrates how to build Retrieval Augmented Generation (RAG) systems for complex PDFs containing both text and tables using LlamaIndex and Unstructured.io tools. The key insight is that complex PDFs with tables require special processing: converting PDFs to HTML first, then using Unstructured.io's embedded table parser to create hierarchical node references that enable recursive retrieval. This allows the system to answer questions about both qualitative text and quantitative tabula

@AI-Makerspace · 01:00:08

Why this video is valuable: PDFs with embedded vector graphics, tables, and multi-column formats are notoriously hard to parse. This session teaches you how to leverage Unstructured.io tools and LlamaIndex to structure complex PDFs into tabular nodes.

Mastering Retrieval for LLMs - BM25, Fine-tuned Embeddings, and Re-Rankers
Mastering Retrieval for LLMs - BM25, Fine-tuned Embeddings, and Re-Rankers
@TrelisResearch01:50:0812590
Mastering Retrieval for LLMs - BM25, Fine-tuned Embeddings, and Re-Rankers

This tutorial demonstrates that combining BM25 keyword search with fine-tuned semantic similarity models achieves superior retrieval performance (over 90% accuracy) compared to using either technique alone, with fine-tuning providing better latency-performance trade-offs than reranking approaches; the key pipeline involves: (1) document chunking with sentence-aware boundaries, (2) BM25 for keyword-based retrieval, (3) fine-tuned encoder models for semantic similarity, and (4) optional reranking

@TrelisResearch · 01:50:08

Why this video is valuable: Perfect for mastering the retrieval step. This in-depth masterclass explains why dense vector search alone can fail (especially with obscure product IDs, SKU numbers, or exact keywords) and how to configure hybrid search pipelines combining sparse keyword BM25 indices with dense vector systems.

Module 4 Knowledge Checkpoint

  • Understand why chunk size vs. chunk overlap tradeoffs exist, and how token-based overlap preserves context across splits.
  • Code a RecursiveCharacterTextSplitter pipeline using separators ["\n\n", "\n", " ", ""].
  • Describe the structural difference between sparse retrieval (BM25 token match index) and dense retrieval (vector spatial match).
  • Explain how late chunking leverages self-attention layers to distribute global contextual information before creating individual chunk arrays.

Module 5: Advanced RAG & Production Optimization

This final module focuses on advanced techniques to make your RAG application performant, secure, and production-ready. You will build query expansion and re-ranking routines, configure agentic feedback loops, integrate with Model Context Protocol (MCP) environments, and quantitatively score your outputs using the RAGAS evaluation framework.

Recommended Videos

Advanced RAG 06 - RAG Fusion
Advanced RAG 06 - RAG Fusion
@samwitteveenai13:0525986
Advanced RAG 06 - RAG Fusion

RAG Fusion is a technique that improves Retrieval-Augmented Generation (RAG) by rewriting a single user query into multiple related queries, performing separate vector searches for each, and then combining the results using reciprocal rank fusion to produce a more comprehensive and accurate final answer.

@samwitteveenai · 13:05

Why this video is valuable: This video covers Query Translation. Learn how to write query expansion and "RAG Fusion" modules. This involves instructing an LLM to decompose a singular user query into multiple search variants, retrieving separate document spaces, and sorting results using Reciprocal Rank Fusion (RRF).

Advanced RAG Techniques with Arcee Trinity Mini (100% Local)
Advanced RAG Techniques with Arcee Trinity Mini (100% Local)
@juliensimonfr04:1119327
Advanced RAG Techniques with Arcee Trinity Mini (100% Local)

This segment explains reranking, a post-retrieval technique that adds an evaluation step to improve precision. After initial retrieval, each chunk is scored against the query using a reranking model, and top-scoring chunks are selected for answer generation. The presenter demonstrates combining multiple techniques (query rewriting, hybrid retrieval, reranking) to create a sophisticated system. While no single technique works optimally for all scenarios, combining them significantly improves perf

@juliensimonfr · 04:11

Why this video is valuable: Focuses on re-ranking, a standard optimization pattern for production. You will learn how to implement a two-stage retrieval pipeline: retrieve a large candidate chunk pool (e.g., top 100) using cheap bi-encoder cosine similarity, and filter down to the top 5 using a highly accurate local cross-encoder model.

Session 7: RAG Evaluation with RAGAS and How to Improve Retrieval
Session 7: RAG Evaluation with RAGAS and How to Improve Retrieval
@AI-Makerspace37:2135426
Session 7: RAG Evaluation with RAGAS and How to Improve Retrieval

This video explains how to evaluate Retrieval-Augmented Generation (RAG) systems using the RAGAS framework, which measures four key metrics: Context Precision (relevance of retrieved context to the question), Context Recall (ability to retrieve all relevant context), Answer Relevancy (how relevant the generated answer is to the question), and Faithfulness (whether the answer is fact-checkable without hallucinations). The speakers demonstrate advanced retrieval techniques including the Parent Doc

@AI-Makerspace · 37:21

Why this video is valuable: A stellar, comprehensive session on evaluation. You will learn to measure system performance using the RAGAS framework across four foundational evaluation vectors: Context Precision, Context Recall, Faithfulness, and Answer Relevancy.

RAG & MCP Fundamentals – A Hands-On Crash Course
RAG & MCP Fundamentals – A Hands-On Crash Course
@freecodecamp12:1742311
RAG & MCP Fundamentals – A Hands-On Crash Course

Retrieval Augmented Generation (RAG) connects language models to private data sources for accurate, context-aware answers. The process involves three stages: retrieval (finding relevant documents), augmentation (enriching prompts with retrieved info), and generation (LLM produces response). RAG differs from prompt engineering (creating better instructions) and fine-tuning (training on examples). RAG excels for dynamic factual information where policies change frequently, while fine-tuning suits

@freecodecamp · 12:17

Why this video is valuable: Explores state-of-the-art Model Context Protocol (MCP) implementations. This lesson introduces open standards for connecting LLMs securely to modular external data sources, enterprise databases, and dynamic microservice infrastructures.

RAG in Production - LangChain & FastAPI
RAG in Production - LangChain & FastAPI
@codingcrashcourses853311:5118069
RAG in Production - LangChain & FastAPI

When deploying RAG applications in production, three critical best practices must be followed: (1) Always use APIs as a standardized interface and security layer between clients and vector databases, avoiding direct database connections; (2) Implement asynchronous programming to prevent blocking code that slows down concurrent requests, using LangChain's async execution methods like run_in_executor for threadpool-based parallelism; (3) Avoid reindexing documents on every change by using digest-b

@codingcrashcourses8533 · 11:51

Why this video is valuable: This video details the core steps for moving RAG prototypes to production. It covers wrapping LLM loops inside secure FastAPI wrappers, implementing asynchronous streaming endpoints, and managing production API credentials.

Module 5 Knowledge Checkpoint

  • Program a query translator that rewrites complex input statements into distinct sub-questions.
  • Understand how a cross-encoder model computes joint attention between a query and a candidate document, and why it is too slow to use for initial dense retrieval.
  • Set up a RAGAS test suite that automatically evaluates system outputs for hallucinations using LLM-as-a-judge patterns.
  • Explain how Model Context Protocol (MCP) abstracts data schemas for secure tool use by agents.

Course Map

Below is the conceptual dependency flow of the curriculum. Move sequentially to ensure foundational concepts are fully established before tackling complex pipeline optimizations.

Key People Index

The following researchers, developers, and educators are featured across the modules of this curriculum:

  • Jeremy Howard: Co-founder of Fast.ai, a leading researcher who demystifies Deep Learning and language modeling for developers.
  • Umar Jamil: A technical educator known for deep math, transformer architectural dissections, and indexing theory.
  • Greg Kamradt: Creator of foundational advanced chunking approaches (such as the embedding-variance semantic chunker).
  • Jerry Liu: Co-founder of LlamaIndex, pioneer of hierarchical data index structures for LLMs.

Final Self-Assessment

Complete this comprehensive self-assessment before deploying your RAG pipelines to staging or production environments.

  • Context vs Weight Separation: Can you clearly outline to stakeholders why fine-tuning a model on custom documents is inefficient for real-time information updates compared to RAG?
  • API Orchestration: Can you write a clean, asynchronous Python loop using LangChain/LlamaIndex that retrieves chunks from a store, styles a prompt, and streams the generation output?
  • Data Cleansing: Have you implemented a custom pre-processing parser to handle headers, footers, page numbers, and inline figures inside raw PDF formats?
  • Semantic Chunking: Have you analyzed your average document layout and configured either an embedding-boundary chunker or a recursive text splitter with tailored overlaps?
  • Database Engineering: Do you know how to build metadata filters inside your vector index (e.g., filtering by tenant ID, date, or department) to secure data retrieval boundaries?
  • Hybrid Search: Is your search pipeline configured to blend lexical (BM25) and semantic vector similarity search?
  • Re-ranking Optimization: Does your runtime query engine use a cross-encoder re-ranking model to score the top-K retrieved vector nodes before prompt injection?
  • LLM-as-a-Judge Validation: Are you running systematic regressions on your model outputs using RAGAS to check for Faithfulness (hallucination checks) and Answer Relevancy?
  • Microservice Architecture: Is your RAG core wrapped in a FastAPI gateway using connection pooling, error boundaries, and rate limits to handle heavy user traffic?
Explore Further

Related Artificial Intelligence Roadmaps

View All→
artificial-intelligence

Building Recommenders: Two-Tower & Vector DBs

01Foundations of Python & Machine Learning Basics
02Introduction to Recommender Systems
03Deep Learning & Representation Learning
+ 3 more learning modules
75h6 modules
Explore Roadmap
artificial-intelligence

Latent Diffusion: PyTorch, UNet & CLIP

01Deep Learning & PyTorch Foundations
02Convolutional Networks & Autoencoders
03The Math and Mechanics of Diffusion Models
+ 3 more learning modules
80h6 modules
Explore Roadmap
artificial-intelligence

NeRFs: 3D Reconstruction & Instant-NGP

01Math & Python Foundations for 3D Vision
02Photogrammetry & Camera Pose Estimation
03Neural Radiance Fields (NeRF) Core Concepts
+ 2 more learning modules
16h5 modules
Explore Roadmap
artificial-intelligence

TFT Forecasting: Energy Demand & Deep Learning

01Programming & Data Science Foundations
02Introduction to Machine Learning & Time Series
03Deep Learning & Sequential Models
+ 3 more learning modules
38h6 modules
Explore Roadmap