Academic Databases
Explore Our Semantic Databases
STEM & Tech
Math, Physics, CS & Engineering...
Economics
Macro, Micro, Econometrics...
Business
Management, Marketing, Finance...
Health & Life Sciences
Biology, Medicine, Neuroscience...
Humanities & Social Sciences
Psychology, Law, Philosophy, History...
General Studies
General & interdisciplinary studies...
Explore Topics
Features

Running Free Hugging Face LLMs Locally with LangChain

Added: 2026-08-07

Deep Dive

Proficiency in Python programming, including managing virtual environments, package installation, and handling external APIs.

Python Full Course for Beginners

Python Full Course for Beginners

208

This section covers Python's package ecosystem: pip as the package manager for installing external libraries, requirements.txt for dependency management ensuring reproducible environments, and virtual environments for isolating project dependencies. It demonstrates how to create, activate, and manage virtual environments, and how to share dependencies between projects using requirements.txt files.

Click to watch →

APIs for Beginners - How to use an API (Full Course / Tutorial)

APIs for Beginners - How to use an API (Full Course / Tutorial)

4.7M

Python uses virtual environments to isolate project dependencies locally. Creating a virtual environment with python -m venv .venv generates a self-contained directory with its own Python interpreter and packages. Activating the environment ensures pip installs packages locally rather than globally, preventing conflicts between projects.

Click to watch →

آموزش کامل Python | قسمت 14: معرفی Pip، محیط‌های مجازی و ساخت REST API با پکیج Requests

آموزش کامل Python | قسمت 14: معرفی Pip، محیط‌های مجازی و ساخت REST API با پکیج Requests

2.2K

Pip is Python's package manager for installing, updating, and managing packages. It automatically installs with Python. Commands include: pip install [package], pip install [package]==[version], pip install --upgrade [package], pip uninstall [package], pip list. Find packages on pypi.org. Virtual environments solve conflicts by isolating packages per project. Create with python -m venv [folder], activate with Scripts/activate (Windows) or source bin/activate (Mac/Linux), deactivate with deactivate. Install packages within environments using pip. Each project can use different package versions without conflicts.

Click to watch →

Python Packages | Python Packages Tutorial | Python For Beginners | Intellipaat

Python Packages | Python Packages Tutorial | Python For Beginners | Intellipaat

10.5K

External packages are pre-built packages created by other developers that can be installed and used in Python projects. Python uses pip as its package manager to install and manage these external packages. Pip maintains a list of installed packages and handles dependency resolution, automatically downloading and installing all required dependencies. Pip can install packages globally (available system-wide) or locally (within a specific project). Virtual environments isolate package installations for different projects, preventing version conflicts. Pipenv simplifies virtual environment management by creating a virtual environment, generating a Pipfile containing package versions, and creating a Pipfile.lock that locks exact versions for reproducible builds. When sharing projects, other developers can run pipenv install to automatically set up the same environment. To use external packages, install them using pip or pipenv, then import them in your code. Pipenv automatically handles dependencies, creates the virtual environment, and generates the necessary files, ensuring anyone working on the project can easily set up the same environment.

Click to watch →

What are Virtual Environments in Python? - Python Tutorial

What are Virtual Environments in Python? - Python Tutorial

13.8K

Python virtual environments (venv) are isolated Python environments that allow developers to manage project-specific dependencies separately from the global Python installation, solving conflicts when multiple projects require different versions of the same package; they are created using the python -m venv command, activated with scripts/activate (Windows) or source bin/activate (Linux/Mac), and can be managed using requirements.txt files to specify exact package versions for reproducibility.

Click to watch →

Deep Dive

Fundamental concepts of Large Language Models (LLMs), including tokens, context windows, and how model parameters relate to hardware requirements.

Xiaomi's Secret AI Is FREE & Beating Claude

Xiaomi's Secret AI Is FREE & Beating Claude

2.7K

Large language models are characterized by two key metrics: total parameters and context window length. Total parameters represent the number of internal variables (dials and switches) trained to understand language, write code, and reason through problems. More parameters allow the model to hold more information simultaneously. The context window defines how much text the model can process in a single conversation, measured in tokens (roughly 3/4 of a word). A 1 million token context window means the model can handle approximately 800,000 words at once, compared to older models that typically handled only 4,000 tokens (about three pages). The activated parameters (42 billion in Myo V2 Pro) are the portion actually used for each task, while the full trillion remains available as reserve capacity.

Click to watch →

Контекстное окно в Claude Code / Antigravity / Codex: всё что необходимо знать

Контекстное окно в Claude Code / Antigravity / Codex: всё что необходимо знать

2.1K

LLMs are neural networks trained on massive text data that predict the most probable next token. They don't truly understand meaning but find statistical patterns. A token is a basic text unit—English words equal one token, Russian words average 1.5-2 tokens. Modern models like Claude Opus 4.5 have 1 million token context windows, equivalent to 3-4 volumes of 'War and Peace'. Despite this large capacity, context window management is critical because each interaction accumulates tokens, and models don't truly remember conversations but process accumulated context with each request.

Click to watch →

Hugging Face Basics + Open-Source LLMs Explained (Gemma, LLaMA, Mistral) | LLM Bootcamp Hindi 🚀

Hugging Face Basics + Open-Source LLMs Explained (Gemma, LLaMA, Mistral) | LLM Bootcamp Hindi 🚀

25.4K

This section covers the technical foundations of running LLMs. CPU handles tasks sequentially, making it slow for AI workloads, while GPU handles thousands of calculations simultaneously through parallel processing. Modern GPUs like NVIDIA RTX 4090 have 16,384 cores for massive parallel processing. LLMs come in different sizes: small models (DistilBERT, TinyLlama) run on CPUs with limited resources, while large models (GPT-3, Llama 70B) require significant GPU resources. Model parameters are numerical values storing learned knowledge during training. GPT-2 has 124 million parameters, while GPT-3 has 175 billion parameters. More parameters mean more learned patterns and potentially better performance, but also larger file sizes and slower inference. Parameters are stored as weights, values, and rules representing learned relationships from training data.

Click to watch →

AI builder essentials: Tokens, Context Windows, RAG 101

AI builder essentials: Tokens, Context Windows, RAG 101

301

Large Language Models (LLMs) operate on tokens, which are basic units between words and sentences (averaging 3/4 of a word). LLMs function like sophisticated autocomplete, processing input tokens and generating output tokens through mathematical operations on training data. Generative AI infers responses from training data and user context, producing inherently non-deterministic results. Retrieval Augmented Generation (RAG) addresses training data cutoff limitations by augmenting responses with external context. Context windows define how many tokens models can process at once, with expanding windows enabling more complex tasks. Token maxing—using maximum tokens—may not always optimize productivity, as users often lack awareness of true inference costs. As users develop intuition about token scarcity and costs, they will become more discerning about task selection and tool effectiveness.

Click to watch →

Тренінг «Використання штучного інтелекту у сфері відкритих даних»

Тренінг «Використання штучного інтелекту у сфері відкритих даних»

551

Large language models operate on tokens, which are parts of words rather than complete words or characters. The context window determines how many tokens a model can process, with current models like Claude having 200,000 tokens and Google claiming 1 million tokens. Models generate text token by token, predicting the probability of the next token at each step. The temperature parameter controls creativity, ranging from 0 to 2, with even temperature 0 producing different responses to identical prompts. Model parameters serve as a key metric for understanding model size and power, with open source models typically named by parameter count (e.g., Llama 7B, Llama 70B).

Click to watch →

Deep Dive

Familiarity with the Hugging Face ecosystem, particularly how to navigate the Model Hub and understand model configurations.

What Is Hugging Face and How To Use It

What Is Hugging Face and How To Use It

233.1K

The Hugging Face Model Hub contains over 1.5 million open-source AI models organized into categories including computer vision, natural language processing, and audio. Models are sorted by popularity by default. Since all models are open source and free, users can examine the underlying code, deploy models, and even discuss them in community forums. Each model page shows related Spaces that use that particular model.

Click to watch →

“Smol” AI Agents Full Beginner Course

“Smol” AI Agents Full Beginner Course

18.5K

Hugging Face hosts over 1.3 million models across various categories including text generation, image processing, audio, and video. Users can navigate to models.huggingface.co and filter by model type. Each model page shows deployment options including serverless inference API (free) or paid pro subscriptions for certain models. The inference API serverless option allows direct Python code execution using the model ID and a Hugging Face token.

Click to watch →

Construisez vos applications NLP, Computer Vision et Speech avec les modèles Transformers

Construisez vos applications NLP, Computer Vision et Speech avec les modèles Transformers

4.2K

Hugging Face provides a comprehensive ML development stack: Model Hub with 73,000+ pre-trained models; Transformers library for model usage; Accelerate for multi-GPU/TPU training; Optimum for hardware acceleration; Trainer for zero-code AutoML; Spaces for building web apps; Inference API for deployment. Partnerships include AWS SageMaker and Azure Marketplace. The Hub serves as a central repository analogous to GitHub, with contributions from major companies, universities, and individuals. This ecosystem enables rapid prototyping where complex applications can be built in dozens of lines of code using pre-trained models.

Click to watch →

LangChain Models | Indepth Tutorial with Code Demo | Video 3 | CampusX

LangChain Models | Indepth Tutorial with Code Demo | Video 3 | CampusX

341.9K

To configure Hugging Face models locally: (1) Import ChatHuggingFace and HuggingFacePipeline from huggingface_hub, (2) Load environment variables, (3) Create a HuggingFacePipeline object using HuggingFacePipeline.from_model_id() with the model repository ID, (4) Configure parameters like temperature and max_tokens in a dictionary, (5) Pass this configuration to the ChatHuggingFace object, (6) Call .invoke() with a prompt. The pipeline handles model loading and inference locally.

Click to watch →

Fine Tune a Multimodal LLM "IDEFICS 9B" for Visual Question Answering

Fine Tune a Multimodal LLM "IDEFICS 9B" for Visual Question Answering

9.3K

Deploying fine-tuned models to Hugging Face Hub involves: (1) Installing huggingface_hub CLI and logging in with access token, (2) Using model.push_to_hub() with a repository name, (3) Optionally setting private=False to make the model publicly accessible, (4) The repository will contain adapter models, optimizer states, and configuration files that others can use for inference or further fine-tuning.

Click to watch →

Deep Dive

Basic understanding of GPU computing concepts, specifically NVIDIA CUDA, VRAM allocation, and the difference between CPU and GPU inference.

What Every LLM Developer Needs to Know About GPUs with Charles Frye

What Every LLM Developer Needs to Know About GPUs with Charles Frye

4.2K

This section covers the foundational concepts of GPU computing for AI. It distinguishes between CUDA's three layers: PTX (instruction set architecture), the abstract programming model, and specific implementations like CUDA C++. For inference, VRAM is the primary constraint—model weights must fit in GPU RAM; otherwise, performance degrades significantly. The rule of thumb is parameters × precision + 30% overhead for KV caching. Quantization enables aggressive reduction of VRAM requirements by storing weights at lower precision (8-bit or 4-bit), allowing larger models on smaller GPUs. These principles apply across all AI workloads, from inference to fine-tuning.

Click to watch →

¿Estafa? | DESMONTANDO LA FALACIA de la VRAM en las GPU: Te explico lo que NADIE dice, con PRUEBAS.

¿Estafa? | DESMONTANDO LA FALACIA de la VRAM en las GPU: Te explico lo que NADIE dice, con PRUEBAS.

67.7K

NVIDIA's graphics driver contains a mechanism that allocates system shared memory before the GPU's dedicated VRAM is fully utilized. This occurs when VRAM usage exceeds approximately 75-80% of capacity (around 6GB+ on the 4060). The driver stores data in system RAM instead of filling the GPU's VRAM, which reduces performance. This explains why the 4060 shows poor performance despite not technically saturating its 8GB VRAM - the system memory allocation is the actual bottleneck.

Click to watch →

CUDA Part A: GPU Architecture Overview and CUDA Basics; Peter Messmer (NVIDIA)

CUDA Part A: GPU Architecture Overview and CUDA Basics; Peter Messmer (NVIDIA)

128.9K

CUDA has a hierarchical memory model: host memory (CPU RAM), global memory (GPU RAM accessible by all threads), shared memory (fast scratchpad within each Streaming Multiprocessor for thread block cooperation), registers (private to each thread), and local memory (global memory allocated per-thread when registers are insufficient). Memory bandwidth varies dramatically: PCI Express transfers are 5-10 GB/s, Global memory to SMX is ~200 GB/s, and register access is near-instantaneous. CUDA memory allocation uses cudaMalloc (pointer-to-pointer argument) and cudaFree. cudaMemcpy transfers data with direction specified by cudaMemcpyHostToDevice or cudaMemcpyDeviceToHost. Kernel launches are non-blocking, allowing CPU-GPU overlap, while cudaMemcpy operations are blocking by default to ensure safe synchronization.

Click to watch →

Inteligência Artificial EXPLICADA de maneira SIMPLES em APENAS 20 minutos

Inteligência Artificial EXPLICADA de maneira SIMPLES em APENAS 20 minutos

43K

CPUs are designed for low-latency operations, while GPUs are optimized for massive parallel processing. NVIDIA's CUDA technology enables efficient GPU utilization for AI workloads. Most NVIDIA AI tools are open-source and largely free, and consumer GeForce GPUs can support many AI development tasks. NVIDIA provides containers with pre-configured environments for easy setup.

Click to watch →

CPU vs GPU: Which is More Powerful?

CPU vs GPU: Which is More Powerful?

121.4K

CPUs are designed for sequential general-purpose computing with powerful cores handling complex logic, branching, and decision-making. GPUs are designed for parallel processing with thousands of smaller cores working simultaneously on independent calculations. CPU architectures emphasize high clock speeds, large caches, and complex instruction sets, while GPU architectures optimize for high-throughput parallel processing with simpler cores and smaller caches. The Sieve of Eratosthenes systematically eliminates multiples of each prime to find all primes up to a limit. A segmented sieve hybridizes CPU and GPU work: the CPU calculates primes only up to the square root, then delegates the heavy work of marking multiples through the entire sieve to the GPU. CUDA (Compute Unified Device Architecture) is Nvidia's parallel processing platform enabling GPU acceleration. CUDA code resembles C++ but requires the global keyword to indicate GPU execution. The nvcc compiler handles both CPU and GPU code compilation. CUDA kernels execute asynchronously, with atomic operations ensuring thread-safe memory access when multiple threads modify shared memory simultaneously.

Click to watch →

Accessing Free Models

Environment Setup

Token Authentication

Basic Model Usage

GPU Acceleration

Model Selection Guide

LangChain Integration

Advanced Pipeline

Accessing Free Models

0:00

Playing Section

1
Introduces using Hugging Face and LangChain for free local models.
2
Explains the purpose and role of the Transformers package.
3
Requires an account to accept model license terms.

Deep Dive

Model compression and optimization techniques such as quantization (GGUF, AWQ) to run larger models on limited consumer hardware.

All You Need To Know About Running LLMs Locally

All You Need To Know About Running LLMs Locally

307K

Various quantization formats enable running larger models on limited hardware. GGUF is a predecessor of GGML in binary format supporting different quantization schemes and running on CPU within single files. EXL 2 mixes quantization levels within models achieving average bit rates between 2-8 bits per weight, being the fastest optimization but only available on Nvidia GPUs. AWQ sets smaller weights to zero and rounds others to nearest thresholds. GPTQ applies layerwise quantization minimizing output error. These methods reduce model size by shrinking precision or quantizing weights, though they may slightly degrade performance.

Click to watch →

Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ)

Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ)

37.7K

This tutorial explains three common quantization methods for running large language models on limited hardware: GPTQ (post-training quantization optimized for GPU inference), GGUF (formerly GGML, focused on CPU execution with optional GPU layers), and AWQ (activation-aware weight quantization that prioritizes important weights). The choice depends on hardware constraints—use native loading if GPU capacity permits, GPTQ for moderate GPUs, GGUF for very small GPUs, and AWQ for newer applications where available.

Click to watch →

Long continuous AI video is here! Free & open-source

Long continuous AI video is here! Free & open-source

152.4K

For systems with limited VRAM, GGUF quantized versions of One 2.2 provide significant memory reduction. Available compression levels (Q2-Q6) reduce model sizes from 15GB to as little as 5.3GB. Q2 fits in 6GB VRAM, Q3 in 8GB, while Q5-Q6 require 12GB. The workflow adapts by replacing standard nodes with unit loader nodes after installing comfy_gguf. This optimization enables professional-grade video generation on consumer-grade hardware, democratizing access to advanced AI video tools.

Click to watch →

How to run GLM-OCR, a state-of-the-art OCR model, 100% locally

How to run GLM-OCR, a state-of-the-art OCR model, 100% locally

611

Model quantization reduces computational and memory requirements by decreasing weight precision from 16/32-bit floats to lower bit representations (Q2=2bits, Q3=3bits, Q4=4bits). Llama CPP implements official quantizations in GGUF format, an efficient storage format for model weights and their quantized variants. This enables running large models on consumer hardware by trading minimal accuracy loss for dramatic size reduction. The GGUF format supports multiple quantization levels, allowing users to balance performance against hardware constraints. This technology makes advanced models accessible to individuals without specialized GPU infrastructure.

Click to watch →

LLM Quantization Explained

LLM Quantization Explained

389

LLM quantization is a technique that reduces the memory footprint of large language models by storing their parameters in fewer bits (such as FP16, BF16, FP8, INT8, or INT4), enabling models like the 70B Llama 3 to run on consumer hardware that would otherwise require 140GB of memory; this process involves three key components—number formats determining bit precision, quantization methods like GPTQ and AWQ that preserve model quality by identifying important weights, and file formats like GGUF that store the compressed model for local execution—though lower precision introduces trade-offs where models may degrade in reasoning, coding, and instruction-following capabilities.

Click to watch →

Deep Dive

Implementing Retrieval-Augmented Generation (RAG) by integrating local vector databases (like Chroma or FAISS) with local LLMs for private document querying.

Local RAG LLM with Ollama

Local RAG LLM with Ollama

3.4K

RAG provides context for LLM questions by storing documents in a vector database instead of fine-tuning the model. When a question is asked, it's encoded and searches for K nearest neighbors to provide context. This approach makes the LLM the reasoning engine while documents provide current information. The tutorial demonstrates implementing RAG locally using Chroma DB, ollama, and pandas packages, enabling all processing to run on your own computer without cloud dependencies.

Click to watch →

Build a Private Chat My PDF Data RAG System with LangChain, Ollama, FAISS Vector Store & Llama 3.2

Build a Private Chat My PDF Data RAG System with LangChain, Ollama, FAISS Vector Store & Llama 3.2

14.4K

A private Retrieval-Augmented Generation (RAG) system enables secure, local question answering by embedding documents into vectors, storing them in a vector database (FAISS), retrieving relevant chunks using cosine similarity, and generating responses with a local LLM (Llama 3.2 via Ollama), ensuring data privacy while leveraging AI capabilities.

Click to watch →

How RAG (Retrieval-Augmented Generation) talks with your private data using LLM | AI Heroes 2023

How RAG (Retrieval-Augmented Generation) talks with your private data using LLM | AI Heroes 2023

1.8K

Retrieval-Augmented Generation (RAG) is an AI technique that combines Large Language Models (LLMs) with vector databases to enable natural language queries on private data without requiring model retraining. The system works by first converting documents into numerical vectors (embeddings) that capture semantic meaning, then using mathematical similarity calculations to retrieve relevant documents when a user asks a question. The retrieved documents are combined with the original question and sent to an LLM, which generates a natural language answer while providing source citations. This approach solves key LLM limitations including hallucination (providing answers without sources) and outdated knowledge, while maintaining data privacy by keeping information in local vector databases.

Click to watch →

Easy 100% Local RAG Tutorial (DeepSeek-R1) + Full Code

Easy 100% Local RAG Tutorial (DeepSeek-R1) + Full Code

4.1K

This tutorial demonstrates how to implement a completely private Retrieval-Augmented Generation (RAG) system on your local machine by combining a locally-run large language model (DeepSeek-R1 via Llama) with ChromaDB vector database. The system processes local documents through tokenization, chunking, and embedding using Sentence Transformers, then stores these embeddings in ChromaDB for efficient similarity searches. When users ask questions, the system embeds their query, searches the vector database to find relevant document chunks, and uses those chunks as context for the language model to generate informed answers. This approach enables private, on-premise AI assistants without API keys, though it requires significant computational resources compared to cloud-based alternatives.

Click to watch →

Local RAG Using Llama 3, Ollama, and PostgreSQL

Local RAG Using Llama 3, Ollama, and PostgreSQL

22.2K

A private Retrieval Augmented Generation (RAG) application can be built locally using open-source tools to ensure data privacy and control, where Meta Llama 3 serves as the language model, Ollama runs the embedding and generation models locally, and PostgreSQL with pgai and pgvector extensions functions as the vector database for storing documents and performing similarity searches.

Click to watch →

Deep Dive

Building autonomous agents and multi-step chains using LangChain's advanced state management and tool integration.

Agents Week 2026 | Лекция 1.2 Tools. MCP

Agents Week 2026 | Лекция 1.2 Tools. MCP

4.7K

Building agents with LangChain/LangGraph involves creating an agent state (list of messages), an LLM node that appends messages, a tool node containing all available tools, and a router that checks for tool calls. If tools were called, route to a tool execution node; otherwise, route to the end node. After tool execution, route back to the LLM for another iteration. For multi-turn conversations, use a reducer function that appends messages to preserve state across iterations. A flight booking agent example demonstrates this: tools for getting booking details, searching flights, and canceling bookings, with the agent asking for user confirmation before performing destructive actions. This enables agents to interact with databases and perform real-world actions. The agent can handle multi-step computations by chaining tool invocations, such as calculating 17 × 23 × 10 by calling multiply twice with intermediate results.

Click to watch →

16 Months of Building AI Agents in 60 Minutes

16 Months of Building AI Agents in 60 Minutes

200.2K

Chains are the core LangChain concept where language models and tools are placed sequentially to achieve results. The input of one model becomes the output of the previous, enabling complex multi-step tasks. An AI agent is a language model with tools that can make decisions and reason about which tools to use based on queries and environment. Unlike basic chatbots, agents are autonomous—they perform tasks without constant human interaction, can autocorrect errors, and handle complex workflows like checking dates, searching APIs, and generating outputs independently.

Click to watch →

Python Advanced AI Agent Tutorial - LangGraph, LangChain, Firecrawl & More!

Python Advanced AI Agent Tutorial - LangGraph, LangChain, Firecrawl & More!

96.2K

This tutorial demonstrates how to build advanced AI agents using Python frameworks LangChain and LangGraph, combined with Firecrawl for web research. The instructor shows two approaches: a simple agent that lets the LLM randomly call tools, and an advanced agent using LangGraph's state graph to enforce a predictable multi-step workflow. The advanced agent follows a structured process: extracting candidate tools from search results, researching each tool individually, and generating recommendations based on the accumulated data. This approach provides more control over the agent's behavior compared to letting the LLM freely decide which tools to use.

Click to watch →

The LangChain Cookbook Part 2 - Beginner Guide To 9 Use Cases

The LangChain Cookbook Part 2 - Beginner Guide To 9 Use Cases

86.2K

This section covers advanced LangChain capabilities. Evaluation tests pipelines by comparing predictions against ground truth. Tabular data querying uses SQL Database Chains to automatically discover tables and columns. Code understanding loads documentation for queries and generation. API integration enables models to determine endpoints and interpret responses. Chatbots use conversation buffer memory for context. Agents are autonomous decision-makers that reason about tasks, select tools from toolkits, and execute actions sequentially, representing an active area of development.

Click to watch →

Building end-to-end AI Agent in LangChain | Generative AI using LangChain | Video 18 | CampusX

Building end-to-end AI Agent in LangChain | Generative AI using LangChain | Video 18 | CampusX

109.8K

Building AI agents with LangChain requires: an LLM for reasoning capabilities, a set of tools for performing actions, and a prompt template that guides behavior. The create_react_agent function combines these components. The Agent handles high-level planning and decision-making about which tools to use, while the Agent Executor handles actual tool execution. This separation allows clear orchestration of multi-step tasks. The Agent generates thoughts and decides actions, while the Executor executes tools and returns observations.

Click to watch →

Deep Dive

Deploying local LLMs as production-ready microservices using serving frameworks like Ollama, vLLM, or FastAPI, paired with user interfaces like Gradio or Streamlit.

I Tested All 4 LLM Deployment Methods So You Don't Have To | Ollama, LLama.cpp, LM studio, vLLM

I Tested All 4 LLM Deployment Methods So You Don't Have To | Ollama, LLama.cpp, LM studio, vLLM

103

vLLM enables production-grade LLM deployment handling multiple concurrent API requests efficiently. Deployment requires creating GPU instances (A100 with 80GB VRAM), installing via pip, and resolving dependency conflicts. Commands specify model IDs, hosts, ports, context sizes, GPU counts, and model parameters. vLLM optimizes for commercial applications serving multiple users simultaneously. Framework selection depends on use case: LM Studio suits casual users with GUI simplicity; llama.cpp offers developer flexibility; vLLM handles production-scale serving. Most individual users should choose between LM Studio for ease or llama.cpp for customization.

Click to watch →

Запускаю DeepSeek на домашнем сервере с видеокартой. Подписка на Chat GPT больше не нужна.

Запускаю DeepSeek на домашнем сервере с видеокартой. Подписка на Chat GPT больше не нужна.

151.3K

Deploying LLMs locally requires a complete software stack. Ollama is installed via a single script and automatically configures as a system service that starts on boot. The installer detects hardware and configures CPU or GPU backend. Ollama's web service runs on port 11434, which can be made accessible via Nginx reverse proxy for hostname-based access without port numbers. Open Web UI provides a graphical chat interface running as a Docker container, requiring 2GB RAM and 2 vCPUs. It connects to Ollama via API configuration and allows switching between multiple installed models. The interface requires manual configuration of the Ollama API connection and provides a more user-friendly experience than terminal-based interactions.

Click to watch →

รู้จักกับ Local LLM และการใช้งานผ่าน Ollama API

รู้จักกับ Local LLM และการใช้งานผ่าน Ollama API

23.2K

Local LLM deployment requires tools and platforms for model management and deployment. Hugging Face serves as a central hub hosting over 400,000 models, datasets, and applications, with Spaces providing demonstration environments. LLM Studio is a desktop application providing a user interface for running local LLMs on multiple operating systems, exposing API endpoints for programmatic access. Ollama is a framework for running LLMs locally, supporting models like Llama, Falcon, Mistral, Gemini 2, and DeepSeek. It provides command-line access through 'ollama' commands: 'ollama serve' starts the API server on port 11434, 'ollama run' downloads and runs models, and 'ollama list' displays installed models. Ollama acts as a standardized interface, making it easier to switch between models without changing application code.

Click to watch →

NEP ORIENTATION & SENSITIZATION PROGRAMME JUN B 06 July 2026

NEP ORIENTATION & SENSITIZATION PROGRAMME JUN B 06 July 2026

185

Ollama is an open-source framework for local LLM deployment. Requirements: Windows, i7 processor (10th gen), 8GB RAM (16GB recommended). Features: create custom LLMs from pre-trained models, run models without internet, complete privacy and security. Commands: create, run, pull, list. LM Studio provides similar functionality with chat interface and file upload capability. Both tools enable organizations to develop tailored AI models while maintaining data privacy.

Click to watch →

Advanced RAG Series | LLM Basics + Your First AI App Using FastAPI! (OpenAI & Ollama)

Advanced RAG Series | LLM Basics + Your First AI App Using FastAPI! (OpenAI & Ollama)

150

Ollama enables running LLMs locally on your computer without API keys. Download and install Ollama from ollama.com, then pull desired models (like llama3) using 'ollama pull llama3' command. Models consume significant disk space, so verify available storage before downloading. Once installed, access models locally through the Ollama server. Like OpenAI, you can customize behavior using system messages when invoking the local LLM.

Click to watch →

Accessing Free Models

Environment Setup

Token Authentication

Basic Model Usage

GPU Acceleration

Model Selection Guide

LangChain Integration

Advanced Pipeline

Accessing Free Models

0:00

Playing Section

1
Introduces using Hugging Face and LangChain for free local models.
2
Explains the purpose and role of the Transformers package.
3
Requires an account to accept model license terms.

223.8K views5.7K22:58@TechWithTimOriginal Release: 2025-02-18

This tutorial demonstrates how to access and run thousands of free AI models from HuggingFace locally on your computer using just a few lines of Python code with the Transformers and LangChain libraries. The process involves setting up a virtual environment, installing dependencies (transformers, langchain, huggingface_hub), obtaining a HuggingFace access token, and using the pipeline API to load and execute models like text summarization or question answering. For GPU acceleration, install CUDA and PyTorch with GPU support. You can browse available models at huggingface.co/models and filter by task type, then integrate them with LangChain to build more sophisticated applications with memory and multiple model connections.

[00:00:00]today I'm going to show you how to access some of the best models that exist completely for free and locally on your own computer we're going to be doing that in just a few very simple lines of code using hugging face and Lang chain I'm going to show you how to use various hugging face models for free inside of a python application and then we'll connect them to Lang chain so we can build something even more interesting with that said let's get into the video so quickly let's get into some information that's important to understand we're going to be using something called Transformers now this is a free open source package that you can install with python using pip if you're familiar with that and it allows you to utilize tons of different models that come from hugging face now hugging face is a platform you can see it right here that has all kinds of Open Source models available some of them you do need to pay for or you need some specific access but most of them are completely free and you can just download them and use them on your own computer now what the Transformers package does is just make it extremely easy to access these free hugging face models and to download them and use them on your computer you'll see here that in just a few lines of code you can run pretty much any of these models assuming you have the correct hardware and if you wanted to go further you could even fine-tune these models and add a bunch of advanced configurations so we're going to be using Transformers and we're going to use this in combination with something called Lang chain now Lang chain is another python package that just makes it a lot easier to work with llms so if you wanted to use one of these Transformer models you can connect this with Lang chain and then add things like memory connect multiple models together and I'll show you the basics of that in this video so that's the information now we're going to go ahead and start getting all of this set up now first things first we're just going to make an account on hugging face this is because a lot of these models require that you accept some terms or their license agreement before you're able to actually download them and use them so go to hugging face I'll leave this link in the description you should be able to go up here somewhere in the top right hand corner and just make a new account once you make a new account we're going to go to access token we're just going to leave this page open and we'll come back to it in 1 second so to get started here we need to set up our environment to be able to execute these different python packages now this does take a second so just bear with me once the environment is configured then it's very easy to write the code now you'll notice that what I'm using here is pie charm now that's the IDE that I'm going to use for this video and I actually have a long-term partnership going with them so if you would like to use pie charm as well which I would definitely recommend then you can click the link down below and I can give you a three month free extended trial of the pie charm professional Edition there's a Community Edition which is completely free and then there's the professional Edition which obviously has a lot of other features and specifically if you're into data science you'll see there's a lot of features here that can help you out for example it has an integration with hugging face directly works very well with tensorflow pytorch cond Git jupyter notebooks databases all kinds of stuff and you'll see some of the great features in this video of py charm anyways you can use whatever IDE that you want but if you do want to check this out again I have the those links below okay so first things first we want to create that virtual environment so what I'm going to do is open up my terminal here and from my terminal I'm going to execute a command that will create this environment for me now you do need python installed in order for this to work and what the virtual environment will do is just create an isolated area where we can have the dependencies specifically for this project so in order to do that we're going to type python DM VV and then venv now I'm running on windows so I'm using the python command but if you're on or Linux you can use the Python 3 command in case you get any errors with this what this does is use VV to create a new virtual environment called venv or VV and then once this step is finished we're going to activate that virtual environment so you've seen here that in my editor I just opened a new folder I just called it HF tutorial and then you'll notice that a new directory is created called venv that's my virtual environment and the next step is to activate the virtual environment so that we're using it when we install various dependencies so to do this manually from the command line you're going to write the following command and this is if you're on Windows it's going to be slash the name of your virtual environment which should be then V if you followed this command SL scripts with a capital slash activate if you type this you should see that you get a prefix with the name of your virtual environment in your terminal now if you're on Mac or Linux the command will be different and it will be Source again do/ theame of your virtual environment slash and then this time it's going to be bin with a lowercase and then activate this will then activate the the virtual environment for you and if you're working inside of py charm and you want to use the correct interpreter you can press this button down here and then you can simply go to add new interpreter add local interpreter press on existing and then select The Interpreter that's in your current directory so you can see HF tutorial venv scripts python.exe press okay and then it will select that and you'll see that it pops up down here and now all of your autocomplete will work with those packages now when you want to run your python code you simply type Python and then the name of the file if you called it main.py and then you can hit enter and as long as you're in the virtual environment then all of this will work okay next step we're going to make a new file inside of our directory called requirements.txt now this is where we're going to put the various requirements that we need in order to work with the Transformers and Lang chain Library so inside of here we're going to type Transformers if we spell that correctly we're going to do Lang chain and we're going to add Lang chain Dash and then this is going to be hugging face which will allow us the integration with Lang chain okay so these are the three main packages that you need now that you have the requirements.txt file created from your virtual environment you're going to type the command pip install dasr and then requirements.txt this will read the requirements.txt file and install all of those in your virtual environment so I'm going to go ahead and press enter you can see these are all cashed because I've installed them previously and then they will get installed if you're on Mac or Linux again you can try the command pip three in case that one doesn't work for you okay so all of that's been installed and the next step here is to Simply get our hugging face token now we need the hugging face token because a lot of the models just require that you accept some license agreement they're free to use but you have to essentially check a box and that's connected to your hugging face account so if you try to pull certain models it will give you an error and that's because you haven't added the token so what we're going to do is from our user access tokens again you can just go to hugging face create a new account press on your profile press here on access tokens and simply create a new one so if I go create a new token I can just call this python 2 or something just give this a read mode just give it read and press on create token copy this token obviously don't leak it to anyone like I am right now and then what we're going to do is type the following we're going to say hugging face- CLI and then sorry we're not going to paste the token we're going to say login okay now this command will work as long as you've installed Transformers in your ual environment so I'm going to hit enter and you're going to notice that it simply ask for my token so I'm just going to paste the token inside of here and then hit enter and then I'm just going to say yes for my git credential when you paste this you're not going to see the token so just paste it and then hit enter to go to the next line okay so now the token has been added you can see the currently active token is the name of the token and now we'll be able to pull those models and kind of accept the terms and conditions okay so we're almost done here that is pretty much the environment setup now what we're going to do is make a new file so I'm going to go new file I'm going to call this main.py and I inside of here s we're going to write the code that will allow us to start using some Transformer models so just bear with me while I write some basic code and then we can explain it after okay so first thing we're going to do here is we're going to say from Transformers import and then we're going to import pipeline okay now pipeline is a simplified way of running various models if you want to do this in a more Advanced way there are ways to do that but I'm going to keep it simple for this video now after that I'm going to Define my model so I'm going to say model is equal to you can see I'm getting some autocomplete here in py charm uh and I'm actually not going to use that I'm just going to say pipeline and then I'm going to paste in here the following okay just bear with me summarization and then model is equal to the following so believe it or not this is actually as easy as it is to run a model inside of python from Transformers or from hugging phase what I'm doing is I'm using this thing called pipeline pipeline just automatically sets everything up for me and all I have to do is specify the task so there's various different tasks like summarization text classification text generation and then the model that I want to use for this task I'm going to show you how you get the models in just one second but this is one that we can use for summarizing text okay we specify the model and then all we have to do is say response is equal to model and then we pass in this case text to summarize and it it will literally just summarize the text for us using this machine learning model the first step is it's going to download this it's going to take a second to download once it's downloaded it will then just summarize it for us when we call this model function and then we can print out the response okay so believe it or not that's literally all you need to do now there's some more setup steps and things that I'll talk about in a second but for now I just want to run this code and make sure it works so manually from your terminal you can just go Python and then main.py assuming you're in your virtual environment but because I'm in py charm I'm just going to use the Run button here and it will just run it for me um in my terminal and let's see if we get any errors after running this okay so you can see this did indeed work and it gave me this summary text now it doesn't really make a lot of sense because I just passed something that doesn't make sense to the model like it was only three words so it didn't really know how to interpret this but then you can see that it said summary text and then it gave me kind of this random summary of something that's not really relevant to this if we had a larger text it would actually summarize it properly so don't worry about that this is just a quick example now for you this probably took a second to run because you needed to download the model First Once the model is downloaded it will be stored on your machine and then you can use it very quickly after that now you'll also notice that we're getting some kind of warnings I'll show you how to mute those in one second and you'll notice that it says that the device is using CPU now this is the main thing I want to focus on how do we get this to use your GPU because obviously if you have a GPU you want to use that and it's going to be hundreds if not thousands of times faster so let me show you how we do that and we'll talk talk about also how we get different hugging face models just bear with me there's a lot of stuff to cover okay so first let's look at how we can run these models using our GPU now in order to do that you are going to need an Nvidia GPU or at least that's what I'm going to recommend because that's what works the best now in order for your Nvidia GPU to be used for this you need to download and install Cuda now I'm just going to go to the Cuda website I'll link this in the description as well but you can simply type Cuda toolkit and then you're just going to download that okay so if you go Cuda toolkit on Google you're going to need to download this again it's only available for Linux or Windows and then once Cuda is downloaded and installed you'll be able to move on to the next step so again if you want this to work for your GPU download Cuda the link will be in the description okay once Cuda is downloaded what you're going to want to do is restart any terminal instance that you have so close your IDE reopen it and you're going to type the following command just to verify that it's installed you're going to type EnV cc-- version if you do this you should see some kind of output and it will tell you what version of coua that you have okay so in my case I have 12.6 you probably have 12.8 or something more recent if you just download it like today when you're watching this video so now the Cuda is installed we can move on to the next step which is going to be to install pytorch specifically for GPU so I'm going to copy in a command which again I will leave in the description and it's going to look like this uh sorry let me just get out of this because for some reason when I copied this it automatically ran which I don't want it to do but you can see that it says pip install torch torch Vision torch audio and then I specify the index URL and at the end of the index URL you'll see this number cu12 something now in your case if you're downloading one uh sorry 12.8 for Cuda you're going to change this to 128 in my case it was 126 so I'm going to download 126 okay what this is going to do is install pytorch 4 GPU for you in your virtual environment and then you'll be good to go in order to use this for your GPU so again this command will be in the description but simply hit enter here install this in your virtual environment and then you'll be good to go okay so that was successfully installed so I can go ahead and close the terminal and what we're going to do to verify that this is working is we're going to paste the following code in our Python program now all we need to do is import torch and then we're simply just going to check if our GPU is available and what the name of that GPU is so all we have to do is just print out torch. ca. isil and torch. ca. getet device name we're specifying index zero which should be the first GPU if you had multiple gpus then you would use various indexes to ask um to access them but this is what we're going to do to verify that we do have this installed properly so I'm going to run my code and we're just going to look at the top of the program and ensure that it says true and it says the name of our GPU which is doing right here now by default if this is installed this should start using the GPU so we'll say the device is set to Cuda zero that ref references that you are using the GPU at index zero but if you want to manually specify to use the GPU then what you can do is you can specifi device is equal to and then if you wanted CPU you could specify CPU if you wanted GPU you can simply put zero here or you can put in quote zero I don't believe it matters or one okay and then you're specifying hey I want to use device zero device Z is your GPU so you don't need the these lines of code now but that will verify that this is indeed working so now that we've got this running on GPU we want to learn about how to run various models cuz in the title I told you there are tons of free models that you can use so any model that you want to run you can get from hugging face now notice here in pie charm there's actually a hugging face button and this tells me all of the models that I currently have downloaded so you can see that there's a bunch of models that I have installed on my system everything from mistl I think I have a llama model these Facebook models and you might be wondering how the heck do you know which ones to get and how to use them well you can get that from the hugging face model directory okay now in pie charm funny enough it actually has a cool feature where you can right click here and click on insert HF model when you do this it will open up the models from hugging face and you can browse them directly in the IDE but in case you're not using pycharm I want to show you how to do this more manually so if you go to your browser and you go to hugging face and we just go back to the main website here what you're able to do is press on models let me just wait for this thing to go away so we can see it okay so if you click on models from here you can specify or kind of search the models based on which ones work with the Transformers Library so if you go up to the top here and you press on libraries and then you select Transformers because that's what we're using here and then you go back to tasks now you're going to be able to specify by task or sort by task while also looking at the ones that work with Transformers so anything that you want probably exists here in hugging face for example you have object detection text to image text classification and the easiest ones to get started with are the natural language processing ones so for example if I'm looking for text summarization I click on summarization and then I can sort this by trending or by for example most downloads and I can view these models so if I look at this one which I believe is the one that I'm already using you'll see here that we can scroll through it gives us an example of exactly how to use this from our code and you can also press this button right here that says use the model click on Transformers and it shows you the code and you can read the documentation directly here so if you're wondering how do you get the models that's how you get them now I just want to show you that if I click into a random model here let me find some other one it's possible that at some points some of these are gated now I'm having trouble finding a gated one right now but sometimes you will need to go directly to the model website here and you'll need to press a button that says like accept license agreement so if you get an error when you're trying to use these models it's possible that there is a license agreement that you need to accept that you haven't yet accepted so you would just need to go to the model page and then press accept when you're signed in with your account that's it okay in my case I'm going to keep using the summarization one for right now but you can use any model that you want and again you can filter them here and if you find one you want to use just check if it has that license agreement and just check that before you um start trying to download it so you don't get any errors okay so at this point I'm going to assume that you've got this running properly on your computer and you understand that if you want to pick a model you go to hugging face and you can read through all the models that are there and you can pretty much just put them in the pipeline now will almost always tell you how to use the model from hugging face so you can just directly read the documentation and you can see what task and what model to be using now what I'm going to do is just paste in a simple example here where we now go and we connect this model to Lang chain which allows us to make this a little bit more advanced and use it in a more real world application so what I'm going to do is just paste in this example it's not very long it's just 20 lines of code and quickly walk through it so you can see that what I've done here is I've imported hugging face pipeline from Lang chain hugging face this allows me to wrap a hugging face model as a lang chain model which then lets me connect it in a lang chain chain if you're unfamiliar with Lang chain I actually have a tutorial on it I'll put it up on screen right now but the point is it allows you to use multiple models together and make more advanced applications and just deal with llms now for this pipeline I'm using a heex generation Pipeline with mistol mistol is a larger model you require 16 GB of vram to use this so I'm just giving you an example and I've specified the maximum length and that it's okay to truncate the response I've then created my llm by wrapping my model in this hugging face pipeline then what I've done let me just close this here is I've created a prompt template where I've specified some variables that I want to embed inside of here so I want to explain some Topic in detail for a blank or agey old to understand then what I've done is I've created a lang chain chain this chain takes the template and essentially fills it in with the correct variables and then passes it to my llm then I ask the user for a topic and some age and I execute the chain so I say response is equal to chain which I've created right here do invoke and then the variables that I want to pass here so topic and age I put inside of a python dictionary so if I now go and run this code let me just open this up from the bottom of my screen it will take a second and then I should be asked to enter the topic and the age and then in that case I'm able to continue and get the output okay so for the topic I'm going to go with Quantum Computing I don't know if that's spelled correctly but that's fine and then for the age I'll go with maybe 10 or something okay and then we're just going to wait one second if you get this output or these errors don't worry I'm going to show you how you can mute them in just one second but they're just giving you some logs on what's going on so this can take a second to generate because this is a a very large model and it is utilizing my GPU so once that's finished I'll be right back and you'll see the response okay so we just got the response to be honest with you I don't think a 10-year-old will be able to understand this but whatever it gave me the reply and you can see it's showing up now in my terminal okay so that's one quick example I will leave this code available via GitHub in the description in case you want to check it out but what I want to do now is just copy in a more advanced example and show you another cool thing that you could build using Transformers okay so first things first I want to direct your attention to the following two lines of code on line four and line six if you don't want to see those red error messages or the warnings pop up then you can use this okay so this is simply importing the set verbosity error verbosity is like how much you want to show how much detail you want to see and when you call this function it will toggle that so it will essentially turn it off so you're not going to see those error messages popping up okay so just keep that in mind that's probably something important that you want to know this code will be available from the description again and what I've set up here is actually kind of a more advanced pipeline using Lang chain that uses three different models from hugging face so I have one that does summarization I have another one that does a refinement of the summarization and then I have one that does question answering the question answering is quite cool it takes in a bunch of context and then what it will do is answer questions based on that so my idea here is that I'm going to have some large text that I want to summarize and then afterwards I'm going to allow the user to ask questions about that summary so I've set up a summary template where I said I want to summarize the following text in a certain amount of length so a certain amount of characters okay and then what I do is I create a summarization chain where I essentially take the template I pass it to my summarizer which is a hugging face Pipeline and then after that I take that output and I pass that to my refiner which is another model so I'm using the chain to kind of pass execution between different models I then get some text that I want to summarize I get the length so short medium or long and then what I do is I invoke my Chain by passing the text to summarize and the length after that's finished I print out the summary and then I just go into a while loop where I ask the user if they want to ask any questions anytime they ask a question I pass that to my QA pipeline where I pass in the question and the context which is simply the summary and then I print out the answer okay so again I'll leave this code in the description but let's run this and just see how it works and I'm just going to grab some large piece of text here about AI that I will kind of pass into this in one sec when it starts running okay so it asked me for the text to summarize so let's paste that this is just a bunch of information about Ai and then I'm going to say enter the length let's go with medium length Okay let's give this a second to generate the summary so it's given us uh the summary there and then we can ask a question so I'm going to say where is AI used and then you can see Netflix and Amazon I'm going to say how you know important is AI uh one of the most significant technology advancements in the 21st century and it's pulling that all out of this context so I'm just going to type exit now to get out of that and then we'll quit the program and there you go that is the tutorial on how to use Transformers and Lang chain to really use any model you want completely for free locally on your out computer I know that was a lot of information but I wanted to make sure that this was valuable to you if you like the video make sure to leave a like subscribe to the channel and I will see you in another one [Music]

Up Next

Whisper Benchmark: GPU vs API Transcription Speed Analysis

@bhattbhavesh91

1.7K views•2023-10-09

IFS Therapy Demonstration: Complete Session with Unburdening

@IFSCA

95.9K views•2021-01-13

Python Pandas Tutorial: Data Analysis Fundamentals in 30 Minutes

@TechWithTim

155.1K views•2025-08-06

Game of Thrones Opening Credits: A Cinematic Analysis

@gameofthrones

46.3M views•2011-04-18

Related Study Plans & Knowledge Roadmaps

Structured learning paths in General & Interdisciplinary Studies

Browse General & Interdisciplinary Study Plans

general-studies

Containerization: Evolution, Logistics & Trade

01Ancient Trade & Bulk Cargo: Amphorae to Barrels

02The Industrial Revolution & Cardboard Packaging

03The Containerization Revolution: Malcom McLean

+ 2 more learning modules

Explore Roadmap

general-studies

Textile Tech: Evolution, Labor & Global Impact

01Pre-Industrial Textiles: Handloom & Cottage Industry

02Mechanization: The Spinning Jenny to the Power Loom

03The Factory System & Global Labor Transformation

+ 2 more learning modules

Explore Roadmap

general-studies

Pharmacology: Botany, Patents & Biopiracy

01Foundations of Ethnobotany & Traditional Medicine

02The Birth of Synthetic Chemistry: Aspirin & Penicillin

03The Battle Against Malaria: Quinine to Artemisinin

+ 3 more learning modules

Explore Roadmap

general-studies

Writing Surfaces: Papyrus, Parchment & Paper

01Ancient Foundations: Clay, Stone, and Egyptian Papyrus

02Parchment, Vellum, and the Rise of the Codex

03The Chinese Rag Paper Revolution and the Silk Road

+ 3 more learning modules

Explore Roadmap