Academic Databases
Explore Our Semantic Databases
STEM & Tech
Math, Physics, CS & Engineering...
Economics
Macro, Micro, Econometrics...
Business
Management, Marketing, Finance...
Health & Life Sciences
Biology, Medicine, Neuroscience...
Humanities & Social Sciences
Psychology, Law, Philosophy, History...
General Studies
General & interdisciplinary studies...
Explore Topics
Features

Decision Trees and Classification Trees Explained (Machine Learning Basics)

Added: 2026-08-07

Deep Dive

Basic concepts of supervised machine learning, particularly the difference between classification and regression tasks.

AI Basics for Beginners

AI Basics for Beginners

593.3K

Supervised machine learning requires labeled data where each input has a corresponding output label. The two primary task categories are classification and regression. In classification, the goal is to map input to one of several discrete categories—for example, classifying emails as spam or non-spam, identifying images as cats or dogs, or determining if a car is damaged. Binary classification involves two output categories, while multiclass classification handles multiple categories (like categorizing news articles into business, sports, technology, or health). In regression, the output is a continuous numerical value rather than a fixed category—for instance, predicting home prices based on features like area, bedrooms, and age.

Click to watch →

O que é classificação e regressão em machine learning?

O que é classificação e regressão em machine learning?

10.6K

Supervised learning problems are divided into classification and regression. Classification predicts discrete categories or classes (like spam vs. not spam, or fraudulent vs. legitimate transactions). The machine assigns each input to one of predefined categories. Regression predicts continuous numerical values (like property prices, stock values, or product demand). The key difference is that classification outputs are categorical labels while regression outputs are quantities. Choosing the right approach depends on whether the problem requires categorizing data or predicting numerical values.

Click to watch →

Sesión 3 IA

Sesión 3 IA

23.4K

Supervised learning has two categories: (1) Classification - predicts discrete labels (fraud/no fraud, risk levels); (2) Regression - predicts continuous values (final grades, blood pressure). Applications include predictive maintenance, fraud detection, and student performance prediction. The choice depends on whether the outcome is categorical or numerical. Many supervised learning models originated as unsupervised models before being labeled.

Click to watch →

Deep Learning Crash Course for Beginners

Deep Learning Crash Course for Beginners

1.1M

Supervised learning is the most common machine learning paradigm where models learn from labeled training data consisting of input-output pairs. Classification predicts discrete class labels (e.g., spam/not spam, handwritten digit recognition), while regression predicts continuous numerical values (e.g., house prices, student test scores). Linear regression models relationships using best-fit lines or hyperplanes. Popular classification algorithms include linear classifiers, support vector machines, decision trees, k-nearest neighbors, and random forests. Each algorithm is suited to different problem characteristics and data distributions.

Click to watch →

2. Supervised Learning- Machine Learning

2. Supervised Learning- Machine Learning

105.3K

Supervised learning is a machine learning approach where a computer learns patterns from labeled data (input-output pairs) to make predictions on new, unseen data; this category is divided into two main types: classification problems, where the output is a categorical class membership (e.g., spam/ham email classification), and regression problems, where the output is a continuous real value (e.g., predicting height or weight).

Click to watch →

Deep Dive

Fundamental understanding of tabular datasets, including features (independent variables) and target labels (dependent variables).

S1-L2-nutshell-Module2: ML by Data or Task (UVa CS 4774 Machine Learning Course)

S1-L2-nutshell-Module2: ML by Data or Task (UVa CS 4774 Machine Learning Course)

343

Tabular data is the most common input format for machine learning libraries like scikit-learn, organized as matrices where rows represent samples and columns represent features. The data is split into X (features) and Y (target) matrices. Tabular data can contain real numbers or symbolic values for both features and targets. In genomics, data often has high dimensionality with many features (genes, 20,000-30,000) but few samples (patients, 100-200). Similarly, text data has high dimensionality with large vocabularies (hundreds of thousands to millions of words) and large sample sizes from internet data.

Click to watch →

What is Tabular Data

What is Tabular Data

8.1K

Tabular data is a structured format organized into rows and columns (also called flat data), commonly sourced from CSV files, Excel spreadsheets, or joined database tables; in this context, a 'data point' refers to a single row/record, 'features' are the column properties describing each data point that serve as model inputs, and the 'label' is the target variable being predicted by the machine learning model.

Click to watch →

Занятие 20. Машинное обучение на табличных данных

Занятие 20. Машинное обучение на табличных данных

165

This extensive section provides a comprehensive overview of machine learning on tabular data. The three main tasks are: (1) Regression - predicting continuous values using linear regression (minimizing MSE) or regression trees (minimizing variance within leaves); (2) Classification - predicting discrete labels using cross-entropy loss or tree-based approaches; (3) Ranking - sorting candidates for recommendation systems. Key challenges include: (1) Large-scale target variables requiring transformations like log or square root; (2) Outliers requiring careful analysis to determine if they represent meaningful patterns or measurement errors; (3) Overfitting (high train, low test quality) requiring regularization; (4) Underfitting (low quality everywhere) requiring better pattern capture. Data preprocessing strategies address three main categories: (1) Outliers - handled through transformations or removal; (2) Missing values - imputed with zero, mean, median, or predicted from other features; (3) Inconsistent categorical data - resolved through TF-IDF vectorization followed by clustering to identify patterns, then deriving regex rules. For text features, the standard pipeline involves TF-IDF vectorization, dimensionality reduction (SVD), and clustering to group similar samples. The section emphasizes that 70-80% coverage of values is sufficient for effective preprocessing and that regular expressions can be learned through practice and debugging tools.

Click to watch →

Manipulação de Dados em Python/Pandas - #01 Conceitos Básicos

Manipulação de Dados em Python/Pandas - #01 Conceitos Básicos

28.1K

Variables in a dataset can be classified as independent or dependent. Independent variables (also called features, attributes, or input variables) exist without depending on other variables in the dataset. Dependent variables (also called target, output, or response variables) have dependencies on at least some of the independent variables. For example, in a fuel price dataset, state and unit of measure are independent variables, while the average resale price is a dependent variable because it is influenced by the state and other factors.

Click to watch →

Day 1 | Introduction to Machine Learning & Model Building

Day 1 | Introduction to Machine Learning & Model Building

5.5K

A dataset is a structured collection of data organized in tabular format, similar to an Excel sheet. Rows represent individual samples or observations (e.g., each flower in the Iris dataset). Columns represent features or variables (input variables X) and labels/targets (output variable Y). Features are the measurable properties used for prediction (e.g., sepal length, sepal width, petal length, petal width in the Iris dataset). Labels are the target variable being predicted (e.g., species type).

Click to watch →

Deep Dive

Elementary probability and statistics, specifically how to calculate proportions and understand ratio-based metrics.

Century Series - Probability | NDA 1 2023 MATHS CLASSES | YODHA BATCH | ARPIT SIR

Century Series - Probability | NDA 1 2023 MATHS CLASSES | YODHA BATCH | ARPIT SIR

37.2K

This segment covers conditional probability with gender using the formula P(A|B) = P(A ∩ B) / P(B), demonstrating with smoking statistics (70% of men smoke, 30% of women smoke). The segment also covers ratio-based probability calculations, showing how to determine probabilities from given ratios (e.g., 2:3 ratio means 2/5 probability). The instructor demonstrates counting favorable outcomes (numbers divisible by 6 from 1-100: 16 numbers) and dividing by total outcomes (100). The segment also covers probability of getting specific numbers on dice and sequential match winning scenarios.

Click to watch →

[数B][統計#18] 母比率、標本比率とは何か？ [統計的な推測]

[数B][統計#18] 母比率、標本比率とは何か？ [統計的な推測]

97K

This lesson covers the fundamental concepts of statistical estimation for proportions. Population proportion (母率) represents the true proportion of a characteristic in the entire population, while sample proportion (標本比率) is the proportion observed in a sample. When sampling n items from a population with proportion p, the count follows a binomial distribution with expected value np and variance np(1-p). The sample proportion R = x/n has expected value p and variance p(1-p)/n, making it an unbiased estimator. For large sample sizes, R approximates a normal distribution, enabling probability calculations using standard normal tables. The practical application demonstrates calculating the probability that a sample proportion falls within a specified range, such as finding that with p=0.1, n=400, and range [0.07, 0.13], the probability is approximately 95.44%.

Click to watch →

SBI PO 2026 Quants Rapid Revision Marathon | Complete Quantitative Aptitude Class

SBI PO 2026 Quants Rapid Revision Marathon | Complete Quantitative Aptitude Class

14.4K

A comprehensive method for solving complex DI questions: When given total shirts manufactured by companies R and T (41% = 820), and white shirts by R and T (270), find black shirts. Total black shirts = 820 - 270 = 550. Given black shirts ratio between R and T is 3:1, and total shirts for R is 620, black shirts for R = 400. White shirts for R = 620 - 400 = 220. For ratio-based problems, given white shirts ratio between companies S and A is 5:3, and white shirts for company A is 260, find total shirts for company A. If 5 units = 260, then 1 unit = 52. Total units = 8, so total shirts = 8 × 52 = 416.

Click to watch →

Data Interpretation | 200 Prelims Level DI Set - 7 | IBPS | SBI | RRB PO/CLERK 2024 | Arun Sir

Data Interpretation | 200 Prelims Level DI Set - 7 | IBPS | SBI | RRB PO/CLERK 2024 | Arun Sir

12.9K

Ratio-based calculations are essential in DI questions. The instructor demonstrates: (1) If total adult shoes in shop E is 180 and ratio of Reebok to Adidas is 4:5, then Reebok = (4/9) × 180 = 80 and Adidas = (5/9) × 180 = 100; (2) For kids shoes with ratio 2:3, if total is 60, then first brand = (2/5) × 60 = 24 and second brand = (3/5) × 60 = 36; (3) For percentage calculations, find total first, then apply percentage. The instructor emphasizes that these questions are not difficult but require practice to handle larger statements efficiently.

Click to watch →

OLASILIK Yeni Nesil Soru Çözümü | LGS 2025 | King Serisi #4

OLASILIK Yeni Nesil Soru Çözümü | LGS 2025 | King Serisi #4

28.3K

This segment demonstrates solving probability problems using ratios and integer constraints. Given blue:green probability ratio of 3:4 and orange probability of 15/64, you express blue as 3k and green as 4k. The total blue+green = 49, so 7k = 49, giving k = 7. Thus blue = 21. The critical constraint is that item counts must be integers, so you verify that solutions yield whole numbers. This method combines ratio reasoning with algebraic solving and integer validation to find exact counts from probability information.

Click to watch →

Deep Dive

An introductory grasp of model evaluation concepts, such as the distinction between training and testing datasets.

What is the difference between Training dataset and Test dataset?

What is the difference between Training dataset and Test dataset?

18.8K

In machine learning, the training dataset is used to build and tune the model by discovering predictive relationships and learning data behavior, while the test dataset is used to impartially evaluate the model's accuracy on unseen data; keeping these datasets separate prevents overfitting and ensures the model generalizes well to new data.

Click to watch →

A Gentle Introduction to Machine Learning

A Gentle Introduction to Machine Learning

1.4M

Machine learning relies on two distinct datasets: training data and testing data. Training data is used to fit models (like fitting a black line or green squiggle to data points). Testing data consists of new observations not seen during training. The critical process involves using testing data to evaluate how well models make predictions. By measuring distances between actual and predicted values across all testing samples, we calculate error metrics. The model with the smallest total error is preferred, even if it fits training data less perfectly. This separation prevents overfitting and ensures models generalize to real-world scenarios.

Click to watch →

ML 2 : LearnTraining VS Testing Dataset with Examples #machinelearning

ML 2 : LearnTraining VS Testing Dataset with Examples #machinelearning

92.7K

In machine learning, the training dataset contains labeled examples (attributes and class labels) used to teach a model patterns and relationships, while the testing dataset contains unlabeled examples used to evaluate the model's ability to make accurate predictions on new, unseen data; the training phase uses supervised learning with labeled inputs to build the model, and the testing phase assesses model accuracy by comparing predicted outputs against actual labels.

Click to watch →

Интенсив Специализации Machine Learning // занятие курса «Специализация Machine Learning» 1 День

Интенсив Специализации Machine Learning // занятие курса «Специализация Machine Learning» 1 День

989

In machine learning, each data point is called an object or example, characterized by a set of features (attributes). The target variable (label) is what we want to predict. A dataset is the complete collection of objects with features and labels, divided into training data (used to teach the model) and test data (used to evaluate performance). Model evaluation uses metrics like accuracy or mean squared error. The model must be tested on unseen data to ensure it can generalize to new situations. If trained only on data from one region, it may not perform well on data from other regions.

Click to watch →

Statistics for Data Science | Data Science tutorial | Probability and Statistics | Great Learning

Statistics for Data Science | Data Science tutorial | Probability and Statistics | Great Learning

6K

In machine learning, data is split into training and testing sets. The training set teaches the model, while the testing set evaluates performance—similar to studying from exam materials and then taking a separate exam. A confusion matrix shows True Positive, True Negative, False Positive, and False Negative predictions. Precision measures how many predicted positive cases were correct. Recall measures how many actual positive cases were correctly identified. These metrics determine model reliability and help assess performance.

Click to watch →

Tree Basics

Tree Structure

Feature Splitting

Gini Index

Impurity Calc

Numeric Splits

Root Selection

Tree Refinement

Overfitting Fix

Tree Basics

0:00

Playing Section

1
Explains decision tree fundamentals and classification/regression distinction.
2
Introduces data type mixing and varied numeric thresholds.
3
Describes tree traversal from root to leaf for predictions.

Deep Dive

Ensemble learning methods, specifically Random Forests, which combine multiple decision trees to reduce variance and improve accuracy.

CH11 - Machine Learning (ML) - Decision Trees and Random Forest algorithm

CH11 - Machine Learning (ML) - Decision Trees and Random Forest algorithm

239

Ensemble learning combines multiple models to improve performance. Random Forest implements this by creating multiple decision trees through bagging: each tree trains on random data subsets with replacement, and each node considers only random feature subsets. This diversity reduces variance and improves generalization. Final predictions aggregate individual tree outputs through voting (classification) or averaging (regression). Advantages include reduced overfitting, robustness to noise, and automatic feature importance estimation. Disadvantages include higher computational cost and reduced interpretability compared to single trees.

Click to watch →

Машинное обучение 5. Деревья решений

Машинное обучение 5. Деревья решений

798

Random Forest is an ensemble method that combines multiple decision trees to reduce variance. It uses bootstrap sampling (sampling with replacement) to create diverse training datasets for each tree. Additionally, random subspace methods randomly select subsets of features for each tree, further increasing diversity. The final prediction is obtained by averaging (regression) or voting (classification) across all trees. This ensemble approach significantly reduces variance while maintaining low bias, making Random Forest a robust baseline model for tabular data.

Click to watch →

Machine Learning 05 | Decision Tree & Random Forest | DA | GATE Crash Course

Machine Learning 05 | Decision Tree & Random Forest | DA | GATE Crash Course

3.3K

Random Forest is an ensemble method combining multiple decision trees (base learners) to improve accuracy and robustness. Also called bagging or bootstrap aggregation, it creates multiple decision trees from bootstrap samples (with replacement) of the training data. Each tree votes independently, and final predictions use majority voting for classification or mean value for regression. This approach leverages the wisdom of crowds, where diverse trees collectively outperform single trees. The technique reduces variance and prevents overfitting through averaging.

Click to watch →

Machine Learning algorithms explained, with Vadim Smolyakov - HS#18

Machine Learning algorithms explained, with Vadim Smolyakov - HS#18

158

Random Forests are ensemble methods that combine multiple decision trees to improve predictive performance and reduce variance. The algorithm introduces randomness in two ways: by sampling features at each split and by training on bootstrap samples of the data. This decorrelation of individual trees is crucial because it allows the ensemble to minimize variance through averaging. In regression settings, Random Forests minimize mean squared error, which can be decomposed into bias squared plus variance. By averaging many decorrelated trees, Random Forests achieve a good balance between bias and variance, making them more robust than individual decision trees.

Click to watch →

Random Forests

Random Forests

660

Random Forests is an ensemble learning method that combines multiple decision trees to improve predictive accuracy and reduce variance; it works by creating multiple bootstrap samples (with replacement) from the training data, training a separate decision tree on each sample, and then aggregating predictions through majority voting, where each tree's vote contributes to the final classification decision.

Click to watch →

Deep Dive

Advanced gradient boosting algorithms such as XGBoost, LightGBM, and CatBoost.

771: Gradient Boosting: XGBoost, LightGBM and CatBoost — with Kirill Eremenko

771: Gradient Boosting: XGBoost, LightGBM and CatBoost — with Kirill Eremenko

10.4K

Gradient boosting is a powerful ensemble technique that iteratively combines weak learners (typically decision trees) to improve model performance by focusing on errors from previous iterations; XGBoost (2014) optimized gradient boosting with histogram-based splits and regularization, LightGBM (2017) achieved 20x faster training through exclusive feature bundling and leaf-wise growth, and CatBoost (2017) excels at handling categorical features automatically using ordered target encoding, making these algorithms the go-to solutions for tabular data problems in industry and business applications.

Click to watch →

Lec 53 Supervised Machine Learning Tutorial with Python (Tutorial II)

Lec 53 Supervised Machine Learning Tutorial with Python (Tutorial II)

311

XGBoost, LightGBM, and CatBoost are optimized gradient boosting implementations. XGBoost uses histograms for splits and supports pruning and missing values. LightGBM uses histogram-based learning and leaf-wise tree growth for faster convergence. CatBoost handles categorical features natively and uses ordered boosting. These libraries typically outperform scikit-learn's vanilla gradient boosting.

Click to watch →

Методы и алгоритмы машинного обучения 12. Градиентный бустинг II

Методы и алгоритмы машинного обучения 12. Градиентный бустинг II

306

This lecture explores advanced gradient boosting algorithms beyond the basic gradient boosting framework. XGBoost introduces second-order optimization using Hessians (Hessian-based Newton method) and regularization terms for tree complexity control. LightGBM employs leaf-wise tree growth (instead of level-wise), gradient-based sampling for efficient training, and histogram-based split finding. CatBoost specializes in handling categorical features through target encoding and progressive sampling strategies. These libraries represent the state-of-the-art in gradient boosting for tabular data, with applications in recommendation systems, search engines, and various machine learning tasks.

Click to watch →

PART-7 // AI Engineering Masterclass 2026 🚀 | From Zero to AI Hero with Real-World Projects

PART-7 // AI Engineering Masterclass 2026 🚀 | From Zero to AI Hero with Real-World Projects

212

Boosting is an ensemble technique that sequentially combines weak learners to form a strong learner, with each model focusing on correcting errors from previous models. Gradient boosting builds models sequentially by minimizing a loss function using gradient descent, with key parameters including learning rate (typically 0.01-0.3), number of estimators, and tree depth. XGBoost (Extreme Gradient Boosting) is an advanced implementation designed for speed and performance, featuring parallel processing, automatic handling of missing data, L1/L2 regularization, and tree pruning. Essential hyperparameters include learning rate (eta), n_estimators, max_depth, subsample, colsample_bytree, lambda (L2 regularization), and alpha (L1 regularization). LightGBM (Light Gradient Boosting Machine) is an efficient implementation for large datasets featuring histogram-based splitting (discretizes continuous features, reducing memory usage), leafwise tree growth (adds splits to highest-loss leaves for efficiency), and GPU support for accelerated training. CatBoost handles categorical features natively without manual encoding, using statistical encoding that preserves order and relationships, with order boosting reducing prediction bias through permutation-based training. Comparison: XGBoost is moderate speed with moderate memory, LightGBM is fast with low memory, CatBoost is fast with moderate memory. Best use cases: XGBoost for general purpose models, LightGBM for large datasets, CatBoost for categorically heavy datasets.

Click to watch →

Module 10- Theory 3: Advanced ML boosting techniques: XGboost, Catboost, LightGBM

Module 10- Theory 3: Advanced ML boosting techniques: XGboost, Catboost, LightGBM

1.9K

This extensive section provides a detailed comparison of three leading gradient boosting libraries. XGBoost (2014, University of Washington) emphasizes speed and scalability through regularization, cache-aware computation, and depth-first pruning with built-in cross-validation. LightGBM (2016, Microsoft) uses leaf-wise growth, handles categorical features automatically, and supports multi-GPU acceleration for large-scale learning. CatBoost (2017, Yandex) excels at categorical variables without preprocessing, making it ideal for datasets with many text-based categories. Key differences include tree growth strategies (depthwise vs leafwise vs symmetric), parallelization capabilities, categorical feature handling, and splitting methods (histogram-based vs gradient-based vs greedy). The instructor advises choosing CatBoost for many categorical features, LightGBM for large-scale data, while noting all perform competitively on most tasks.

Click to watch →

Deep Dive

Alternative split evaluation metrics, such as Information Gain (Entropy) for classification and Mean Squared Error (MSE) for regression trees.

Module 3- Part 2- ML boosting algorithms XGBoost, CatBoost and LightGBM

Module 3- Part 2- ML boosting algorithms XGBoost, CatBoost and LightGBM

11.1K

Decision trees select features and cutoffs based on information gain, which measures how much purity is achieved at terminal nodes. For regression trees, Mean Squared Error (MSE) is minimized. For classification, three metrics are used: error rate, entropy, and Gini index. Entropy and Gini index are preferred over error rate because they are more sensitive to changes in probabilities. While entropy is computationally slightly slower than Gini index, the difference is negligible for most structured datasets, and either can be selected through grid search during model tuning.

Click to watch →

Gini Index and Entropy|Gini Index and Information gain in Decision Tree|Decision tree splitting rule

Gini Index and Entropy|Gini Index and Information gain in Decision Tree|Decision tree splitting rule

175.8K

Entropy is an alternative splitting criterion to Gini Index, commonly known as information gain. The formula for entropy is: Entropy = -Σ(probability of class j × log₂(probability of class j)), where the summation is over all classes. Like Gini, entropy measures impurity, but uses logarithms in its calculation. The split with the lowest entropy (highest information gain) is selected for tree splitting.

Click to watch →

Summer School 2024, Session - 5: Decision trees and ensemble methods

Summer School 2024, Session - 5: Decision trees and ensemble methods

810

Entropy measures disorder using the formula: Entropy = -Σ(Pi × log2(Pi)). Pure nodes have entropy = 0, while maximally impure nodes have maximum entropy. Information Gain measures entropy reduction: IG = Entropy(parent) - Weighted Average Entropy(children). The split with highest Information Gain is chosen. For example, with parent entropy 0.971 and children entropies 0.811 and 1.0 (weights 0.4 and 0.6), IG = 0.971 - 0.924 = 0.047. Information Gain and Gini impurity both measure impurity but use different mathematical approaches. Both can be used for decision tree construction, with Information Gain being more intuitive as it directly measures information reduction.

Click to watch →

Методы и алгоритмы машинного обучения 9. Решающие деревья

Методы и алгоритмы машинного обучения 9. Решающие деревья

297

For regression trees using mean squared error, the optimal leaf prediction is the mean of target values in that node. The information gain formula involves calculating the weighted average of MSE for left and right child nodes, with the gain being the difference between parent MSE and this weighted average. For classification trees using misclassification error, the optimal leaf prediction is the most frequent class. The information gain is calculated as 1 minus the proportion of the most frequent class (1 - p*). These criteria measure how much a split reduces uncertainty about the target variable.

Click to watch →

Decision Trees - Split Criteria

Decision Trees - Split Criteria

1.2K

Regression trees use squared error (L2) and absolute error (L1) to measure dispersion. Squared error minimizes at the mean, so terminal nodes use the mean of Y values; absolute error minimizes at the median, so terminal nodes use the median. Deviance generalizes these concepts to various distributions, equivalent to entropy for classification and squared loss for regression under normal assumptions. Information gain quantifies split improvement by comparing pre-split impurity to post-split weighted impurity. The best split maximizes information gain through brute-force search across all features and cutoff values.

Click to watch →

Deep Dive

Practical hyperparameter tuning techniques (e.g., setting maximum depth or minimum split samples) using cross-validation.

MLP End Term Revision

MLP End Term Revision

944

Hyperparameter tuning methods include Grid Search CV and Randomized Search CV with parameters: estimator (model to tune), param_grid (dictionary of parameter combinations), scoring (evaluation metric - accuracy for classification, R2 for regression), cv (cross-validation folds), refit (refit on best parameters), verbose, n_jobs (parallel processing). The number of models trained equals parameter combinations multiplied by CV folds. For example, with 9 parameter combinations and 5-fold CV, 45 models are trained. Key model parameters: Decision Tree (max_depth, min_samples_split, min_samples_leaf, ccp_alpha), KNN (n_neighbors, p for distance metric - p=1 Manhattan, p=2 Euclidean), Random Forest (n_estimators, criterion (gini, entropy, log_loss), max_depth, min_samples_split, min_samples_leaf, max_features for randomization).

Click to watch →

Tutorial: ÁRBOLES DE CLASIFICACIÓN en Python

Tutorial: ÁRBOLES DE CLASIFICACIÓN en Python

2.7K

This section covers advanced techniques for improving model reliability and deployment. Cross-validation divides data into K subsets (K=5), training K models and averaging performance scores for robust generalization estimates. Hyperparameter tuning controls tree complexity through max_depth, min_samples_split, and min_samples_leaf. Reducing max_depth from 14 to 5 decreased complexity from 110 to 22 leaves while improving performance. Generating predictions uses the predict() method on new data. The tutorial emphasizes that while training is simple, controlling hyperparameters prevents overfitting and improves generalization. Practical deployment involves applying trained models to unseen data for classification decisions.

Click to watch →

Hyperparameter Optimization: This Tutorial Is All You Need

Hyperparameter Optimization: This Tutorial Is All You Need

114.3K

All hyperparameter optimization should be performed within a cross-validation loop to prevent overfitting to the training data. Cross-validation involves splitting the dataset into multiple folds, training the model on some folds while validating on others, and averaging the results across all folds. This ensures that the hyperparameters selected generalize well to unseen data rather than just performing well on the specific training split used during optimization.

Click to watch →

Live Day 4- Discussing Decision Tree And Ensemble Machine Learning Algorithms

Live Day 4- Discussing Decision Tree And Ensemble Machine Learning Algorithms

170.7K

This segment focuses on hyperparameter tuning for decision trees to optimize model performance. The instructor explains key hyperparameters including the maximum depth of the tree, the minimum number of samples required to split a node, and the minimum number of samples required to be at a leaf node. The content covers how to use cross-validation to find the optimal hyperparameter values that balance model complexity and generalization. The segment discusses the trade-off between model complexity and overfitting, explaining that deeper trees can capture more patterns but may also capture noise in the training data. The instructor demonstrates practical approaches to hyperparameter tuning, including grid search and random search methods.

Click to watch →

Machine Learning: Testing and Error Metrics

Machine Learning: Testing and Error Metrics

116.2K

Cross-validation solves the problem of using testing data for hyperparameter selection. The data is split into three parts: training set (for training models), cross-validation set (for selecting hyperparameters), and testing set (final evaluation). Multiple models with different hyperparameters (e.g., polynomial degrees 1-4) are trained on the training set. Their performance (e.g., F1 score) is evaluated on the cross-validation set to select the best hyperparameters. Finally, the selected model is tested on the untouched testing set to ensure generalization.

Click to watch →

Tree Basics

Tree Structure

Feature Splitting

Gini Index

Impurity Calc

Numeric Splits

Root Selection

Tree Refinement

Overfitting Fix

Tree Basics

0:00

Playing Section

1
Explains decision tree fundamentals and classification/regression distinction.
2
Introduces data type mixing and varied numeric thresholds.
3
Describes tree traversal from root to leaf for predictions.

1.1M views22.9K18:08@statquestOriginal Release: 2021-04-26

Decision trees are machine learning models that make decisions by recursively splitting data based on feature values; classification trees predict categorical outcomes by evaluating impurity measures like Gini impurity at each split, where lower impurity indicates better separation of classes, and the tree is built by selecting the feature that maximizes information gain at each node until reaching pure leaves with majority class assignments.

[00:00:00]i like decision trees how about you stat quest hello i'm josh darmer and welcome to statquest today we're going to talk about decision and classification trees and they're going to be clearly explained here is a simple decision tree if a person wants to learn about decision trees then they should watch this stat quest in contrast if a person does not want to learn about decision trees then check out the latest justin bieber video instead in general a decision tree makes a statement and then makes a decision based on whether or not that statement is true or false it's no big deal when a decision tree classifies things into categories it's called a classification tree and when a decision tree predicts numeric values it's called a regression tree in this case we're using diet to predict a numeric value for mouse size note for the remainder of this video we are going to focus on classification trees however if you want to learn more about regression trees fear not there's a whole stat quest dedicated to regression trees the link is in the description below now here's a more complicated classification tree it combines numeric data with yes no data so it's okay to mix data types in the same tree also notice that the tree asks about exercising multiple times and that the amount of time exercising isn't always the same so numeric thresholds can be different for the same data lastly the final classifications can be repeated for the most part classification trees are pretty easy to work with you start at the top and work your way down and down until you get to a point where you can't go any further and that's how you'll classify something note so far i've been labeling the arrows with true or false but usually it is just assumed that if a statement is true you go to the left and if a statement is false you go to the right so sometimes you see true and false labels sometimes you don't it's no big deal oh no it's the dreaded terminology alert the very top of the tree is called the root node or just the root these are called internal nodes or branches branches have arrows pointing to them and they have arrows pointing away from them lastly these are called leaf nodes or just leaves leaves have arrows pointing to them but there are no arrows pointing away from them bam now that we know how to use and interpret classification trees let's learn how to build one from raw data this data tells us whether or not someone loves popcorn whether or not they love soda their age and whether or not they love the 1991 blockbuster cool as ice starring vanilla ice so we will use this data to build this classification tree that predicts whether or not someone loves cool as ice now pretend you've never seen this tree before and let's see how to build a tree starting with just data the first thing we do is decide whether loves popcorn love soda or age should be the question we ask at the very top of the tree to make that decision we'll start by looking at how well loves popcorn predicts whether or not someone loves cool as ice to do this we'll make a super simple tree that only asks if someone loves popcorn and then we'll run the data down the tree for example the first person in the dataset loves popcorn so they go to the leaf on the left and because they do not love cool as ice we'll keep track of that by putting a 1 under the word no the second person in the data set also loves popcorn so they also go to the leaf on the left and because they also do not love cool as ice we increment no to two the third person does not love popcorn so they go to the leaf on the right and because they love cool as ice we put a 1 under the word yes likewise we run the remaining rows down the tree keeping track of whether or not each one loves cool as ice bam now let's do the exact same thing for love soda at the two little trees we see that neither one does a perfect job predicting who will and who will not love cool as ice specifically these three leaves contain mixtures of people that do and do not love cool as ice dread it's another terminology alert because these three leaves all contain a mixture of people who do and do not love cool as ice they are called impure in contrast this leaf only contains people who do not love cool as ice because both leaves in the love's popcorn tree are impure and only one leaf in the love soda tree is impure it seems like love soda does a better job predicting who will and who will not love cool as ice but it would be nice if we could quantify the differences between love's popcorn and love soda the good news is that there are several ways to quantify the impurity of the leaves one of the most popular methods is called genie impurity but there are also fancy sounding methods like entropy and information gain however numerically the methods are all quite similar so we will focus on genie impurity since not only is it very popular i think it is the most straightforward so let's start by calculating the genie impurity for love's popcorn to calculate the genie impurity for love's popcorn we start by calculating the genie impurity for the individual leaves the genie impurity for the leaf on the left is 1 minus the probability of yes squared minus the probability of no squared so we start out with one then we subtract the squared probability of someone in this leaf loving cool as ice which is one the number of people in the leaf who loved cool as ice divided by the total number of people in the leaf four and then the whole term is squared lastly we subtract the squared probability of someone in this leaf not loving cool as ice which is three the number of people in the leaf who did not love cool as ice divided by the total number of people in the leaf squared and when we do the math we get 0.375 so let's put 0.375 under the leaf on the left so we don't forget it now let's calculate the genie impurity for the leaf on the right just like before we start out with one then we subtract the squared probability of someone in this leaf loving cool as ice and the squared probability of someone in this leaf not a loving cool is ice and when we do the math we get 0.444 now because the leaf on the left has four people in it and the leaf on the right only has three people in it the leaves do not represent the same number of people thus the total genie impurity is the weighted average of the leaf impurities we start by calculating the weight for the leaf on the left the weight for the left leaf is the total number of people in the leaf four divided by the total number of people in both leaves seven then we multiply that weight by its associated genie impurity 0.375 now we add the weighted impurity for the leaf on the right which is the total number of people in the leaf 3 divided by the total number of people in both leaves 7 times the associated genie impurity 0.444 and when we do the math we get 0.405 so the genie impurity for love's popcorn is 0.405 likewise the genium purity for love soda is 0.214 now we need to calculate the genie impurity for age however because age contains numeric data and not just yes no values calculating the genie impurity is a little more involved the first thing we do is sort the rows by age from lowest value to highest value then we calculate the average age for all adjacent people lastly we calculate the geniu impurity values for each average age for example to calculate the gd impurity for the first value we put age less than 9.5 in the root and because the only person with age less than 9.5 does not love cool is ice we put a 0 under yes and a 1 under no then everyone with age greater than or equal to 9.5 goes to the leaf on the right now we calculate the genie impurity for the leaf on the left and get zero and this makes sense because every single person in this leaf does not love cool as ice so there is no impurity then we calculate the genie impurity for the leaf on the right and get 0.5 now we calculate the weighted average of the two impurities to get the total genie impurity and we get 0.429 likewise we calculate the genie impurities for all of the other candidate values these two candidate thresholds 15 and 44 are tied for the lowest impurity 0.343 so we can pick either one in this case we'll pick 15. however remember that we are comparing genie impurity values for age loves popcorn and love soda to decide which features should be at the very top of the tree earlier we calculated the genie impurity values for love's popcorn and love soda and now we have the genie impurity for age and because love soda has the lowest genie impurity overall we know that its leaves had the lowest impurity so we put love soda at the top of the tree bam now the four people that love soda go to a node on the left and the people that do not love soda go to a node on the right now let's focus on the node on the left all four people that love soda are in this node three of these people love cool as ice and one does not so this node is impure so let's see if we can reduce the impurity by splitting the people that love soda based on love's popcorn or age we'll start by asking the four people that love soda if they also love popcorn because two of the four people that love soda also love popcorn they end up in the leaf on the left the remaining two people that love soda but do not love popcorn end up on the right and the total genie impurity for this split is 0.25 so let's put 0.25 here so we don't forget now we test different age thresholds just like before only this time we only consider the ages of people who love soda and age less than 12.5 gives us the lowest impurity zero because both leaves have no impurity at all so let's put zero here now because zero is less than 0.25 we will use age less than 12.5 to split this node into leaves note these are leaves because there is no reason to continue splitting these people into smaller groups likewise this node consisting of the three people who do not love soda is also a leaf because there is no reason to continue splitting these people into smaller groups now there is just one last thing we need to do before we are done building this tree we need to assign output values for each leaf generally speaking the output of a leaf is whatever category that has the most values in other words because the majority of the people in these leaves do not love cool as ice the output values are does not love cool as ice and because the majority of the people in this leaf love cool as ice the output value is love's cool as ice hooray we finished building a tree from this data double bam now if someone new comes along and we want to predict if they will love cool as ice then we run the data down our tree and because they love soda they go to the left and because they are 15 so age less than 12.5 is false they end up in this leaf and we predict that they will love cool as ice triple bam okay now that we understand the main ideas of how to build and use classification trees let's discuss one technical detail remember when we built this tree only one person in the original data set made it to this leaf because so few people made it to this leaf it's hard to have confidence that it will do a great job making predictions with future data and it is possible that we have overfit the data note if the term overfit is new to you don't don't instead check out the stack quest on bias and variance in machine learning regardless in practice there are two main ways to deal with this problem one method is called pruning and there's a whole stack quest dedicated to it so check it out alternatively we can put limits on how trees grow for example by requiring three or more people per leaf now we end up with an impure leaf but also a better sense of the accuracy of our prediction because we know that only 75 percent of the people in the leaf love to cool as ice note even when a leaf is impure we still need an output value to make a classification and since most of the people in this leaf love cool as ice that will be the output value also note when we build a tree we don't know in advance if it is better to require three people per leaf or some other number so we test different values with something called cross validation and pick the one that works best and if you don't know what cross validation is check out the quest bam now it's time for some shameless self-promotion if you want to review statistics and machine learning offline check out the statquest study guides at statquest.org there's something for everyone hooray we've made it to the end of another exciting stat quest if you like this stat quest and want to see more please subscribe and if you want to support statquest consider contributing to my patreon campaign becoming a channel member buying one or two of my original songs or a t-shirt or a hoodie or just donate the links are in the description below alright until next time quest on

Up Next

K-Nearest Neighbor Algorithm: Classification & Regression Guide

@IBMTechnology

106.1K views•2024-09-02

Building Real-Time ML Pipelines with Feature Stores and MLOps Frameworks

@ODSCAI

5.1K views•2022-02-20

Random Forests Explained: Build, Use & Evaluate ML Models

@statquest

1.4M views•2018-02-05

Neural Networks Explained: Math, Layers, and Learning Fundamentals

@3blue1brown

21.9M views•2017-10-05

Related Study Plans & Knowledge Roadmaps

Structured learning paths in Artificial Intelligence

View All Artificial Intelligence Topics

artificial-intelligence

Building Recommenders: Two-Tower & Vector DBs

01Foundations of Python & Machine Learning Basics

02Introduction to Recommender Systems

03Deep Learning & Representation Learning

+ 3 more learning modules

Explore Roadmap

artificial-intelligence

Latent Diffusion: PyTorch, UNet & CLIP

01Deep Learning & PyTorch Foundations

02Convolutional Networks & Autoencoders

03The Math and Mechanics of Diffusion Models

+ 3 more learning modules

Explore Roadmap

artificial-intelligence

NeRFs: 3D Reconstruction & Instant-NGP

01Math & Python Foundations for 3D Vision

02Photogrammetry & Camera Pose Estimation

03Neural Radiance Fields (NeRF) Core Concepts

+ 2 more learning modules

Explore Roadmap

artificial-intelligence

TFT Forecasting: Energy Demand & Deep Learning

01Programming & Data Science Foundations

02Introduction to Machine Learning & Time Series

03Deep Learning & Sequential Models

+ 3 more learning modules

Explore Roadmap