Directional Derivative Explained | Vector Calculus Tutorial

A level surface f(x) = C has dimension m-1, meaning one coordinate is dependent on others. The gradient ∇f(x_0) is perpendicular to every smooth curve lying on the level surface passing through x_0, making it the normal vector to the level surface at x_0. A smooth curve γ(t) on the level surface satisfies f(γ(t)) = C for all t. Differentiating with respect to t gives ∇f(γ(t))·γ'(t) = 0, showing ∇f is perpendicular to every tangent vector of every smooth curve on the level surface. The tangent plane to the level surface at x_0 = (x_0,y_0,z_0) is the plane passing through x_0 with normal vector ∇f(x_0), with equation: f_x(x_0,y_0,z_0)(x-x_0) + f_y(x_0,y_0,z_0)(y-y_0) + f_z(x_0,y_0,z_0)(z-z_0) = 0. The normal line passes through x_0 in the direction of ∇f(x_0), with parametric equation (x,y,z) = (x_0,y_0,z_0) + t∇f(x_0) and canonical form (x-x_0)/f_x = (y-y_0)/f_y = (z-z_0)/f_z.

To find the tangent plane and normal line to a surface at a specific point, first express the surface as a level surface f(x, y, z) = constant, then compute the gradient vector ∇f at that point; the gradient vector is perpendicular to the surface and serves as the normal vector for the tangent plane equation A(x-x₀) + B(y-y₀) + C(z-z₀) = 0 and as the direction vector for the parametric equations of the normal line x = x₀ + At, y = y₀ + Bt, z = z₀ + Ct.

The gradient of a function is always perpendicular to its level surfaces (or level curves), which allows us to find tangent planes to surfaces and tangent lines to curves by computing the gradient at a given point and using the point-normal form equation. For a surface given as z = f(x,y), we rewrite it as F(x,y,z) = f(x,y) - z = 0, then the gradient ∇F = (∂F/∂x, ∂F/∂y, -1) gives the normal vector to the tangent plane at any point. Similarly, for a curve given as f(x,y) = c, the gradient ∇f = (∂f/∂x, ∂f/∂y) gives the normal vector to the tangent line at any point.

The tangent plane to a level surface f(x,y,z) = c at a point P₀ is the plane passing through P₀ and normal to the gradient vector ∇f(P₀); its equation is ∂f/∂x(P₀)(x-x₀) + ∂f/∂y(P₀)(y-y₀) + ∂f/∂z(P₀)(z-z₀) = 0. The normal line to the surface at P₀ is the line through P₀ parallel to the gradient vector, with parametric equations x = x₀ + ∂f/∂x(P₀)t, y = y₀ + ∂f/∂y(P₀)t, z = z₀ + ∂f/∂z(P₀)t. For surfaces defined implicitly as z = f(x,y), rewrite them as F(x,y,z) = f(x,y) - z = 0 and apply these formulas.

This comprehensive lesson covers four interconnected topics in multivariable calculus. First, the gradient vector ∇f = (∂f/∂x, ∂f/∂y, ∂f/∂z) represents the direction of maximum increase of a function. Second, the directional derivative Dᵤf(P) = ∇f(P) · u measures the rate of change in direction u, computed by finding partial derivatives, evaluating at point P, finding unit vector u, and computing the dot product. Direction can be specified as unit vector, angle θ, or two points. Third, the tangent plane to surface f(x,y,z) = c at P(x₀,y₀,z₀) is f_x(P)(x-x₀) + f_y(P)(y-y₀) + f_z(P)(z-z₀) = 0. Fourth, the normal line is x = x₀ + t f_x(P), y = y₀ + t f_y(P), z = z₀ + t f_z(P). Applications include exponential functions like f(x,y,z) = e^x + y + z and polynomial functions like f(x,y,z) = 2x + y + 2z². The directional derivative approximates function change: Δf ≈ Dᵤf(P) · Δs.

This comprehensive segment introduces Lagrange multipliers as a universal method for finding extrema of multivariable functions under constraints. The process involves: (1) introducing λ (Lagrange multiplier), (2) constructing the Lagrangian L = objective + λ·constraint, (3) computing partial derivatives ∂L/∂x, ∂L/∂y, ∂L/∂z, ∂L/∂λ, (4) setting all partial derivatives to zero to form a system of equations, (5) solving the system to find critical points. Applied to the same problem, this yields identical results (x=4, y=2, z=4, min=96) but works for any differentiable function regardless of complexity.

The method of Lagrange multipliers finds extreme values of a multivariable function subject to constraints. For f(x₁,...,xₙ) with constraint g(x₁,...,xₙ)=K, solve ∇f = λ∇g together with the constraint. This creates n+1 equations for n+1 unknowns (variables and λ). The solution points are evaluated in the original function to identify maxima and minima. Example: maximizing f(x,y)=x²+2y² on x²+y²=1 yields critical points (±1,0) and (0,±1), giving minima of 1 and maxima of 2.

This segment covers finding global extrema of multivariable functions and the Lagrange multipliers method. For f(x,y) = (x²+y²)e^(2-x-y) on the first quadrant, we find critical points by setting partial derivatives to zero: f_x = e^(2-x-y)(2x - x² - y²) = 0 and f_y = e^(2-x-y)(2y - x² - y²) = 0. This implies x = y, and solving gives critical points (0,0) and (1,1). Using the second derivative test with the Hessian determinant D = f_xx·f_yy - (f_xy)², we find D > 0 and f_xx > 0 at (0,0), indicating a local minimum, while D < 0 at (1,1), indicating a saddle point. For constrained optimization, we use Lagrange multipliers: to maximize f(x,y,z) = xyz subject to g(x,y,z) = xy + yz + xz - 12 = 0, we set up f_x + λg_x = 0, f_y + λg_y = 0, f_z + λg_z = 0, and g(x,y,z) = 0. This gives yz + λ(y + z) = 0, xz + λ(x + z) = 0, xy + λ(x + y) = 0, and xy + yz + xz = 12. Solving these equations yields the critical points that satisfy both the objective function and the constraint.

This tutorial covers three fundamental optimization techniques for functions of multiple variables: (1) Finding and classifying critical points using the second derivative test, which involves solving ∂f/∂x = 0 and ∂f/∂y = 0 to locate critical points, then using the discriminant D = f_xx × f_yy - (f_xy)² to classify them as local minima, maxima, or saddle points; (2) Finding global extrema on closed and bounded regions by evaluating the function at critical points inside the region and along all boundary edges, reducing multi-variable problems to single-variable ones by holding one variable constant; (3) Applying Lagrange multipliers to find extrema subject to constraints by introducing a Lagrangian function L = f - λg and solving the system of equations formed by setting partial derivatives to zero, then analyzing cases to find optimal points.

This video covers finding maximum and minimum values of functions of two variables, including global and local extrema, critical points (boundary points, stationary points where gradient equals zero, and singular points where the function is not differentiable), and the second derivative test for classification. It also introduces Lagrange multipliers for constrained optimization problems, where extrema are found by solving ∇f = λ∇g along with the constraint g = 0, automatically determining whether the solution is a maximum or minimum.

Gradient descent is widely used in machine learning for: (1) Linear regression, (2) Logistic regression, (3) Neural networks, (4) Support vector machines, (5) Many other optimization problems. The algorithm's ability to find optimal solutions makes it fundamental to modern machine learning.

Gradient Descent is applied to real-world problems such as predicting processing time based on temperature data. The algorithm finds the optimal parameters (m and b) for the linear equation y = mx + b that best fits the data. This demonstrates how gradient descent enables practical machine learning applications by automatically finding the best-fit line through iterative optimization.

Optimization solves practical problems: rectangle with area 100 minimizes perimeter at L=B=10 (P=40); average cost C(Q) = 0.001Q² - 0.08Q + 65 + 5000/Q minimizes at Q=500. Gradient descent is an iterative method: xₖ₊₁ = xₖ - α∇f(xₖ). For f(x₁, x₂) = x₁² - 2x₁x₂ + 2x₂² + 2x₁, starting at (0, 0) with α=0.5: x₁ = (-1, -1), x₂ = (-2, -1), x₃ = (0, -1). The algorithm converges toward the minimum. This method is widely used in machine learning for parameter optimization.

Gradient descent is used for: (1) Linear regression to find the best-fit line, (2) Logistic regression for classification, (3) Neural networks for complex pattern recognition, (4) Recommendation systems to optimize user recommendations. The algorithm's ability to minimize loss functions makes it applicable to any optimization problem where the goal is to find the best parameters for a given objective.

Gradient descent is an optimization algorithm used to find the best parameters (intercept and slope) for a regression line. The problem of finding the optimal line is equivalent to finding the minimum point on a 3D graph that represents the error function. Gradient descent is the backbone underlying most modern machine learning techniques, including neural networks and Transformers, making it essential knowledge for anyone pursuing machine learning.