Creating Custom Datasets for NVIDIA Instant NGP (NeRF) Tutorial

Structure from Motion (SfM) is a photogrammetric technique that reconstructs 3D geometry from 2D images by exploiting how feature points move across multiple photographs taken from different viewpoints; unlike traditional photogrammetry which requires precise platform control and known parameters, SfM uses algorithms like SIFT (Scale-Invariant Feature Transform) to automatically identify matching features and perform bundle adjustment to solve for camera positions and 3D point cloud geometry, making it an accessible alternative to expensive LiDAR systems for creating detailed topographic models from unstructured image collections.

Structure from Motion (SfM) is a computer vision technique that reconstructs 3D geometry from multiple 2D images by simultaneously estimating camera poses and 3D point positions. The essential matrix E encodes the relative rotation and translation between two calibrated cameras, satisfying the epipolar constraint x₁ᵀEx₀ = 0 for corresponding points. With at least eight point correspondences, E can be estimated up to scale using linear algebra, and camera poses (R, T) are recovered through SVD, yielding four possible solutions resolved by the chirality constraint ensuring points lie in front of both cameras. Triangulation then computes 3D point positions from camera projections. For uncalibrated cameras, the fundamental matrix F replaces E, but introduces additional ambiguity. Multi-view SfM extends this approach, though accumulated errors cause drift, mitigated by bundle adjustment optimization.

Structure from Motion (SfM) is a photogrammetry technique that reconstructs a sparse 3D scene from a set of still images by computing camera poses (rotation and translation) for each view, using a pipeline that includes feature extraction, match computation with RANSAC filtering, initialization with an adequate initial pair, bundle adjustment via L-BFGS optimization, and iterative refinement using epnp and resectioning algorithms, with key quality checks based on separation angle and reprojection error to remove unreliable 3D points.

Structure from Motion (SfM) is a computer vision technique that reconstructs 3D scenes from multiple 2D images by establishing correspondences between feature points across views, then using the essential matrix and epipolar geometry to estimate camera poses and 3D point positions; Bundle Adjustment refines this reconstruction by simultaneously optimizing both the 3D structure and camera parameters through non-linear least squares optimization to minimize reprojection errors across all image observations.

Structure from Motion (SfM) photogrammetry is a computational technique that reconstructs 3D objects and terrains from overlapping photographs by analyzing feature correspondences across images. The workflow involves: (1) Feature detection using algorithms like SIFT to identify matching points between images, (2) RANSAC filtering to remove outlier matches, (3) Bundle adjustment to determine camera positions and generate a sparse point cloud, (4) Multi-View Stereo (MVS) to create a dense point cloud, and (5) post-processing to generate final products like meshes, elevation models, and orthophotos. Image texture significantly influences reconstruction quality, with textured surfaces yielding better results than uniform ones. This method differs from classical photogrammetry by automating the reconstruction process without requiring pre-defined flight patterns or metric cameras.

To run Python scripts from the command line: (1) Write code in a text editor and save with .py extension, (2) Open Command Prompt, (3) Navigate to the directory containing the script using 'cd' command, (4) Execute the script by typing 'python filename.py'. The file extension (.py) is critical for proper recognition by the Python interpreter.

This tutorial covers essential Python commands for command line interface (CLI) operations on Windows, including: checking Python installation path using 'where python', running Python scripts with 'python filename.py', executing single-line Python statements directly in Command Prompt, verifying Python version with 'python --version', installing external packages using 'pip install package_name', launching Python interpreter with 'python.exe', and installing Python via system package managers like Chocolatey, Homebrew, or RPM.

To make CLIs executable without typing 'python main.py', package them using setup tools. Create a setup.py file defining package metadata, dependencies, and entry points. The entry_points section maps command-line names to Python functions. Install with pip install -e for development (editable mode) or pip install for distribution. Editable mode allows code changes to be immediately reflected without reinstalling.

This section covers creating and executing Python scripts systematically. Create project directories with mkdir, navigate with cd, and generate scripts using touch hello.py. Edit scripts in IDLE, write code like print("hello world"), and save with Command+S. Execute scripts from Terminal with 'python3 hello.py'. Additionally, learn about pip, the Python Package Manager, verified with 'pip3 --version'. Pip enables installation of third-party packages to extend Python functionality, providing access to thousands of libraries for enhanced development capabilities.

pip is Python's package manager for installing third-party libraries. The -m flag runs modules as scripts (e.g., python -m pip list). Click is a library for building command-line interfaces with @click.command() and @click.option(). Understanding package management and CLI tools is essential for Python development.

Camera parameters are divided into extrinsic parameters (describing the camera's position and orientation in 3D space through six rigid-body transformation parameters: three translations and three rotations) and intrinsic parameters (describing internal camera characteristics including the camera constant/focal length, principal point offset, scale differences, and shear, totaling five parameters for an affine camera model). The complete mapping from 3D world coordinates to 2D image pixels involves a sequence of transformations: rigid-body transformation from world to camera coordinates, ideal perspective projection from camera to image plane, affine transformation from image plane to sensor coordinates, and optional nonlinear distortion correction. The projection matrix P = K[R|t] combines extrinsic (rotation R and translation t) and intrinsic (calibration matrix K) parameters, where K contains the five intrinsic parameters. Due to the central projection losing one degree of freedom, the mapping cannot be perfectly inverted - from a pixel coordinate, we can only determine the ray direction from the camera origin to the 3D point, not its exact depth.

A camera projection matrix P maps 3D world points to 2D image points through central projection, where all light rays converge at a finite camera center; this matrix can be factorized into intrinsic parameters K (containing focal lengths, skew, and principal point) and extrinsic parameters [R|t] (rotation and translation), giving the general form P = K[R|t], with the camera center located at the null space of the first three columns of P, and the principal axis direction determined by the cross product of the first two rows of the 3×3 matrix M.

Camera intrinsic parameters define how the camera projects 3D points onto the 2D image plane. These include focal length (fx, fy) which determines the field of view, and principal point (ppx, ppy) which represents the intersection of the optical axis with the image plane. Together with extrinsic parameters, these define the complete camera projection model necessary for accurate 3D reconstruction.

Camera parameters are divided into intrinsic (focal length, sensor size, principal point) and extrinsic (position and orientation in 3D world) parameters. The world coordinate system transforms to camera coordinates via rotation R and translation T: P_camera = R × P_world + T. The perspective projection matrix M = K × [R | T] combines intrinsic and extrinsic parameters, with image coordinates obtained by dividing the first two rows by the third row. A matrix M = [A | B] is a perspective projection matrix if det(A) ≠ 0. For zero skew, (A1 × A3) and (A2 × A3) must be orthogonal; for unit aspect ratio, their norms must be equal. Perspective projection is defined only up to a scale factor. Alternative models include orthographic projection (preserving parallelism), parallel projection (accounting for pinhole position), weak perspective projection (combining perspective and orthographic projections), and pilot perspective projection. Geometric camera calibration estimates parameters from image data using a calibration grid with known 3D coordinates. The calibration equations u_i = (M1 × P_i) / (M3 × P_i) and v_i = (M2 × P_i) / (M3 × P_i) form linear systems solved using linear least squares.

The complete camera projection model combines intrinsic and extrinsic parameters. The projection of a 3D point X to pixel coordinates is given by: x = K[R|t]X, where K is the intrinsic matrix, R is the rotation matrix, and t is the translation vector. The extrinsic parameters describe the camera's position and orientation in the world coordinate system. This model allows conversion between 3D world coordinates and 2D pixel coordinates.