Build a Custom BitTorrent Client: Implementation Guide

Async/await addresses the fundamental problem of IO-bound programs blocking the CPU during operations like file reads, network requests, or database queries. When programs wait for IO, the CPU sits idle despite having capacity for other work. Two primary solutions exist: threads (using OS-level execution contexts) and async/await (using cooperative multitasking with event loops). Async/await frameworks implement cooperative multitasking through event loops and callbacks, where functions tagged as async can be suspended and resumed. The framework maintains a queue of events and processes them sequentially, calling back when operations finish. This approach exists at multiple abstraction levels: basic non-blocking IO with callbacks, Futures and Promises, and language-level async/await keywords. Coroutines enable this by allowing functions to suspend and resume while preserving execution state. Once async patterns are introduced, they spread pervasively throughout codebases, creating bifurcation between synchronous and asynchronous APIs. The optimal approach depends on the workload: IO-bound workloads benefit from async/await's ability to keep the CPU busy, while CPU-bound workloads may benefit from threads' ability to utilize multiple cores.

Parallel processing means tasks execute simultaneously using multiple resources (walking and talking at the same time). Concurrency means multiple tasks progress during a unit of time, but only one task executes at any given moment due to resource contention (speaking or drinking but not both). Asynchronous programming is essential because real-world applications rely heavily on I/O operations (web services, databases, file systems, logging) that occur outside the CPU. While blocking is necessary to receive results, asynchronous programming allows the underlying thread to be freed while waiting for I/O completion.

Three distinct concepts are often confused: (1) Concurrent execution - multiple tasks running simultaneously on a single CPU by time-slicing, (2) Parallelism - truly simultaneous execution across multiple CPU cores, (3) Asynchronous processing - handling I/O operations without blocking the CPU. Async/await is primarily designed for asynchronous processing where the CPU can work on other tasks while waiting for slow I/O operations. Unity's UI thread handles user input and graphics rendering; heavy processing on this thread causes freezing. Async/await is ideal for moving heavy processing off the UI thread, then returning results back when complete.

Asynchronous programming involves several key concepts: Continuation (the mechanism that resumes execution after an operation completes), Task (the abstraction representing an asynchronous operation), and Thread Pool (the system that manages execution of asynchronous tasks). These concepts work together to enable efficient asynchronous execution without blocking threads. Understanding these fundamentals is essential before diving into specific implementations like Task and async/await.

This section introduces asynchronous programming concepts using a factory analogy. Robots perform tasks while supervisors assign and launch them, allowing supervisors to work on other tasks while robots execute assigned work. Three task response types exist: no feedback (void return), notification only (event triggering), and return value (data output). The supervisor assigns tasks to robot worker functions, which execute independently. This parallel execution model improves efficiency by allowing multiple tasks to run simultaneously rather than waiting sequentially for each to complete.

The Buffer class provides binary data handling with safe (Buffer.alloc) and unsafe (Buffer.allocUnsafe) allocation. Safe allocation zeros memory, preventing information leakage; unsafe allocation is faster but may contain garbage data. Buffers can be constructed from arrays, strings, or other buffers. Buffer.concat combines multiple buffers. Encoding methods like toString() with UTF-8 or Base64 convert binary data to strings. Base64 uses 6 bits per character, requiring 4 characters for 3 bytes.

Buffer is a binary data container in Node.js, distinct from JavaScript arrays. Arrays are dynamic collections that grow and shrink, while Buffers are fixed-size containers for binary data. Computers process data in binary (0s and 1s), with the smallest unit being a byte (8 bits). Buffer creation methods include Buffer.from(array) for wrapping arrays, Buffer.alloc(size) for zero-initialized buffers, and Buffer.allocUnsafe(size) for faster allocation with uninitialized memory. Buffer values are assigned using bracket notation, where values are interpreted as bytes. A single byte can represent values from 0 to 255 (2^8 = 256 possible values). When values exceed 255, only the lower 8 bits are stored, and higher bits are discarded. This truncation behavior is fundamental to binary data manipulation.

The Byte Array Operations component simplifies binary data manipulation in Crypto 2. Key operations include: Concatenate (combines arrays), Sub Array (extracts portions by offset/length), Index Of (finds byte positions), Replace (substitutes values), AND/OR/XOR (bitwise operations), NOT (inverts bits), and Circular Shift (rotates bits with wrap-around). The String Decoder converts text to binary formats like hexadecimal. These operations enable direct byte-level manipulation for cryptographic protocols, eliminating the need for hex string conversions and simplifying offset handling.

Binary data consists of zeros and ones (0s and 1s), where 1 represents True and 0 represents False. A memory buffer is a dedicated section of memory for storing binary data. The Binary Data plugin provides unprocessed access to this buffer with safe memory access for JavaScript. One byte equals 8 bits, which are the fundamental binary data units. Signed integer types (int) store both positive and negative numbers with ranges based on bit size: int8 (-128 to 127), int16 (-32,768 to 32,767), int32 (-2.1B to 2.1B), int64 (-9.2E18 to 9.2E18). Unsigned integer types (uint) store only positive numbers, effectively doubling the positive range. Base64 encoding converts binary data to text format using 64 characters for storage in text files. ASCII assigns numerical codes to characters, while UTF-8 is the most widely used character encoding supporting characters from virtually all writing systems worldwide. Endianess refers to byte storage order: Big Endian stores most significant byte first, Little Endian stores least significant byte first. Buffer operations include comparing length, reading values at specific offsets, copying data between buffers, and filling ranges with repeated values. Data conversion between binary and text formats enables storage in text files and display as readable text. Compression algorithms (JPEG for images, Deflate for general data) reduce file size by eliminating redundant data. Reading operations include getting buffer length, reading specific data types at specified offsets, and decoding text. The Get URL expression provides a blob URL for accessing binary data in the current session's memory. Loading images involves using AJAX requests to fetch image files, storing binary data in the buffer, and using Set Image From URL with the blob URL. The plugin integrates with File Input for user file selection, enabling users to upload files and process them programmatically.

Byte streams (InputStream/OutputStream) read/write binary data (images, music, videos). FileInputStream/FileOutputStream open files for reading/writing. Buffered streams (BufferedInputStream/BufferedOutputStream) improve performance by reading/writing larger chunks. Always close streams after use to release resources. The try-with-resources pattern simplifies resource management by automatically closing streams.

Cryptography encompasses three fundamental categories: Hashing, Encryption, and Key Exchange. Hashing algorithms convert variable-length data into fixed-size outputs for data integrity verification. Common standards include MD5 (128-bit), SHA-1 (160-bit), SHA-2 (256-bit), and SHA-3 (512-bit). The core purpose is ensuring transmitted or stored files remain unaltered. During file transfer, the sender generates a hash and transmits it alongside the data. The recipient recalculates the hash upon receipt; matching hashes confirm data integrity, while mismatches indicate tampering. This technique is essential for software distribution verification, ensuring downloaded files match original versions.

Hashing is a cryptographic technique that verifies data integrity by generating a unique fixed-size output (hash value) from input data, where even the smallest change to the input produces a completely different hash output. Common hashing algorithms include MD5 (128-bit/32 hex characters), SHA-1 (160-bit/40 hex characters), and SHA-2 variants (256, 384, or 512 bits). Hashing is also known as digital fingerprinting or cryptographic hash, and is widely used for email transmission, password storage in Unix and Windows systems (NTLM), and data integrity verification.

Hashing creates one-way cryptographic fingerprints that change materially when input data is altered, providing data integrity and non-repudiation. Common algorithms include MD2/MD4/MD5 (128-bit), SHA-0/SHA-1 (160-bit, now insecure), SHA-2 (256/512-bit), SHA-3 (Keccak, highly secure), Whirlpool (512-bit), and RIPEMD. Birthday attacks exploit probability theory to find collisions where different inputs produce same hash outputs, compromising hash uniqueness. Digital signatures combine hashing and asymmetric encryption: the sender hashes the message then encrypts the hash with their private key. Recipients decrypt with the public key and re-hash the received message; matching hashes prove authenticity and integrity.

Hash functions verify data integrity by comparing hash digests before and after transmission. Before sending data (like an email), compute and send the hash digest alongside the content. The recipient recomputes the hash and compares it to the received digest. If they match, the data hasn't been corrupted or altered during transit; if they differ, corruption or tampering occurred.