Reverse Engineering ELF File Format: A Beginner's Guide to Linux Binaries

Static analysis tools: Ghidra, IDA Pro, Binary Ninja, Radare2 (general-purpose); Jadx (Java bytecode); dnSpy (.NET); decompile6 (Python bytecode). Dynamic analysis tools: GDB, rr, Frida, QEMU. Static vs dynamic linking: static linking includes all library code in the executable (self-contained but larger); dynamic linking loads libraries at runtime (smaller but requires external libraries). The 'file' command identifies linking type.

Three primary applications are used for reverse engineering: IDA Pro and Ghidra are debuggers/disassemblers that show assembly code and allow analysis of game code; Ghidra is preferred because it's open-source and free with a 3D compiler feature. IDA Pro has a free version but lacks features compared to its paid version which costs several thousand euros. HxD or similar hex viewers/binary viewers are used for examining binary data directly.

Ghidra, released by NSA, is a free disassembler and decompiler suitable for beginners. IDA Pro is the commercial standard with extensive community support. Both tools enable reverse engineering of binaries to understand program behavior at the assembly level, which is fundamental for exploit development and vulnerability research.

Radare2's r2pm package manager enables installation of decompilers like Ghidra (pdg command) and Deck (pdd command), which convert assembly code into more readable C-like representations, making binary analysis significantly easier by showing high-level code structures such as function calls, conditionals, and loops instead of raw assembly instructions.

IDA Pro (Interactive Disassembler) is a commercial reverse engineering tool performing disassembly and decompilation, historically expensive ($6,000/year). Binary Ninja offers a more affordable alternative at hundreds of dollars. Retargetable decompilers aim to reconstruct actual compilable source code from binaries, enabling cross-platform recompilation. Ghidra, an open-source tool developed by the NSA and released publicly, provides powerful analysis features including decompilation to pseudo-C code, custom format support, and platform independence. Practical reverse engineering involves systematic analysis techniques: Bottom-up approach starts with exports and entry points to understand program structure, while Top-down approach searches for known strings and traces references to reconstruct logic. These practical skills enable understanding compiled software behavior without source access, revealing program functionality through systematic binary analysis.

This comprehensive training covers x64 Linux binary exploitation techniques, including stack-based buffer overflow exploits, Return to Libc and Return Oriented Programming (ROP) to bypass NX (non-executable stack) protection, format string vulnerabilities to leak stack canaries and libc addresses, and chaining multiple vulnerabilities to bypass ASLR, NX, and stack canary protections simultaneously. The training demonstrates practical exploit development using tools like GDB, jeff, ropper, and pawn tools, with a focus on understanding 64-bit architecture limitations such as canonical addressing (where only 48 bits of 64-bit addresses are usable).

This comprehensive segment covers the foundational phases of solving a binary exploitation challenge. The presenter demonstrates downloading a CTF binary, identifying it as a 64-bit ELF executable, and performing initial reconnaissance using file, ltrace, and strings commands. The key vulnerability identified is the use of gets(), a deprecated C function lacking buffer bounds checking. Through static analysis with Ghidra, the presenter identifies a 48-byte buffer filled via gets() and determines the offset of 56 bytes needed to overwrite the return address. The segment introduces Return-Oriented Programming (ROP) as the solution when NX protections prevent shellcode injection, explaining how ROP chains existing code snippets ending with return instructions to achieve arbitrary code execution despite security defenses.

Return-Oriented Programming (ROP) is a binary exploitation technique used when traditional buffer overflow exploits are prevented by security mechanisms like stack canaries. Instead of injecting shellcode onto the stack, ROP chains together small code snippets called 'gadgets' that end with a return instruction. These gadgets are typically found in existing code sections and allow attackers to execute arbitrary operations by manipulating the call stack. Stack control refers to the ability to overwrite the instruction pointer (EIP) register with a custom address, giving attackers control over what the program executes next. Once EIP is controlled, attackers can redirect program execution to their chosen location. Effective binary exploitation requires systematic debugging techniques including GDB's pattern create command for identifying buffer overflow offsets, analysis of GOT and PLT tables for locating resolvable function addresses, and ROP gadget search tools for identifying useful code snippets ending with return instructions.

This video demonstrates an advanced return-to-libc attack that bypasses PIE (Position Independent Executable) and ASLR (Address Space Layout Randomization) protections by combining a format string vulnerability to leak the PIE base address with a buffer overflow to leak the libc address, ultimately achieving remote code execution through ROP (Return-Oriented Programming) chains that call libc.system('/bin/sh').

This comprehensive section covers binary exploitation techniques from buffer overflows to return-to-libc attacks. Key methods include: (1) Understanding why gets() is dangerous (reads without buffer size validation), (2) Stack layout analysis showing buffer (24 bytes), 8-byte variable, and 8-byte saved RBP totaling 40 bytes before return address, (3) Cyclic pattern analysis using GDB to determine the exact offset, (4) Little-endian address formatting for 64-bit binaries, (5) Return-to-function attacks where the return address is overwritten with the target function address, (6) Binary protection analysis using checksec to understand protections (NX, PIE, stack canaries), (7) ROP (Return Oriented Programming) for bypassing NX protection, (8) Finding pop rdi and pop rsi gadgets using ropper for 64-bit binaries, (9) Finding string addresses in binaries using GDB search or ropper, (10) ROP chain construction with padding for R15 register alignment, (11) Using pwntools to automate ROP chain construction, (12) Return-to-libc attacks for bypassing NX protection, (13) Leaking libc function addresses (like puts) to determine libc base, (14) Using glibc database for function offsets, (15) Calculating offsets to target functions (system) and strings ('/bin/sh'), (16) Stack alignment requiring return gadgets for 16-byte alignment, (17) Multi-stage exploitation requiring state reset (counter reset in BSS section), (18) Using gets() to write null bytes to reset counters, (19) Automated exploit development with pwntools creating ROP objects, (20) Remote exploitation by changing binary path to remote address and port. The challenge demonstrates how buffer overflows, ROP gadgets, and return-to-libc attacks enable function calls with specific parameters in protected binaries.

Reverse engineering involves finding vulnerabilities in software without source code through fuzzing and protocol research. The process is iterative: write a simple fuzzer, research the protocol deeper, improve the fuzzer, and repeat. Python Arsenal is a resource containing approximately 140 libraries for reverse engineering tasks, featuring a search engine for finding libraries for specific tasks. Python integrates with major tools through wrappers: WineDbg for Windows debugging, GDB for Linux, and Capstone for disassembly. Binary format parsing libraries (PE, ELF, Mach-O) abstract away manual parsing complexity. Python scripts can automatically analyze binaries to identify libraries lacking security protections like ASLR, DEP, and NX. Libraries like Capstone provide abstraction layers across multiple formats with semantic analysis. For unsupported architectures, researchers can write custom disassemblers by describing instruction formats, registers, and calling conventions. QEMU enables emulation of any described architecture through an intermediate language. Static analysis provides code coverage while dynamic analysis provides accurate runtime values. Decompilers like HexRays provide Python APIs for traversing code trees. Frida enables dynamic instrumentation across Windows, Linux, macOS, iOS, and Android using JavaScript executed by Google's V8 engine. Unicorn provides a lightweight, fast emulation engine optimized for integration into custom tools. Emulation enables analyzing unknown code by providing inputs and observing outputs, allowing discovery of algorithms like encryption without understanding implementation details.

Python serves as a fundamental scripting language for malware analysis automation and tool integration. It enables creation of custom analysis scripts, automation of repetitive tasks, and interaction with various analysis tools. The Python installer includes an option to add Python to the system PATH environment variable, allowing command-line access without specifying full file paths. This simplifies workflow automation and script execution during malware analysis investigations.

Several tools are available for analyzing ELF files: 'readelf' displays basic information about the file including magic bytes, architecture, and entry point; 'nm' lists symbols; 'objdump' displays the contents of sections; 'ldd' analyzes shared library dependencies; and 'pyelftools' is a Python library for ELF analysis and modification.

Several libraries exist for parsing PE files, including PEfile (Python), which supports multiple file formats including PE, ELF, and Mach-O. These libraries provide standardized APIs for extracting information from binary files, making it easier to perform malware analysis. Using established libraries ensures consistent parsing and reduces the risk of errors in custom implementations.

This comprehensive segment covers the complete static analysis workflow for suspicious binaries. It begins with initial triage: examining file properties, checking file size anomalies, reviewing section headers for unusual entries, and using debloat tools to remove junk data. The analysis progresses to examining DLL characteristics, import/export tables, and entry point behavior. A detailed examination of malware obfuscation techniques follows, including recursive function calls, memory allocation via VirtualAlloc, and XOR encryption where each byte is XORed with a repeating key. The segment culminates in identifying embedded encrypted payloads hidden within legitimate-looking binaries, demonstrating how attackers hide malicious code using simple cryptographic techniques.