Recursive Descent Parsing Algorithm in C with Examples

FIRST sets contain terminal symbols that can appear first in any derivation from a non-terminal. Compute them from bottom to top: add terminals directly, recursively add non-terminal FIRST sets, and handle ε specially. FOLLOW sets contain terminals that can immediately follow a non-terminal in some sentential form. Compute them from top to bottom: initialize with $, propagate through productions, and add FOLLOW sets when ε is present in FIRST sets. These sets enable the parsing table construction.

LL(1) parsing requires computing FIRST and FOLLOW sets for all non-terminals. FIRST(A) contains terminals that can appear first in any derivation from A, including ε if A can derive empty string. For example, if B → D | ε, then FIRST(B) = {D, ε}. FOLLOW(A) contains terminals that can immediately follow A in some sentential form. The start symbol's FOLLOW always includes $. For non-terminals appearing in production rules, we examine subsequent symbols and their FIRST sets, adding to FOLLOW when ε is involved. These sets form the foundation for constructing the LL(1) parsing table.

An LL(1) parsing table is constructed by placing each production A → α in the row for variable A and the column for each terminal in First(α); if First(α) contains ε, the production is also placed in the Follow(A) columns. A grammar is suitable for LL(1) parsing only if no cell in the parsing table contains more than one production, which requires eliminating left recursion and ensuring that First sets of different productions for the same variable are mutually exclusive, and that First(α) and Follow(A) are disjoint when ε is in First(α).

To construct LL(1) table: (1) Create table with rows for non-terminals and columns for terminals + $, (2) For each production A → α, for each terminal a in FIRST(α), add A → α to table[A][a], (3) For each production A → α where ε in FIRST(α), for each terminal b in FOLLOW(A), add A → α to table[A][b], (4) Add accept to table[S][$]. The grammar is LL(1) if no cell has multiple entries.

The Follow function determines terminals that can follow a variable in sentential forms. Follow(start symbol) always contains $. For each production A → αXβ, add First(β) to Follow(X). If β can derive ε, add Follow(A) to Follow(X). The LL(1) parsing table is constructed using First and Follow functions. For each production A → α, and for each terminal a in First(α), add A → α to table[A][a]. For each production A → α where α can derive ε, and for each terminal b in Follow(A), add A → α to table[A][b].

Bottom-up parsing uses shift-reduce operations and is more powerful than top-down parsing. LR parsers scan left-to-right and build rightmost derivations in reverse. The instructor mentions different LR variants (SLR, LALR, CLR) and notes that LR parsers are table-driven with action and goto tables. While LR parsers can handle more languages than top-down parsers, they are harder to implement by hand, which is why parser generators like Yacc are commonly used.

This section covers bottom-up parsing and LR parsing fundamentals. Key topics include: (1) Bottom-up parsing starts from input string and works backwards to starting symbol using rightmost derivations in reverse order, (2) It is more powerful than top-down parsing but more complex to implement, (3) Handle detection is the fundamental operation - identifying substrings matching production right-hand sides, (4) Shift-Reduce parsing uses stack and input buffer with shift (push symbols) and reduce (replace handle with non-terminal) actions, (5) Shift-reduce conflicts occur when both actions are possible, requiring deterministic resolution, (6) LR parsing tables resolve conflicts by providing deterministic decisions, (7) LR0 items are productions with dots at various positions in right-hand sides, (8) The closure function adds productions for non-terminals following the dot, (9) The goto function moves the dot one position to the right, (10) The DFA is constructed by applying closure and goto functions to create states representing parser knowledge.

Bottom-up parsers are also called SR parsers (Shift-Reduce parsers). Shift means pushing onto a stack, and reduce means popping from the stack. LR parsers are named because they scan the input string from left to right (L) and use the reverse of rightmost derivation (R). LR parsers are of four types.

A parser is a program that generates a parse tree for a given string if the string is generated from the underlying grammar; parse trees can be generated using two approaches—top-down parsing (which uses leftmost derivation starting from the start symbol and deciding which production to use at each step) and bottom-up parsing (which uses rightmost derivation in reverse, starting from the string and reducing it back to the start symbol through successive reductions); parsers are classified into top-down parsers (with backtracking and without backtracking, including recursive descent and predictive parsers like LL(1)) and bottom-up parsers (also called shift-reduce parsers, including operator precedence parsers and LR parsers such as LR(0), SLR(1), LALR(1), and CLR(1)), where LR parsers increase in power from LR(0) to SLR(1) to LALR(1) to CLR(1).

Bottom-up parsers include LR parsers and operator precedence parsers. LR parsers work with reverse LR(0) items and can handle left recursion and left factoring without problems. They come in four types: LR(0), SLR(1), LALR(1), and CLR(1), differing in lookahead capabilities. Operator precedence parsers work on operator grammars and can handle both ambiguous and non-ambiguous grammars. LL(1) stands for Left-to-Right scan, Leftmost derivation, and 1 lookahead symbol.

The semantic analyzer takes a syntax tree as input and produces a modified syntax tree (annotated syntax tree). It handles semantic consistency that cannot be caught at parsing stage, including type checking of programming language constructs. The semantic analyzer stores information in the symbol table about variable types, function parameter types, and array dimensions.

Semantic analysis extracts meaning from programs after lexical and syntax analysis. Syntax trees are condensed parse trees where operators and keywords are interior nodes, and operands are leaf nodes. They separate translation from parsing by providing hierarchical structure. Semantic rules encode desired meaning extraction, such as generating syntax trees or computing values. The parser and semantic analyzer work together with translation rules embedded within the parser, producing abstract syntax trees and target code.

The compiler frontend consists of three phases: lexical analysis (identifying lexemes and generating tokens), syntax analysis (verifying tokens follow grammar and using symbol tables), and semantic analysis (verifying program meaning). The Abstract Syntax Tree (AST) captures essential program structure without all grammatical symbols. Expressions are represented by nodes where operators form parents and operands become children. Statements are represented by nodes where the statement type forms the parent with components as children. Semantic actions during syntax analysis create nodes and interconnect them according to grammar productions, building the complete AST while parsing.

The output of the semantic analyzer is an abstract syntax tree (AST), which is a modified version of the syntax tree produced by the syntax analyzer. The AST includes additional information and operators inserted by the semantic analyzer to handle type conversions and other semantic checks. For example, when converting an integer constant to a real constant, the semantic analyzer inserts an 'into real' operator in the tree. This abstract syntax tree serves as the input to the next phase of the compiler - intermediate code generation. The AST represents the program's structure with all semantic information properly resolved.

Parsers construct Abstract Syntax Trees (ASTs) representing program structure, with terminal nodes as tokens and non-terminal nodes as grammatical constructs. Parser ambiguity occurs when input can be parsed multiple ways, resolved by grammar rule precedence. C++ has notorious parsing ambiguities. Semantic analysis performs context-sensitive checks beyond syntax, maintaining symbol tables for type checking and consistency verification. This phase catches errors parsers cannot detect. Together, these phases enable sophisticated code transformation and analysis capabilities.

Compiler generators automatically produce compilers from specifications. Flex (Fast Lexical Analyzer Generator) creates lexical analyzers from token pattern specifications, outputting C code that recognizes these patterns. Yacc (Yet Another Compiler Compiler) and Bison generate parsers from grammar specifications, producing C code that implements syntax analysis. These tools are typically used together: Lex handles token recognition while Yacc/Bison handles grammar-based parsing. The Stanford Parser Generator (PEG) produces parsers that output Abstract Syntax Trees using deterministic approaches. The Wisconsin Parser Generator includes built-in error recovery capabilities.

Two fundamental front-end compiler construction tools are the lexical analyzer generator (Lex) and the parser generator (Yacc). Lex converts token specifications (regular expressions) into scanner programs by transforming them into finite automata, then generating C code. Yacc converts context-free grammars describing source language syntax into parser programs. Together, these tools construct the compiler's front-end, which reads source programs, verifies lexical and syntactic correctness, and produces parse trees. This automated approach significantly reduces the effort required to build compilers for new programming languages.

Lex is a lexical analyzer generator that takes regular expression patterns as input and generates a C file (lex.yy.c) containing a table-driven scanner that tokenizes source code, while Yacc is a parser generator that takes grammar specifications as input and generates C code (y.tab.c and y.tab.h) for a syntax analyzer that constructs parse trees; together, these tools simplify compiler development by automating the generation of scanner and parser components from formal specifications.

Lex is a standard lexical analyzer generator for Unix-based systems that reads lexical analyzer specifications and writes C source code implementing the lexical analyzer. Yacc (Yet Another Compiler Compiler) is an LALR parser generator used to implement syntax analysis. These tools are commonly used together for compiler implementation.

Yacc/Bison is a parser generator that converts grammar specifications into C code for parsing, while Flex is a lexical analyzer generator that converts regular expressions into C code for token recognition; both tools generate C source files (y.tab.c and lex.yy.c respectively) that are then compiled using a C compiler like gcc to create executable parsers that can analyze source code according to specified grammars.