CKY Algorithm for CFG: Construct Parse Tree | NLP Numerical Solved

Transition-based dependency parsing is a greedy algorithmic approach that iteratively builds dependency trees by applying a sequence of transition operations (shift, left arc, and right arc) to a stack and input buffer, starting from a root node and ending when all words are parsed; the arc standard method specifically uses these three operators where shift moves words from the input buffer to the stack, left arc creates a dependency with the top word as head and the second word as dependent while popping the dependent, and right arc creates a dependency with the top word as dependent and the second word as head while popping the dependent, with the oracle guiding transitions through supervised machine learning to select the optimal path toward a valid dependency parse.

The Earley Parsing Algorithm, invented by Jay Earley, is a chart-based parsing method that can handle left-recursive, right-recursive, and ambiguous grammars efficiently, though it operates in O(n³) time complexity in the worst case; the algorithm works by maintaining a state table where each column represents an input position, and for each state, the parser performs scanning (matching terminal symbols) and completion (resolving completed non-terminal symbols) steps to build valid parse trees incrementally.

This comprehensive section covers the fundamental parsing algorithms used in computational linguistics. It begins with CKY (Cocke-Younger-Kasami), a bottom-up dynamic programming algorithm that builds a triangular parsing table, filling cells from left to right and bottom to top to avoid redundant computations while recovering all ambiguous sentence representations. Earley parsing is presented as a top-down alternative that handles any grammar type using three states: completed, in-progress, and predicted constituents. The section then transitions to dependency grammar, explaining how syntactic structure consists of binary asymmetric relationships between lexical items with heads and dependents. Key constraints include lexical affinities, short dependency distances, avoiding intervening material, and valence constraints. The distinction between projective (non-crossing arcs) and non-projective structures is established, noting that dependency grammars naturally handle non-projective phenomena common in languages like Turkish. The section concludes with detailed mechanics of transition-based parsing, explaining the three core data structures (buffer, stack, dependency arcs) and the three fundamental actions (shift, left arc, right arc).

Using 'the tabby cat scratched the couch' as an example: (1) Initialize with root in stack, all words in buffer, empty dependencies; (2) Shift 'tabby' to stack; (3) Apply Left Arc between 'tabby' and 'cat' (mod relation, cat is head); (4) Apply Left Arc between 'the' and 'cat' (det relation, cat is head); (5) Shift 'scratch' to stack; (6) Apply Left Arc between 'cat' and 'scratch' (subj relation, scratch is head); (7) Shift 'the' to stack; (8) Apply Left Arc between 'the' and 'couch' (det relation, couch is head); (9) Apply Right Arc between 'scratch' and 'couch' (obj relation, scratch is head); (10) Connect root to 'scratch' (main verb). The algorithm loops until final state (stack=[root], buffer=[]), selecting the best action at each step. The action selection problem is framed as multiclass classification where the input is the current state and output is the action to take. Features include POS tags, word positions, and word embeddings (dense vectors like 300 dimensions). Neural dependency parsers can be fully end-to-end, learning everything from scratch. Transition-based parsers can only produce projective trees, which causes errors for non-projective sentences. Graph-based parsing finds the best dependency tree by maximizing a scoring function (edge factor model), where the score for a tree is the sum of scores of all its edges. This is a maximum spanning tree problem. Graph-based parsing is quadratic in input length (O(n²)), while transition-based is linear (O(n)).

Parser evaluation uses precision and recall metrics calculated from constituent sets. Let G represent the number of constituents in the gold-standard tree, P represent the number in the parser's output, and C represent the number of correct constituents shared between both. Recall is calculated as (C/G) × 100%, measuring what percentage of gold-standard constituents were correctly recovered. Precision is calculated as (C/P) × 100%, measuring what percentage of parser's proposed constituents are actually correct. These metrics allow comparison of parsing accuracy across different parser models.

Parser performance is evaluated using ParsEval metrics: precision measures the proportion of predicted constituents matching the gold standard, while recall measures the proportion of gold standard constituents correctly identified. The F-score combines both metrics. Treebanks enable cross-linguistic research as similar constituent structures (NPs, VPs, prepositional phrases) appear across languages despite variations in specific node labels. Examples include Italian, Turkish, Japanese, and Arabic treebanks. Computational linguistics uses n-ary grammar rules rather than strictly binary rules, allowing more natural representation of language structures while remaining convertible to binary formats for parsing efficiency. This theoretical agnosticism accommodates multiple linguistic frameworks without committing to any single theory.

Precision in dependency parsing measures what percentage of predicted dependency relations are correct (true positives divided by predicted positives). Recall measures what percentage of correct dependency relations are retrieved from the total correct elements (true positives divided by actual positives). F1 score is the harmonic mean of precision and recall. These metrics, similar to those used in POS tagging and NER tasks, evaluate how well the parser identifies grammatical relationships.

Precision measures the proportion of true positives among all positive predictions (TP/(TP+FP)), addressing false positives. Recall measures the proportion of true positives among all actual positives (TP/(TP+FN)), addressing false negatives. F1 score combines both using harmonic mean: 2PR/(P+R), balancing precision and recall. Metric selection depends on problem context: spam filtering prioritizes precision to avoid losing important emails, while disease testing prioritizes recall to prevent missing infected cases.

Syntactic parsing is an NLP technique that analyzes the grammatical structure of sentences to understand how words relate to each other, enabling machines to interpret meaning by identifying relationships between words and phrases through processes like tokenization, part-of-speech tagging, and hierarchical representation construction using either rule-based methods (context-free grammars, hidden Markov models) or statistical approaches (probabilistic models, machine learning), with applications in machine translation, text summarization, question answering, and speech recognition.

Parser adaptation extends beyond maximizing parse accuracy to optimizing for downstream task performance. Pre-ordering demonstrates this principle by incorporating syntax into machine translation through word reordering before translation, improving BLEU scores by 1-1.5 points on Japanese-to-English translation. Augmented loss perceptron training formalizes this approach by iteratively training on both parse accuracy and task evaluation scores, sampling from k-best parse lists and updating parameters based on which parse yields better task performance. This extrinsic training paradigm recognizes that different tasks value different syntactic analyses—parsers optimized for question answering may differ from those optimized for translation. The research demonstrates that adapting parsers to specific applications leads to better downstream task outcomes than generic accuracy maximization, establishing a new paradigm where parser training objectives align directly with application requirements rather than abstract linguistic measures.

Before LLMs, NLP required explicit linguistic structure prediction. Part-of-speech (POS) tagging assigns grammatical categories (verb, noun, adjective) to words based on context, handling word ambiguity (e.g., 'book' as verb or noun). Named Entity Recognition (NER) identifies entities like persons, organizations, and locations, using BIO notation (Begin, Inside, Outside) for multi-word phrases. NER applications include targeted sentiment analysis, question answering, and cultural analytics (studying entity depictions in large text collections). Syntactic parsing assigns grammatical structure: constituency parsing breaks sentences into hierarchical phrase structures (noun phrases, verb phrases), useful for languages with fixed word order. Dependency parsing describes direct grammatical relations between words, with Universal Dependencies (UD) providing a unified schema across 100+ languages. Syntactic structure helps with: (1) Question answering (identifying subject-object relationships), (2) Machine translation (handling different word orders), (3) Understanding complex sentences (e.g., 'visiting relatives can be annoying' has different meanings based on structure).

Parsing provides structural knowledge that enables better NLP performance. For sentiment analysis, understanding word relationships helps determine if a sentence is positive or negative. For machine translation, translating to trees rather than strings improves translation quality. For information extraction, understanding word relationships helps extract facts from text. For question answering, knowing what a word indicates requires structural knowledge. Parsing can recover structure without understanding word meanings, using only part-of-speech tags, which allows learning from relatively few data.

Semantic role labeling is the process of assigning labels to words or phrases in a sentence to indicate their semantic roles, such as agent (who performs an action), predicate (the action itself), theme (what is being acted upon), and recipient (who receives the result), enabling machines to understand sentence meaning; semantic parsing extends this by converting natural language utterances into machine-understandable logical forms through system architectures including token analysis, syntactic parsing, and semantic interpretation, with applications ranging from question-answering systems to code generation.