CMPT 621 Lecture 5b: Ranked Retrieval & Vector Space Model

Added:

Neural Dense Retrieval and Semantic Search

While traditional Ranked Retrieval models like the Vector Space Model (VSM) and BM25 rely on sparse, lexical representation (matching exact keywords), they suffer from the 'vocabulary mismatch' problem and ignore word order and context. A major modern counterpoint is Neural Dense Retrieval. This paradigm utilizes deep learning models (such as Transformers) to encode queries and documents into dense, low-dimensional vector representations that capture deep semantic meaning. Instead of measuring mere term overlap, dense retrieval calculates similarity based on conceptual alignment. This directly addresses VSM's limitations regarding synonyms, polysemy, and syntax, offering a shift from syntactic keyword matching to true semantic understanding, albeit at a higher computational cost.

Basic Information Retrieval concepts, including the Boolean retrieval model and the structure of an inverted index.
Fundamental linear algebra, specifically vector representation of text, dot products, and vector magnitude.
Standard text preprocessing techniques such as tokenization, stemming, lemmatization, and stop-word elimination.
The basic statistical concepts of term frequency (TF) and document frequency (DF) in a corpus.
Evaluation methodologies for ranked retrieval, including Mean Average Precision (MAP), Normalized Discounted Cumulative Gain (NDCG), and Precision@K.
Optimization and query refinement techniques such as Relevance Feedback, Rocchio Algorithm, and Pseudo-Relevance Feedback.
Advanced probabilistic retrieval frameworks, including Language Models for Information Retrieval (LMIR) and Kullback-Leibler (KL) divergence.
Transitioning from sparse vector models to dense retrieval using deep learning embeddings and vector databases.
Learning to Rank (LTR) frameworks that utilize machine learning algorithms to optimize search engine ranking functions.
672 views10likes1:15:43@tamer_elsayedOriginal Release: 2021-02-19

The Vector Space Model represents documents and queries as vectors in a high-dimensional space where each dimension corresponds to a term in the vocabulary, with values representing term weights (such as TF-IDF). Cosine similarity measures the relevance between a query and documents by computing the cosine of the angle between their vectors, which equals the dot product when vectors are normalized to unit length. This approach addresses the limitation of TF-IDF by considering document length through normalization, where shorter documents containing query terms receive higher relevance scores than longer ones. The algorithm iterates through query terms, accumulates contributions from documents containing those terms, and ranks documents by their cosine similarity scores.