Content-Based vs Collaborative Filtering for Recommendation Systems

This lecture demonstrates how to implement two deep learning approaches for recommendation systems: Neural Collaborative Filtering (NCF) and Autoencoders. NCF combines user and item vectors through a multi-layer perceptron to predict ratings, while Autoencoders compress and reconstruct user-item matrices to learn latent patterns. The instructor shows practical implementation using PyTorch, including GPU acceleration setup, data preprocessing with MovieLens dataset, train-test splitting, model architecture design with dropout and activation functions, and hyperparameter tuning. Key concepts include tensor operations, loss functions (MSE), backpropagation, and the importance of proper data preparation and model configuration for effective recommendation systems.

Deep learning has revolutionized recommender systems by enabling automated feature extraction, handling heterogeneous data types (images, text, audio), and modeling sequential user behavior through techniques like item embeddings (extending Word2Vec to user-item interactions), autoencoders for collaborative filtering, content-based feature extraction using CNNs/RNNs, and session-based recommendations with recurrent neural networks, achieving approximately 30% improvement over traditional methods.

Neural collaborative filtering replaced dot products with multi-layer perceptrons to learn complex similarity functions. The architecture projects user and item one-hot encodings into latent spaces using embedding layers, concatenates vectors, and applies MLP layers with sigmoid activation for binary classification. Element-wise multiplication of user and item representations before MLP layers allows falling back to matrix factorization if learned weights become zero. Modern systems focus on representation learning to create unified vector representations from diverse feature sources—including structured features (job titles, education) and unstructured text—concatenated and passed through embedding layers. Autoencoders provide alternatives by learning compressed representations focusing reconstruction only on observed ratings. Cross and Deep networks enable efficient polynomial relationship learning with fewer parameters than standard regression.

Neural Collaborative Filtering (NCF) is a neural network-based recommender system framework introduced in 2017 that generalizes traditional matrix factorization by replacing the inner product operation with multi-layer perceptron connections, enabling more expressive modeling of user-item interactions in latent space; the model combines Generalized Matrix Factorization (GMF) with MLP modules, followed by concatenation and sigmoid activation, and achieves superior performance on real-world datasets compared to state-of-the-art models.

Neural Collaborative Filtering (NCF) is a deep learning-based recommendation system that addresses the limitations of traditional Matrix Factorization (MF) by combining MF's linear characteristics with Multi-Layer Perceptron's (MLP) non-linear capabilities. The model uses a neural network framework where users and items are first embedded into dense vectors, then processed through multiple layers to learn complex non-linear relationships between users and items. This approach overcomes MF's inability to capture complex user-item interactions, as demonstrated through experiments on MovieLens and Pinterest datasets using Hit Ratio and NDCG evaluation metrics, showing superior performance compared to baseline models.

Precision@K measures the proportion of recommended items that are relevant. Recall@K measures the proportion of all relevant items successfully retrieved. NDCG (Normalized Discounted Cumulative Gain) is the most important ranking metric, rewarding relevant items appearing higher in the list more heavily. It applies a discount factor to account for decreasing importance of lower positions.

Precision measures how many recommended items users interact with (e.g., recommend 3 items, user buys 1 = 33% precision). Recall measures what percentage of items users bought were in recommendations (e.g., user bought 2 items, 1 was recommended = 50% recall). NDCG (Normalized Discounted Cumulative Gain) considers both which items are recommended and their position, penalizing placing highly relevant items lower in the list. These metrics help evaluate recommendation quality, though offline metrics don't always correlate with business outcomes.

Precision@k and Recall@k are evaluation metrics for recommender systems that measure the quality of top-k recommendations; Precision@k calculates the proportion of recommended items that are actually relevant (true positives divided by total recommended items), while Recall@k calculates the proportion of all relevant items that were successfully captured in the top-k recommendations (true positives divided by total relevant items), with higher values indicating better performance.

Common evaluation metrics for recommendation systems include: (1) Precision - proportion of recommended items that are actually relevant; (2) Recall - proportion of relevant items that were recommended; (3) RMSE (Root Mean Squared Error) - measures prediction accuracy for rating prediction tasks. More advanced metrics like NDCG (Normalized Discounted Cumulative Gain) evaluate ranking quality when the order of recommendations matters.

Рекомендательные системы используются для предсказания предпочтений пользователей на основе их взаимодействий с айтемами, где данные могут быть явными (оценки) или неявными (поведение). Основные методы включают: популярные рекомендации (без персонализации), метод соседей (похожие пользователи/айтемы), и матричную факторизацию (SVD). Для оценки качества используются метрики ранжирования: Precision@K, MAP, и NDCG, которые учитывают позицию рекомендаций, так как в рекомендательных системах важна не только релевантность, но и порядок предложений.

For Mercari's 150 million+ images, exact nearest neighbor search is computationally infeasible, requiring approximate methods. Faiss library enables efficient vector compression and indexing for ANN search. Key challenges include: (1) handling 50,000 new items per hour requiring frequent index updates, (2) duplicate images from business users listing identical items, and (3) maintaining search accuracy while managing multiple index types (hourly, daily, monthly). These challenges necessitate innovative solutions for real-time index maintenance and duplicate elimination.

Vector search scalability requires transitioning from brute force (comparing against all vectors) to approximate nearest neighbor (ANN) methods. Brute force works for small datasets (<100k vectors) but fails at scale due to linear time complexity. ANN strategies like IVF (K-means clustering with centroid probing) and HNSW (graph-based hierarchical traversal) sacrifice some recall (90-95%) for dramatic speed improvements. Modern vector databases use segmentation—growing segments until reaching size thresholds (e.g., 512MB), then sealing and distributing to index nodes. This enables horizontal scaling by distributing segments across machines. Multiple query nodes process different segments in parallel, with results combined afterward. This architecture addresses domain shift problems and enables distributed deployment without monolithic indexes.

Scaling to millions of vector searches requires efficient vector search. FAISS (Facebook AI Similarity Search) is an open-source library built by Facebook Research that can scale to billions of vectors with native GPU support. It provides out-of-box optimization strategies including vector compression and approximate nearest neighbor search. In experiments, it achieved 1.2 milliseconds per vector search time on a TPU with a 25GB index containing 13 million vectors. The implementation requires only a few lines of code to create an index, store embeddings, and perform k-NN search.

K-Nearest Neighbors (KNN) computes similarity scores between query and all vectors, ranking and returning top K results, but is computationally heavy for large datasets. Approximate Nearest Neighbor (ANN) algorithms trade accuracy for efficiency. Popular libraries include FAISS (Facebook, 2017), Google's ScaNN (2020), Spotify's Annoy (2013), and HNSW (2016). Vector databases organize vectors into buckets, trees, or graphs using heuristics to increase likelihood that similar vectors are close. Vectors can be quantized or made sparse to reduce computational intensity.