Building a TensorFlow Ranking Model with TF Recommenders

TensorFlow is Google's open-source deep learning library. Before version 2.0, Keras was a separate wrapper; now it's integrated. The Sequential API builds linear stacks of layers. Dense layers create fully connected neurons where each neuron connects to all previous layer neurons. Activation functions (ReLU, sigmoid) introduce non-linearity. Dropout layers prevent overfitting by randomly deactivating neurons during training (e.g., 30% dropout means 30% of neurons are deactivated per iteration).

The Keras Sequential API creates models as sequences of layers. You start with model = Sequential(), then add layers using model.add() specifying the number of neurons and activation function for each dense layer. After defining the architecture, compile the model specifying the loss function, optimizer, and metrics. Finally, train the model using model.fit() with input data and labels.

Keras began as a high-level API that ran on top of TensorFlow, Theano, or CNTK, simplifying neural network creation with clean syntax. Instead of managing tensor math and graph nodes, users define layers (input, hidden, output) in few lines of code. Now fully integrated into TensorFlow, Keras provides an easy entry point for beginners while accessing all of TensorFlow's capabilities through its simplicity.

Keras is a high-level API that can run on top of TensorFlow (or Theano) as a backend. It provides a sequential model interface where you add layers one after another, compile the model with an optimizer and loss function, and then call model.fit() to train automatically. This dramatically reduces the amount of boilerplate code compared to raw TensorFlow, though it sacrifices some control over the underlying computational graph. Keras is particularly useful for rapid prototyping and standard network architectures.

Constructing neural networks in Keras involves sequential layer stacking. Import Sequential from tensorflow.keras.models and Dense from tensorflow.keras.layers. The first Dense layer requires input_shape to define input dimensions (784 for MNIST). Subsequent layers specify only unit counts. Compilation configures training behavior: optimizer (Adam) determines weight updates, loss function (sparse_categorical_crossentropy) measures error, and metrics (accuracy) track performance. This setup prepares the model for training on the prepared dataset.

Embeddings compress high-dimensional categorical features into lower-dimensional dense vectors while preserving semantic relationships. Categorical features undergo one-hot encoding followed by dimensionality reduction, while continuous features map to continuous embedding spaces. User and item embeddings exist in the same vector space, enabling direct comparison. Beyond standard classification loss, regularization terms enforce that cosine similarity relationships are preserved during transformation—penalizing distortions in angles between user and item vectors. This maintains interpretability and utility for downstream tasks. Approximate nearest neighbor search then efficiently finds top T nearest neighbor items (100-200) in milliseconds, trading some precision for dramatic speed improvements essential for production deployment.

Embeddings convert categorical features (user IDs, movie IDs, location) into dense vector representations because these categories lack inherent numerical meaning. Each category ID indexes into an embedding table to retrieve a learned vector. Multi-hot features handle histories by combining multiple embedding lookups (e.g., averaging vectors from watched movies). Embedding tables are learned during training and vary widely in size from hundreds of megabytes to tens of terabytes in large recommendation systems.

Embedding matrices convert high-cardinality categorical variables into dense continuous vectors, enabling neural networks to learn distributed representations where similar categories have similar vector representations. Each category level is assigned a row in the matrix, and during forward pass, the corresponding row is looked up and appended to continuous features. Matrix dimensions are typically set to approximately half the cardinality (but no more than 50). This approach captures semantic relationships that one-hot encoding cannot, allowing the model to generalize across related concepts.

Rather than manually defining features, latent features can be learned directly from user-item interaction data through matrix factorization. This technique decomposes the large U-by-I interaction matrix into two smaller matrices: one representing user embeddings and one representing item embeddings. The number of latent features (K) acts as a hyperparameter balancing information compression against reconstruction error. This approach offers dramatic space efficiency improvements—for systems with millions of users and thousands of items, embedding representations can reduce storage requirements by thousands of times compared to storing the full interaction matrix.

Embedding matrices are lookup tables that map categorical IDs to dense vector representations. In collaborative filtering, user IDs and item IDs are mapped to vectors of latent factors through embedding matrices. The embedding matrix for users has dimensions (number of users × number of latent factors), and the embedding matrix for items has dimensions (number of items × number of latent factors). These matrices are learned during training.

Explicit feedback involves direct ratings (e.g., 1-5 stars), while implicit feedback captures user behavior without explicit ratings (e.g., clicks, views, purchases). Implicit feedback is often considered more honest because users cannot easily fake it. However, implicit feedback is less informative about the intensity of preference.

Recommendation systems must distinguish between explicit feedback (direct ratings, purchases) and implicit feedback (views, clicks, watch time). Explicit feedback is more reliable but scarcer, while implicit feedback is abundant but less certain. Hybrid loss functions combine both signals, assigning higher weight to explicit feedback. This approach was crucial in the Netflix Prize competition, demonstrating that combining rating data with viewing behavior improved prediction quality beyond using either signal alone, though the academic metric didn't fully align with business objectives.

Explicit feedback comes directly from users (thumbs up/down, star ratings). Implicit feedback is gathered by observing user behavior: dwell time (how long users read the answer), copying behavior, and editing choices (which options users select and modify). Implicit feedback often provides more nuanced signals about what actually works.

Implicit feedback (like watch duration) is a continuous variable, making it suitable for regression problems where the model predicts a continuous value. Explicit feedback (like likes/dislikes) is categorical, making it suitable for binary classification problems where the model predicts a class (like or dislike). This distinction affects how recommendation models are designed and trained. Implicit feedback has interpretation challenges—long watch duration might indicate genuine interest, but could also mean the user was distracted, fell asleep, or was trying to figure out why they didn't like the content. These ambiguities make implicit feedback less reliable but more abundant.

Explicit feedback is direct user ratings (e.g., 1-5 stars, reviews) clearly indicating preference. Implicit feedback is indirect signals (e.g., purchases, views, clicks) that may or may not indicate positive preference. Explicit feedback provides clear positive/negative signals, while implicit feedback requires interpretation and may contain ambiguous signals.