GNNs: PyTorch Geometric, Molecules & Anomalies

Learning Goal: Develop and train Graph Neural Networks (GNNs) using PyTorch Geometric to predict molecular properties and identify network anomalies.

• Estimated Total Study Time: 32 Hours
• Prerequisites: Intermediate Python proficiency, basic understanding of linear algebra (matrices, vectors), and basic machine learning concepts (supervised learning, classification vs. regression).

---

Module 1: Python & Deep Learning Foundations

This module establishes the core computational foundations required for geometric deep learning. You will master the fundamentals of neural network architectures, multi-dimensional tensor operations, backpropagation mechanics, and model building using PyTorch.

Recommended Videos

Neural Networks Explained: Architecture, Weights, and Biases

• Why this video: This video offers an industry-standard visual intuition of how neural networks act as multi-layer mathematical functions. It clarifies the concepts of weights, biases, and activation functions, preparing you to understand the node-wise transformations inside Graph Neural Networks.
• Knowledge Checkpoint:
  • Understand the role of activation functions in introducing non-linearity.
  • Explain how weights and biases scale and shift signals across layers.
  • Describe how a high-dimensional input vector gets mapped to an output classification vector.

PyTorch Crash Course: Deep Learning Framework Fundamentals

• Why this video: This comprehensive walkthrough provides hands-on familiarity with PyTorch tensors, autograd (automatic differentiation), and custom neural network definitions using torch.nn.Module.
• Knowledge Checkpoint:
  • Initialize, reshape, and run basic arithmetic operations on PyTorch tensors.
  • Explain how loss.backward() calculates gradients and how optimizer.step() updates model weights.
  • Implement a basic feedforward neural network subclassing torch.nn.Module.

PyTorch Geometric Tutorial: PyTorch Basics

• Why this video: It bridges standard PyTorch structures with custom training loops, detailing the fundamental mechanics of custom PyTorch datasets (__len__ and __getitem__) and standard models.
• Knowledge Checkpoint:
  • Define custom PyTorch Dataset structures.
  • Construct training loops that cleanly separate the forward pass, loss calculation, backpropagation, and optimizer steps.
  • Save and load model state dictionaries for evaluation.

---

Module 2: Graph Theory & Data Representations

Before feeding networks into GNNs, you must understand how graphs are structurally defined and stored. This module covers graph definitions, adjacency matrices, edge lists, memory complexities, and traditional feature extraction techniques (embeddings).

Recommended Videos

Graph Representation Part 01 - Edge List

• Why this video: This concise video introduces how a graph is mathematically represented as $G = (V, E)$ and explains the "Edge List" representation, which is the foundational design behind PyTorch Geometric's edge_index format.
• Knowledge Checkpoint:
  • Define a graph mathematically using vertex sets ($V$) and edge sets ($E$).
  • Explain how an Edge List stores connections and why it represents an alternative to matrix formats.
  • Identify the memory layout difference between directed and undirected edges in a list.

Lec-30 Graphs-II

• Why this video: This video contrasts three fundamental representation paradigms: Adjacency Matrices, Adjacency Lists, and Edge Lists. It analyzes memory spatial complexity, helping you understand why deep learning on massive graphs avoids dense representations.
• Knowledge Checkpoint:
  • Differentiate between Adjacency Matrices and Adjacency Lists.
  • Explain why an adjacency matrix uses $O(V^2)$ memory and why it is inefficient for sparse graphs.
  • Calculate the degree of a node given its row in an adjacency matrix.

Stanford CS224W: ML with Graphs | Lecture 2.1 - Traditional Feature-Based Methods: Node

• Why this video: Part of Jure Leskovec’s legendary Stanford course, this lecture shows how graphs were traditionally analyzed using structural characteristics (node degree, centrality, clustering coefficient) before end-to-end representation learning.
• Knowledge Checkpoint:
  • Define the difference between node degree and degree distribution.
  • Explain how centrality metrics (eigenvector, closeness, betweenness) measure node importance.
  • Differentiate between local neighborhood structures and global positions within a graph.

---

Module 3: Core Graph Neural Networks (GNNs)

Here, you will learn the theoretical and mathematical mechanics of Graph Convolutional Networks (GCNs) and the unified framework of Message Passing. This module shifts from static features to learning dynamic, localized neighbor representations.

Recommended Videos

Graph Neural Networks - A Perspective From the Ground Up

• Why this video: This video explains how GNNs extend deep learning architectures to non-Euclidean structures. It demonstrates how CNNs are actually a specialized, regular case of spatial GNNs.
• Knowledge Checkpoint:
  • Explain why traditional ML (like CNNs or MLPs) fails on non-grid, permutation-invariant graph structures.
  • Illustrate the spatial intuition of aggregating information from immediate (1-hop) neighbors.
  • Define "Permutation Invariance" and "Permutation Equivariance" in graph operations.

Graph Convolutional Networks (GCN) | GNN Paper Explained

• Why this video: This video provides an in-depth breakdown of the seminal paper by Kipf & Welling. It breaks down the spatial-spectral mathematical formulations behind Graph Convolutional Networks.
• Knowledge Checkpoint:
  • Write and explain the GCN propagation rule: $H^{(l+1)} = \sigma(\tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2} H^{(l)} W^{(l)})$.
  • Explain the purpose of adding self-loops ($\tilde{A} = A + I_N$) inside the propagation operator.
  • Understand the role of symmetric normalization ($\tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2}$) in stopping explosion or vanishing of node feature vectors.

Theoretical Foundations of Graph Neural Networks

• Why this video: Delivered by Petar Veličković (DeepMind, co-author of Graph Attention Networks), this lecture walks through the fundamental message passing framework: Send, Receive, Aggregate, and Update.
• Knowledge Checkpoint:
  • Break down GNN computation steps into local Message, Aggregation, and Update functions.
  • Contrast why aggregate functions must be permutation-invariant (e.g., sum, mean, max).
  • Identify how stacking $K$ message-passing layers allows a node to collect structural features from $K$-hops away.

---

Module 4: Programming GNNs with PyTorch Geometric

Now that you have the theoretical basics, you will learn to implement GNNs practically. You'll set up PyTorch Geometric (PyG), handle graph tensors (x, edge_index), process structural data batches, and write GCN architectures.

Recommended Videos

PyTorch Geometric Tutorial: Data Handling in PyTorch Geometric (Part 1)

• Why this video: This video introduces PyG's core Data class. You will learn how node features, connectivity arrays, and targets are structured in tensor memory.
• Knowledge Checkpoint:
  • Describe the attributes of the torch_geometric.data.Data object (x, edge_index, y).
  • Define the shape and data type requirements of edge_index (must be a torch.long tensor of shape [2, E]).
  • Check if a graph object is directed, has isolated nodes, or contains self-loops using PyG utilities.

PyTorch Geometric Tutorial: Introduction to PyTorch Geometric (Part 2)

• Why this video: This tutorial walks you through loading standard benchmark graph datasets (like Cora) and inspects train/val/test masks for semi-supervised training.
• Knowledge Checkpoint:
  • Load built-in benchmark datasets from torch_geometric.datasets.
  • Explain how train, test, and validation masks restrict loss evaluation during training on a single large graph.
  • Check the number of node features and target classes in a loaded dataset.

Graph Neural Networks (GNN) Using PyTorch Geometric | Stanford University

• Why this video: This short video isolates and explains how the core MessagePassing base class in PyTorch Geometric dynamically coordinates message propagation based on edge_index.
• Knowledge Checkpoint:
  • Explain the purpose of overriding the forward, message, and update methods in custom layers.
  • Identify which step handles neighbor aggregation (e.g., aggr="add", "mean", or "max").
  • Describe how propagate initiates the underlying message passing loop.

Node Classification on Knowledge Graphs Using PyTorch Geometric

• Why this video: This coding-centric video shows you how to structure node classification models, run the forward pass using convolutional layers (GCNConv), and calculate classification loss.
• Knowledge Checkpoint:
  • Set up a custom GNN model featuring layers like GCNConv.
  • Use log-softmax output and negative log-likelihood loss (F.nll_loss) for multi-class classification.
  • Construct training/validation metrics tracking accuracy.

🛠️ Implementation Note on Mini-batching: In node classification on large graphs, we typically train on the entire graph at once. For graph-level tasks (like molecular property prediction in the next module), PyG pools multiple individual graphs into a single giant disjoint graph (block-diagonal adjacency matrix) using torch_geometric.loader.DataLoader. This enables efficient GPU parallelization without losing individual graph boundaries.

---

Module 5: Molecular Machine Learning & Property Prediction

In this module, you will apply GNNs to chemistry. You will learn to use RDKit to parse SMILES strings (text representations of molecules), convert them into graph datasets, and build GNN models to predict molecular characteristics.

Recommended Videos

13. Molecular Strings and Fingerprints (RDKit Tutorial)

• Why this video: This video introduces RDKit, the standard Python tool for cheminformatics. It shows how RDKit processes structural molecular data.
• Knowledge Checkpoint:
  • Describe how SMILES strings represent molecular topologies in text format.
  • Load a SMILES string and generate an RDKit molecule object in Python.
  • Differentiate between topological/structural molecular descriptions and traditional molecular fingerprints.

Data Science for Computational Drug Discovery Using Python (Part 1)

• Why this video: This tutorial demonstrates the pre-processing pipeline for drug design datasets and explains SMILES syntax, preparing you to write clean conversion pipelines.
• Knowledge Checkpoint:
  • Read and write SMILES notation (identifying representations for carbon rings, double bonds, and heteroatoms).
  • Build clean chemical data tables using pandas.
  • Understand the pipeline from chemical compound database download to model inputs.

Predict Target from External SMILES Strings - Part 9

• Why this video: This practical coding walkthrough demonstrates how a GNN predicts lipophilicity directly from SMILES chemical strings, covering graph feature generation and the inference loop.
• Knowledge Checkpoint:
  • Set up the pipeline that feeds chemical string arrays to prediction models.
  • Explain how node features are extracted from individual atoms (atomic number, hybridization, formal charge).
  • Understand how a GNN outputs a single regression score (such as water solubility or binding affinity) for an entire molecular graph.

Discovering New Molecules Using Graph Neural Networks

• Why this video: This conceptual overview details how generative and predictive GNNs evaluate and generate drug candidates. It provides industrial context for your molecular modeling projects.
• Knowledge Checkpoint:
  • Explain why treating molecules as spatial graphs is superior to linear chemical sequence formats.
  • Understand how models evaluate molecular graphs to predict target properties (toxicity, efficacy, metabolic clearance).
  • Explain the concept of using iterative graph modifications for compound design.

💡 Bridge the Gap: Custom SMILES to PyG Conversion

While the video pool covers chemical tools (RDKit) and general GNN coding, you must connect the two manually in your code. Below is the standard Python template to convert a molecular SMILES string into a PyG Data object:

import torch from rdkit import Chem from torch_geometric.data import Data

def smiles_to_graph(smiles_str, label=None): # Parse the SMILES string with RDKit mol = Chem.MolFromSmiles(smiles_str) if mol is None: return None

# 1. Extract Node Features (e.g., Atomic Number) atom_features = [] for atom in mol.GetAtoms(): # Represent node features as a list (e.g., [atomic_num, degree]) atom_features.append([atom.GetAtomicNum(), atom.GetDegree()]) x = torch.tensor(atom_features, dtype=torch.float) # 2. Extract Edges (Bonds) edge_indices = [] for bond in mol.GetBonds(): start_idx = bond.GetBeginAtomIdx() end_idx = bond.GetEndAtomIdx() # Bi-directional edges (undirected graph representation) edge_indices.append([start_idx, end_idx]) edge_indices.append([end_idx, start_idx]) edge_index = torch.tensor(edge_indices, dtype=torch.long).t().contiguous() # 3. Create PyG Data object y = torch.tensor([label], dtype=torch.float) if label is not None else None return Data(x=x, edge_index=edge_index, y=y)

Example: Ethanol

pyg_data = smiles_to_graph("CCO", label=1.45) print(pyg_data)

Output: Data(x=[9, 2], edge_index=[2, 16], y=[1])

For deeper dive projects, run a search query for: "RDKit SMILES to PyTorch Geometric graph tutorial" or `"PyTorch Geometric molecular property prediction tutorial"*.

---

Module 6: Graph Anomaly & Fraud Detection

This module focuses on network security, structural anomalies, and financial fraud. You will learn to identify suspicious actors, fraudulent transaction chains, and unusual patterns within large network graphs.

Recommended Videos

Social Network Analysis | Anomaly Detection in Networks | Part 2

• Why this video: This video introduces the core concepts of graph anomaly detection. It details the mathematical and structural criteria that define anomalous nodes, edges, or paths.
• Knowledge Checkpoint:
  • Define what constitutes a structural anomaly versus an attribute anomaly.
  • Explain how community structure detection reveals nodes that bridge unrelated clusters.
  • Describe the differences between static and dynamic (evolving) graph anomaly detection.

Graph Gurus Workshop: Double the Performance of Your Fraud Detection System

• Why this video: This video explores the business and engineering architecture of graph-based fraud systems. It demonstrates how representing entity connections reveals complex fraud rings that standard tabular ML misses.
• Knowledge Checkpoint:
  • Explain how transaction loops, shared entities, and rapid connection creation indicate fraud rings.
  • Contrast entity-relation models with standard tabular transactional modeling.
  • Illustrate how combining graph structural statistics (like PageRank) with GNNs improves classification accuracy.

Knowledge Graphs: The Path to Enterprise — Conduct Surveillance

• Why this video: A short corporate case study on network anomaly detection applied to transaction surveillance. It highlights how analyzing relations can expose financial crime and compliance issues.
• Knowledge Checkpoint:
  • Understand how subgraphs of communications and relationships reveal suspicious trader behavior.
  • Explain how heterogeneous edge types (e.g., chats, emails, trade records) are modeled in a relational graph.
  • Understand how pattern recognition is used to automate corporate compliance systems.

---

💡 Bridge the Gap: Implementing Fraud Detection on the Elliptic Dataset

Because hands-on deep learning tutorials for anomaly detection on real financial datasets are rare, you can use this practical guide to set up a model on the benchmark Elliptic Data Set (a graph of 203,769 Bitcoin transaction nodes, where some are flagged as "licit", some as "illicit" / fraud, and the rest as "unknown"):

1. The Core Approach: Fraud detection is modeled as a semi-supervised binary node classification task. Nodes are transactions, edges represent the flow of cryptocurrency, and labels are 0 (licit) and 1 (illicit).
2. Handling Class Imbalance: Fraud datasets are highly imbalanced (illicit nodes make up <10% of labeled data). You should use a weighted loss function like: $\text{Weighted BCE} = -[w \cdot y \log(p) + (1-y) \log(1-p)]$
3. Draft PyTorch Code Pattern:

import torch import torch.nn.functional as F from torch_geometric.nn import SAGEConv # GraphSAGE is highly robust to scale

class FraudSAGE(torch.nn.Module): def init(self, in_feats, hidden_feats): super(FraudSAGE, self).init() self.conv1 = SAGEConv(in_feats, hidden_feats) self.conv2 = SAGEConv(hidden_feats, 1) # Single output channel for binary classification

def forward(self, x, edge_index): # 1-hop aggregation x = self.conv1(x, edge_index) x = F.relu(x) x = F.dropout(x, p=0.3, training=self.training) # 2-hop aggregation x = self.conv2(x, edge_index) return torch.sigmoid(x) # Map output values strictly to [0, 1] range

Define training loss focusing on labeled indices (excluding "unknown" nodes)

def train_step(model, data, optimizer, pos_weight): model.train() optimizer.zero_grad() out = model(data.x, data.edge_index).squeeze()

# Only calculate loss over labeled nodes (e.g. data.train_mask) loss = F.binary_cross_entropy( out[data.train_mask], data.y[data.train_mask].float(), pos_weight=torch.tensor([pos_weight]) ) loss.backward() optimizer.step() return loss.item()

For more implementation guides, run a search query for: "PyTorch Geometric anomaly detection tutorial" or "Graph fraud detection python implementation PyG Elliptic".

---

Course Map

This map outlines the recommended progression through the modules. Modules 5 and 6 can be taken in parallel once the PyG programming foundations in Module 4 are completed.

---

Key People Index

• Jure Leskovec (Stanford University): Professor and key author of the Stanford CS224W course series. He is a leading pioneer in graph representation learning, network analysis, and structural node embeddings.
• Petar Veličković (DeepMind): Highly influential researcher in GNN architectures. He is the first author of the groundbreaking Graph Attention Networks (GAT) paper and a major advocate for Geometric Deep Learning foundations.
• Thomas Kipf & Max Welling (University of Amsterdam): Authors of the seminal 2016 paper "Semi-Supervised Classification with Graph Convolutional Networks", which bridged spectral graph theory with localized neural aggregators, popularizing modern GCNs.

---

Final Self-Assessment

Test your mastery of the material with this comprehensive self-assessment checklist:

• Can you instantiate a PyTorch tensor, transfer it to a GPU device (cuda or mps), and check its gradient history?
• Can you explain the spatial and computational complexity trade-offs of storing a massive graph as an Adjacency Matrix vs. an Edge List?
• What is the exact purpose of symmetric normalization in GCNs, and how does it prevent gradient explosion in deep layers?
• How does PyTorch Geometric coordinate batch operations for multiple disconnected graphs without mixing edge definitions?
• Given a chemical SMILES string (e.g., CN1C=NC2=C1C(=O)N(C(=O)N2C)C), what are the code steps to parse it using RDKit and build its corresponding edge_index?
• Why must global graph pooling functions (e.g., global_mean_pool) be used instead of simple node classification when predicting molecule-level properties?
• How do you represent heterogeneous graphs with multiple node/edge types in PyG, and why is this useful for anomaly detection in financial systems?
• What metrics should you track (e.g., Precision-Recall AUC) instead of standard accuracy when evaluating anomaly detection models on highly imbalanced fraud datasets?