From Layers to a Classifier
Introduction
Project introduction
From Layers to a Classifier
You have written every piece a graph classifier needs, namely scoring one split, the normalized adjacency matrix, a GCN layer, a neighbor-mean layer, and a measure of how alike the nodes have become.
In the notebook these pieces run on the real Cora citation graph in PyTorch. You will train a words-only baseline first, then see how much averaging over citations adds, with and without any training, and then train a two-layer GCN.
What you will see
Averaging each paper with its neighbors once or twice lifts test accuracy from about 60% to about 80%. Averaging 128 times drops it to about 41%, because every paper in a connected group drifts toward the same vector. A trained 8-layer GCN also falls apart, partly for the same reason and partly because deep networks are harder to train.
You will fix the deep network with residual connections, which let each layer add to a paper's vector instead of replacing it, and finish by comparing three models across five seeds, choosing on validation and reporting test once.
One paper, many messages
2,708 Cora papersPaper 0
How the notebook is organized
01
Baseline and GCN
Words only, then averaging, then a trained two-layer GCN.
02
Break and fix
Measure over-smoothing as depth grows, then add residual connections.
03
Judge honestly
Compare models across five seeds on the fixed split.
Classify Cora and break it
Run the whole investigation in the browser, about 45 minutes.