组队学习-图神经网络 Taks 03

1、知识梳理

1.1 图卷积神经网络

GCN数学定义为：$$\mathbf{X}^{\prime} = \mathbf{\hat{D}}^{-1/2} \mathbf{\hat{A}}\mathbf{\hat{D}}^{-1/2} \mathbf{X} \mathbf{\Theta},$$
是对称归一化矩阵，它的节点式表述为：$$\mathbf{x}^{\prime}_i = \mathbf{\Theta} \sum_{j \in \mathcal{N}(v) \cup\{ i \}} \frac{e_{j,i}}{\sqrt{\hat{d}_j \hat{d}_i}} \mathbf{x}_j$$.

1.2 图注意力神经网络

定义：$$\mathbf{x}^{\prime}_i = \alpha_{i,i}\mathbf{\Theta}\mathbf{x}_{i} +\sum_{j \in \mathcal{N}(i)} \alpha_{i,j}\mathbf{\Theta}\mathbf{x}_{j},$$

$$\alpha_{i,j} =\frac{\exp\left(\mathrm{LeakyReLU}\left(\mathbf{a}^{\top}[\mathbf{\Theta}\mathbf{x}_i \, \Vert \, \mathbf{\Theta}\mathbf{x}_j]\right)\right)}{\sum_{k \in \mathcal{N}(i) \cup \{ i \}}\exp\left(\mathrm{LeakyReLU}\left(\mathbf{a}^{\top}[\mathbf{\Theta}\mathbf{x}_i \, \Vert \, \mathbf{\Theta}\mathbf{x}_k]\right)\right)}.$$

2、使用[PyG中不同的图卷积模块在[PyG的不同数据集]上实现节点分类或回归任务。

2.1 数据获取

      本次实验使用citeseer数据集，Citeseer数据集来自CiteSeer数据库，行代表科学论文，每个属性表示一个作者。共有3327个节点，9104条边，3703个属性，分为6个类。

from torch_geometric.datasets import Planetoid
from torch_geometric.transforms import NormalizeFeatures

dataset = Planetoid(root='data/Planetoid', name='citeseer', transform=NormalizeFeatures())

print()
print(f'Dataset: {dataset}:')
print('======================')
print(f'Number of graphs: {len(dataset)}')
print(f'Number of features: {dataset.num_features}')
print(f'Number of classes: {dataset.num_classes}')

data = dataset[0]  # Get the first graph object.

print()
print(data)
print('======================')

# Gather some statistics about the graph.
print(f'Number of nodes: {data.num_nodes}')
print(f'Number of edges: {data.num_edges}')
print(f'Average node degree: {data.num_edges / data.num_nodes:.2f}')
print(f'Number of training nodes: {data.train_mask.sum()}')
print(f'Training node label rate: {int(data.train_mask.sum()) / data.num_nodes:.2f}')
print(f'Contains isolated nodes: {data.contains_isolated_nodes()}')
print(f'Contains self-loops: {data.contains_self_loops()}')
print(f'Is undirected: {data.is_undirected()}')

Dataset: citeseer():
======================
Number of graphs: 1
Number of features: 3703
Number of classes: 6

Data(edge_index=[2, 9104], test_mask=[3327], train_mask=[3327], val_mask=[3327], x=[3327, 3703], y=[3327])
======================
Number of nodes: 3327
Number of edges: 9104
Average node degree: 2.74
Number of training nodes: 120
Training node label rate: 0.04
Contains isolated nodes: True
Contains self-loops: False
Is undirected: True

2.2 MLP 分类

import torch
from torch.nn import Linear
import torch.nn.functional as F


class MLP(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(MLP, self).__init__()
        torch.manual_seed(12345)
        self.lin1 = Linear(dataset.num_features, hidden_channels)
        self.lin2 = Linear(hidden_channels, dataset.num_classes)

    def forward(self, x):
        x = self.lin1(x)
        x = x.relu()
        x = F.dropout(x, p=0.5, training=self.training)
        x = self.lin2(x)
        return x

model = MLP(hidden_channels=16)
print(model)

model = MLP(hidden_channels=16)
criterion = torch.nn.CrossEntropyLoss()  # Define loss criterion.
optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)  # Define optimizer.

def train():
  model.train()
  optimizer.zero_grad()  # Clear gradients.
  out = model(data.x)  # Perform a single forward pass.
  loss = criterion(out[data.train_mask], data.y[data.train_mask])  # Compute the loss solely based on the training nodes.
  loss.backward()  # Derive gradients.
  optimizer.step()  # Update parameters based on gradients.
  return loss

for epoch in range(1, 201):  
  loss = train()
  if epoch % 10 == 0:
    print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}')

Epoch: 010, Loss: 1.7279
Epoch: 020, Loss: 1.5793
Epoch: 030, Loss: 1.4126
Epoch: 040, Loss: 1.1490
Epoch: 050, Loss: 1.0015
Epoch: 060, Loss: 0.8061
Epoch: 070, Loss: 0.7439
Epoch: 080, Loss: 0.5477
Epoch: 090, Loss: 0.5127
Epoch: 100, Loss: 0.5078
Epoch: 110, Loss: 0.4658
Epoch: 120, Loss: 0.4191
Epoch: 130, Loss: 0.4491
Epoch: 140, Loss: 0.4426
Epoch: 150, Loss: 0.4146
Epoch: 160, Loss: 0.4921
Epoch: 170, Loss: 0.3707
Epoch: 180, Loss: 0.3892
Epoch: 190, Loss: 0.4187
Epoch: 200, Loss: 0.3789

def test():
  model.eval()
  out = model(data.x)
  pred = out.argmax(dim=1)  # Use the class with highest probability.
  test_correct = pred[data.test_mask] == data.y[data.test_mask]  # Check against ground-truth labels.
  test_acc = int(test_correct.sum()) / int(data.test_mask.sum())  # Derive ratio of correct predictions.
  return test_acc

test_acc = test()
print(f'Test Accuracy: {test_acc:.4f}')

Test Accuracy: 0.5820

2.3 GCN图节点分类器

# 训练GCN图节点分类器
model = GCN(hidden_channels=16)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)
criterion = torch.nn.CrossEntropyLoss()

def train():
  model.train()
  optimizer.zero_grad()  # Clear gradients.
  out = model(data.x, data.edge_index)  # Perform a single forward pass.
  loss = criterion(out[data.train_mask], data.y[data.train_mask])  # Compute the loss solely based on the training nodes.
  loss.backward()  # Derive gradients.
  optimizer.step()  # Update parameters based on gradients.
  return loss

for epoch in range(1, 201):
  loss = train()
  if epoch % 10 == 0:
    print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}')


Epoch: 010, Loss: 1.7346
Epoch: 020, Loss: 1.6497
Epoch: 030, Loss: 1.5451
Epoch: 040, Loss: 1.4200
Epoch: 050, Loss: 1.2842
Epoch: 060, Loss: 1.1571
Epoch: 070, Loss: 1.0097
Epoch: 080, Loss: 0.9321
Epoch: 090, Loss: 0.8222
Epoch: 100, Loss: 0.7701
Epoch: 110, Loss: 0.7429
Epoch: 120, Loss: 0.7004
Epoch: 130, Loss: 0.6668
Epoch: 140, Loss: 0.6407
Epoch: 150, Loss: 0.5929
Epoch: 160, Loss: 0.6033
Epoch: 170, Loss: 0.5322
Epoch: 180, Loss: 0.5230
Epoch: 190, Loss: 0.5473
Epoch: 200, Loss: 0.4934

# 测试GCN图节点分类器
def test():
  model.eval()
  out = model(data.x, data.edge_index)
  pred = out.argmax(dim=1)  # Use the class with highest probability.
  test_correct = pred[data.test_mask] == data.y[data.test_mask]  # Check against ground-truth labels.
  test_acc = int(test_correct.sum()) / int(data.test_mask.sum())  # Derive ratio of correct predictions.
  return test_acc

test_acc = test()
print(f'Test Accuracy: {test_acc:.4f}')

Test Accuracy: 0.7120

2.4 构造GAT节点分类神经网络

# 构造GAT节点分类神经网络
import torch
from torch.nn import Linear
import torch.nn.functional as F

from torch_geometric.nn import GATConv

class GAT(torch.nn.Module):
  def __init__(self, hidden_channels):
    super(GAT, self).__init__()
    torch.manual_seed(12345)
    self.conv1 = GATConv(dataset.num_features, hidden_channels)
    self.conv2 = GATConv(hidden_channels, dataset.num_classes)

  def forward(self, x, edge_index):
    x = self.conv1(x, edge_index)
    x = x.relu()
    x = F.dropout(x, p=0.5, training=self.training)
    x = self.conv2(x, edge_index)
    return x

Epoch: 010, Loss: 1.7339
Epoch: 020, Loss: 1.6383
Epoch: 030, Loss: 1.5224
Epoch: 040, Loss: 1.3374
Epoch: 050, Loss: 1.1420
Epoch: 060, Loss: 0.9853
Epoch: 070, Loss: 0.7943
Epoch: 080, Loss: 0.6670
Epoch: 090, Loss: 0.5781
Epoch: 100, Loss: 0.5379
Epoch: 110, Loss: 0.4792
Epoch: 120, Loss: 0.4451
Epoch: 130, Loss: 0.3572
Epoch: 140, Loss: 0.3719
Epoch: 150, Loss: 0.3148
Epoch: 160, Loss: 0.3107
Epoch: 170, Loss: 0.3433
Epoch: 180, Loss: 0.2881
Epoch: 190, Loss: 0.2794
Epoch: 200, Loss: 0.2010

# 测试GAT图节点分类器
def test():
      model.eval()
      out = model(data.x, data.edge_index)
      pred = out.argmax(dim=1)  # Use the class with highest probability.
      test_correct = pred[data.test_mask] == data.y[data.test_mask]  # Check against ground-truth labels.
      test_acc = int(test_correct.sum()) / int(data.test_mask.sum())  # Derive ratio of correct predictions.
      return test_acc

test_acc = test()
print(f'Test Accuracy: {test_acc:.4f}')

Test Accuracy: 0.6100

2.5 对比分析

      以上训练结果克制，GCN的效果最好，GAT的效果次之，MLP的效果最差。下图依次为原始节点的投影图，GCN训练后的分布图和GAT训练后的分布图。