Basic Understand of GCN

Reference:

1. https://tkipf.github.io/graph-convolutional-networks/
2. https://blog.youkuaiyun.com/chanbo8205/article/details/119275065

Input & Output

Takes as input:

• A feature description $x_i$ for every node $i$; summarized in a $N\times D$ feature matrix $X$ ($N$: number of nodes, $D$: number of input features)
• A representative description of the graph structure in matrix form; typically in the form of an adjacency matrix $A$ (or some function thereof)
  Produces a node-level output $Z$ (an $N\times F$ feature matrix, where $F$ is the number of output features per node)

Basic Understanding of GCN

Reference: Kipf & Welling (ICLR 2017)

Every neural network layer can then be written as a non-linear function
$$H^{(L+1)}=f(H^{(L)},A)$$
with $H^{(0)}=X$ and $H^{(L)}=Z$ (or $Z$ for graph-level outputs), $L$ being the number of layers. The specific models then differ only in how $f(\cdot ,\cdot )$ is chosen and parameterized.
Refer to this article for a simple to complex model process: Kipf & Welling (ICLR 2017)

Notes Below:

For a model with out_features=2, the activation layer of softmax and the cross entropy function are required.

class GCN(nn.Module):  
    def __init__(self, in_features, hidden_features, out_features):  
        super(GCN, self).__init__()  
        self.conv1 = GCNConv(in_features, hidden_features)  
        self.conv2 = GCNConv(hidden_features, out_features)  
  
    def forward(self, x, edge_list, device, edge_attr=None, node_mask=None):  
        x = F.relu(self.conv1(x, edge_list, device, edge_attr))  
        x = self.conv2(x, edge_list, device, edge_attr)  
        x = torch.softmax(x, dim=-1)  
        return x
model = GCN(in_features=10, hidden_features=32, out_features=2).to(device)
criterion = nn.CrossEntropyLoss()

For a model with out_features=1, the activation layer of sigmoid and the BCELoss function are required.

class GCN(nn.Module):  
    def __init__(self, in_features, hidden_features, out_features):  
        super(GCN, self).__init__()  
        self.conv1 = GCNConv(in_features, hidden_features)  
        self.conv2 = GCNConv(hidden_features, out_features)  
  
    def forward(self, x, edge_list, device, edge_attr=None, node_mask=None):  
        x = F.relu(self.conv1(x, edge_list, device, edge_attr))  
        x = self.conv2(x, edge_list, device, edge_attr)  
        x = torch.sigmoid(x)
        return x
model = GCN(in_features=10, hidden_features=32, out_features=1).to(device)
criterion = nn.BCELoss()

In contrast:
If you use the binary_cross_entropy function, it usually expects the input and target tensor shapes to match exactly, and for binary classification problems, the input tensor should usually be (N, 1), the target tensor should also be (N, 1), and an error will be reported if there is no match.
CrossEntropyLoss is more suitable for multi-classification problems. When out_feature is 2, CrossEntropyLoss can handle the shape difference between the input and the target tensor well, avoiding the problem of shape error (inner softmax).

LSTM-GCN Model (Brief)

• LSTM is good at handling sequential data, capturing temporal dependencies.
• GCN is effective for graph-structured data, learning node features by aggregating information from neighbors.
  In the example code above, the input is a two-dimensional tensor, if the input is a sequence:
  We need to convert the data into two dimensions before entering it into GCN, and the easiest way is through the LSTM model.
  LSTM processes sequential data first, generating features. Then, these features are fed into GCN, which exploits graph structure.