Edges
shape = (2, number edges)
g = [0,1,2],[2,3,4]
g[0] = [0,1,2] source nodes
g[1] = [2,3,4] destination nodes
Nodes & features
shape = (number of nodes, feature dimensions)
x = torch.randn(4,3)
Edge features
shape = (number of edges, feature dimensions)
x_e = torch.randn(4,2)
Message function
def message_func(g, x, x_e):
src = x[g[0]] 提取source node的feature
dst = x[g[1]] 提取edge node的feature
return src
message = message_func(g, x, x_e)
这里打个message function的比方，return source feature
Scatter / Reduce function
from torch_scatter import scatter
def reduce_func(g,message):
return scatter(message,g[1],dim=0, reduce='sum')
这里比方g是
g = torch.tensor([[0, 0, 2, 3, 1], # Source nodes
[1, 3, 1, 1, 0]], dtype=torch.long)
假设node feature x 是
x = tensor([[ 0.1784, -0.3058, 1.1049],
[-1.1339, 0.2926, -0.6656],
[-0.9919, -0.0605, -0.6803],
[ 0.4678, -1.1036, 1.0191]])
那么根据g[0], source node feature x_j是
x_j = tensor([[ 0.1784, -0.3058, 1.1049], # Node 0
[ 0.1784, -0.3058, 1.1049], # Node 0
[-0.9919, -0.0605, -0.6803], # Node 2
[ 0.4678, -1.1036, 1.0191], # Node 3
[-1.1339, 0.2926, -0.6656]]) # Node 1
每个edge的weight
weight = tensor([[1.7321], # for Edge 1
[1.0000], # for Edge 2
[0.0000], # for Edge 3
[1.7321], # for Edge 4
[1.7321]]) # for Edge 5
weighted source node feature是 x_j * weight
out = tensor([[ 0.3089, -0.5296, 1.9138], # (Edge 1: 1.7321 * Node 0 feature)
[ 0.1784, -0.3058, 1.1049], # (Edge 2: 1.0000 * Node 0 feature)
[-0.0000, -0.0000, -0.0000], # (Edge 3: 0.0000 * Node 2 feature)
[ 0.8103, -1.9114, 1.7651], # (Edge 4: 1.7321 * Node 3 feature)
[-1.9640, 0.5068, -1.1529]]) # (Edge 5: 1.7321 * Node 1 feature)
scatter的话，g[1]是[1, 3, 1, 1, 0]
那先会initialize一个和node feature同样shape的tensor -->(4,3) 为0的tensor
Destination Node 0: Receives contributions from Edge 4: Node 0 feature=[−1.9640,0.5068,−1.1529]
Destination Node 1: Receives contributions from Edges 0, 2, and 3: Node 1 feature=[0.3089,−0.5296,1.9138]+[0.8103,−1.9114,1.7651]=[1.1192,−2.4411,3.6790]
Destination Node 2: No incoming edges (only Edge 2 with weight 0): Node 2 feature=[0.0000,0.0000,0.0000]
Destination Node 3: Receives contribution from Edge 1: Node 3 feature=[0.1784,−0.3058,1.1049]
Example
class GCNConv(MessagePassing):
def __init__(self, in_channels, out_channels):
super().__init__(aggr='add') # "Add" aggregation (Step 5).
self.lin = Linear(in_channels, out_channels, bias=False)
self.bias = Parameter(torch.empty(out_channels))
self.reset_parameters()
def reset_parameters(self):
self.lin.reset_parameters()
self.bias.data.zero_()
def forward(self, x, edge_index):
# x has shape [N, in_channels]
# edge_index has shape [2, E]
print('step1',edge_index)
# Step 1: Add self-loops to the adjacency matrix.
edge_index, _ = add_self_loops(edge_index, num_nodes=x.size(0))
print('step2',edge_index)
print('x',x)
# Step 2: Linearly transform node feature matrix.
x = self.lin(x)
# Step 3: Compute normalization.
row, col = edge_index
print('row',row)
print('col',col)
deg = degree(col, x.size(0), dtype=x.dtype)
print('deg',deg)
deg_inv_sqrt = deg.pow(-0.5)
deg_inv_sqrt[deg_inv_sqrt == float('inf')] = 0
norm2 = (deg_inv_sqrt[row] * deg_inv_sqrt[col]).unsqueeze(1)
print('step3 normalize degrees',norm2)
# Step 4-5: Start propagating messages.
out = self.propagate(edge_index, x=x, norm=norm2) #会先把value传递给message，然后运用scatter合并
print(' step5 propage:', out)
# Step 6: Apply a final bias vector.
out = out + self.bias
return out
def message(self, x_j, norm): # message会分出source j 和target i 以及norm值
# x_j has shape [E, out_channels]
print('step4 normalize source feature',x_j,'norm:',norm)
# Step 4: Normalize node features.
return norm * x_j
假如我的edge_index是
g = torch.tensor([[0,0,2,3],[1,3,1,1]],dtype=torch.long)
x是
x = torch.randn(4,2)
出来的值是
x tensor([[-2.1208, -0.5321],
[-0.4745, 0.9924],
[ 0.1378, 0.6422],
[-0.6585, -1.2959]])
initialize conv layer
conv = GCNConv(2, 3)
Step 1: Add self-loops to the adjacency matrix.
初始的g为
tensor([[0, 0, 2, 3, 1],
[1, 3, 1, 1, 0]])
加完self loop变为
tensor([[0, 0, 2, 3, 1, 0, 1, 2, 3],
[1, 3, 1, 1, 0, 0, 1, 2, 3]])
加self loop的目的是为了不仅仅考虑neigbor的feature，还要加上自身node的feature
那么 source的node是
tensor([0, 0, 2, 3, 1, 0, 1, 2, 3])
target的node是
tensor([1, 3, 1, 1, 0, 0, 1, 2, 3])
根据target算出来的每个node的degree是
tensor([2., 4., 1., 2.])
上面第一个位置的2是因为target node里有两个0，第二个位置是4是因为target node里有四个1，以此类推
根据每个node的degree，
to be continue