面向层次采样的大规模动态图表示学习策略

LARGE-SCALE DYNAMIC GRAPHS REPRESENT LEARNING STRATEGIES WITH LAYER-WISE SAMPLING

  • 摘要: 动态图神经网络传统的全批量训练方式需要计算图数据中所有的节点表示,由于目前硬件设备内存容量和计算能力有限,利用这种训练方式处理大规模的数据时存在困难。提出一种基于层相关重要性采样的动态图神经网络的表示学习策略,在动态图中选择部分节点作为初始节点,这些初始节点通过重要性概率逐层采样邻居节点,采样得到的节点包含动态图的时空邻域信息,能够保证模型训练效果。在四个公开数据集上使用三个不同模型进行实验,结果表明使用该策略进行训练可以降低硬件设备的内存开销,在一定程度上提升模型的训练效果。

     

    Abstract: The original full-batch training for dynamic graph neural networks requires calculating the representation of all the nodes in the graph, which is difficult to handle large-scale data due to the limited memory capacity and computing power of current hardware devices. This paper proposes a representation learning strategies of dynamic graph neural network based on layer dependent importance sampling. Some nodes in the dynamic graph were selected as initial nodes, and these initial nodes sampled neighboring nodes layer by layer based on importance probability. The sampled nodes contained the spatiotemporal neighborhood information of the dynamic graph, which could guarantee the model training effect. Experiments were conducted on four publicly available datasets using three different models. The results show that using this strategy for training can reduce the memory overhead of the hardware device and improve the training effect of the model to a certain extent.

     

/

返回文章
返回