基于互学习知识蒸馏的自训练小样本节点分类方法

SELF-TRAINING FEW-SHOT NODE CLASSIFICATION METHOD BASED ON MUTUAL LEARNING KNOWLEDGE DISTILLATION

  • 摘要: 现有自训练小样本节点分类方法存在两点不足:(1)从少量支持集学习的伪标签易出现错误或过度自信问题;(2)采用固定单向教师模型监督学生模型,无法依学生学习进度动态调整知识传递方式,这均削弱了自训练有效性。为此,提出基于互学习知识蒸馏的自训练小样本节点分类方法(CMLCKD-FSNC)。该方法先构建双教师模型,在基础数据集预训练图卷积网络作为主教师,在基础数据集高阶拓扑图预训练图卷积网络作为辅教师,再初始化与主教师架构一致的学生模型。通过表征蒸馏技术,将主教师所有图卷积层信息传递给学生模型;采用互学习伪标签蒸馏技术,以双教师预测的增强软标签监督学生模型伪标签软预测过程,提升伪标签质量,且教师模型可依学生模型动态调整知识分布,增强学生模型接受能力。在六个图数据集上的实验,验证了该方法的优越性。

     

    Abstract: The existing self-training few-shot node classification methods have two shortcomings: (1) pseudo labels learned from small support set are prone to errors or overconfidence issues; (2) fixed one-way teacher model is adopted to supervise student model, which cannot dynamically adjust knowledge transfer mode according to student learning progress, all of which weaken self-training effectiveness. Therefore, a self-training few-shot node classification method based on mutual learning knowledge distillation(CMLCKD-FSNC) is proposed. This method constructed dual teacher model: pre-training a graph convolutional network as main teacher on basic dataset, pre-training a graph convolutional network as auxiliary teacher on high-order topological graph of basic dataset. A student model consistent with main teacher architecture was initialized. By using representation distillation technology, all graph convolution layer information of main teacher was transferred to student model. Mutual learning pseudo label distillation technology was adopted to supervise student model pseudo label soft prediction with dual teacher prediction enhanced soft labels, improving pseudo label quality, and teacher model can dynamically adjust knowledge distribution according to student model to enhance student model acceptance ability. The superiority of this method was verified through experiments on six graph datasets.

     

/

返回文章
返回