基于逻辑常识的深度哈希跨模态检索方法

DEEP HASH CROSS-MODAL RETRIEVAL METHOD BASED ON LOGICAL COMMONSENSE

  • 摘要: 现有的图文检索方法只利用了实例成对数据中所包含的表面关联而忽略了外部常识的重要性,这可能会妨碍它们推理图文数据间更高级关系的能力。因此,提出一种基于逻辑常识的深度哈希跨模态检索方法,通过逻辑知识图进行抽象概念表征学习,扩充和丰富概念间的语义关联,增强模型对高层语义的解耦能力和可解释性;同时,构建共享语义空间指导图文特征的交互学习,有效减少模态间的"语义鸿沟"。在两个数据集上的实验结果验证了该算法的有效性。

     

    Abstract: Existing image and text retrieval methods only exploit the superficial correlations contained in instance pair data and ignore the importance of external commonsense, which may hinder their ability to reason about higher- level relationships between image and text data. Therefore, a deep hash cross- modal retrieval method based on logical commonsense is proposed. It enriched the semantic associations between concepts by introducing logical knowledge graph for concept representation learning, enhanced the decoupling ability of the model to the higher level of semantics and its interpretability, and reduced the 'semantic gap' between different modalities by constructing a shared semantic space for guiding the interactive learning of graphical and textual features. Experiments on two datasets verify the effectiveness of the LCH algorithm.

     

/

返回文章
返回