基于信息增强多头注意力的多模态情感分析

MULTIMODAL SENTIMENT ANALYSIS BASED ON INFORMATION ENHANCED MULTI-HEAD ATTENTION

  • 摘要: 针对多模态情感分析方法中存在平等对待不同模态特征、各模态信息未能被充分挖掘和融合、情感分类准确率不高等问题,提出一种基于信息增强多头注意力的多模态情感分析模型。该模型通过多任务学习,挖掘其他两种非文本特征 (语音特征和视觉特征) 相对于文本特征的共享语义,进而增强文本情感特征表示。另外,基于任意两种模态间的交互信息对最终情感预测所做贡献不同,设计多头注意力融合网络,通过合理分配语音 - 文本、视觉 - 文本和语音 - 视觉特征,融合不同模态携带的有效信息。实验结果表明,该模型在多模态情感分类任务上的表现优于现有的方法。

     

    Abstract: Aimed at some problems in multimodal sentiment analysis methods, such as equal treatment of different modal features, insufficient mining and fusion of information carried by each modal, and low accuracy of sentiment classification, a multimodal sentiment analysis model based on information enhanced multi-head attention is proposed. We used multi-task learning to mine the shared semantics of other two non-text features (speech features and visual features) relative to text features, and enhanced the representation of text emotion features. Moreover, since the interaction information of any two models made different contributions to the final sentiment prediction, we designed a multi-head attention fusion network, which could effectively fuse the effective information carried by different model through the reasonable allocation of speech-text, visual-text and speech-visual features. The experimental results show that the performance of the model in multimodal emotion classification tasks is better than the existing methods.

     

/

返回文章
返回