Abstract: The traditional rumor detection methods do not consider the importance of multimodal fusion, and lack of reference to entities and their environments. To solve this problem, a rumor detection method based on multi-modal fusion and knowledge perception is proposed. This method used fast R-CNN model and pre-trained BERT model to extract image and text features respectively, and effectively combined attention mechanism with entity and entity context to achieve the purpose of rumor detection. The experimental results on the Weibo and Twitter datasets show that the proposed method is superior to the comparison method in the performance indexes of accuracy, recall, precision and F1 value, and shows excellent performance in the early detection stage.