Abstract: Rating and review information is commonly used in the research of rating prediction recommendation models. However, previous methods for extracting text features from reviews have not fully taken into account contextual semantic information, which limits the effectiveness of rating prediction. To overcome this limitation, the GRU model with BERT pre-training was utilized to process review text information. In addition, time weighting was introduced to the GRU model to accurately capture the dynamic changes in user interests. Attention mechanisms were employed to learn the proportion of fusion between rating and review features. The experimental results demonstrate that this method significantly improves the accuracy of rating prediction compared with existing models.