基于双缓存和消息队列模式的数据一致性策略研究

DATA CONSISTENCY STRATEGY BASED ON DOUBLE CACHE AND MESSAGE QUEUE PATTERN

  • 摘要: 针对目前主流的分布式集群技术在维护数据强一致性方面存在成本较高、维护困难等问题,提出一种在低资源配置下,基于双缓存和消息队列模式的数据一致性策略(DataConsistency-DoubleRedisKafka,DC-R2K),旨在以低时延实现用户无感知的数据最终一致。分析两类数据库的存储结构,构建基于Redis的存储模型,将热点数据写入读写分离的双缓存数据库;使用消息队列对高并发请求中的读请求接收缓冲,并分流读写请求;通过消息队列完成数据的异步更新,尽可能降低数据一致的时间消耗。实验结果表明,在保证数据一致的前提下,DC-R2K相较于使用传统单缓存的应用吞吐量提升13.68%~26.96%,响应时间降低47.22%~86.75%,与使用集群服务器相比资源消耗更低,具有实际应用价值。

     

    Abstract: To address the problems of distributed clustering technology in maintaining strong data consistency with high cost and maintenance difficulties, we propose a data consistency strategy (DataConsistency-DoubleRedisKafka, DC-R2K) based on double cache and message queue mode with low resource allocation, aiming to achieve the final consistency of data without user perception with the lowest latency. This paper analyzed the data storage methods of cached databases and relational databases, and constructed a Redis-based storage model to initialize hot data to a double-cached database with read and write separation. The paper used message queue to receive buffer for read requests in high concurrent requests and divert read and write requests. The paper completed asynchronous update of data through message queue to reduce the time consumption of data consistency as much as possible. The experimental results show that DC-R2K improves the application throughput by 13.68%~26.96% and reduces the response time by 47.22%~86.75% compared with the traditional single cache, with lower resource consumption compared with the use of clustered servers, and the results have practical application value.

     

/

返回文章
返回