The research and implementation of a highly concurrent and highly available system for acquiring personal virtual assets

摘要

This paper focuses on the implementation of a highly concurrent and highly available system for acquiring personal virtual assets. It briefly described ways to get mailboxes and micro-blog data by means of JavaMail and Sina, micro-blog, open API. The system uses memory database Redis as cache to improve the performance of the system and uses MongoDB to store data permanently. DAB-Redis, a new Redis cluster scheme for high availability and load balancing, is proposed. It adopts a method of dynamic allocation and recovery slot, and can automatically perform load balancing at each back-end Redis node. Using the distribute process to assign tasks equally to multiple migrate nodes to realize data migration from Redises to MongoDBs efficiently. Haproxy + Keepalived is used to implement load balancing and high availability for multiple mongos nodes.