SPARK SHUFFLE-BASED REMOTE DIRECT MEMORY ACCESS SYSTEM AND METHOD

摘要

The present invention provides a system and a method for remote direct memory access based on Spark Shuffle. The system comprises a transmitting end and a receiving end; the transmitting end comprises a transmitter, a transmitting channel and a transmitting manager; the transmitter stores the acquired transmission data into the transmitting channel, and invokes the transmitting manager; after being invoked, the transmitting manager generates checking information based on the transmission data and stores the checking information into the transmitting channel; the receiving end comprises a receiver, a receiving channel and a receiving manager; the receiver acquires the transmission data and the checking information in the transmitting channel through a remote direct memory access technique, stores the transmission data and the checking information into the receiving channel, and invokes the receiving manager; after being invoked, the receiving manager verifies the transmission data based on the checking information.