Performance of Hadoop distributed file system operations in a non-native operating system

摘要

Implementations are provided herein for utilizing the two-part nature of HDFS protocol communications received in a non-native HDFS environment to use discriminative information learned in the NameNode request to make the client experience more efficient. NameNode requests can be received by any node among a cluster of nodes. It can be appreciated that in some non-native HDFS, clients can communicate with any nodes among a cluster to perform transactions and need not first communicate with a NameNode. Thus, upon receiving a NameNode request, the non-native HDFS Cluster of Nodes can prefetch data necessary from stable storage to cache memory for efficient reads and/or writes, allocate space on disk for writes, assign the client to communicate with a specific node among the cluster of nodes based on available cluster resources, etc. In this sense, a more efficient client experience can be offered for HDFS clients within a non-native HDFS environment.