An agricultural data storage mechanism based on HBase

摘要

With the development of agricultural space-time localisation, sensor network and cloud computing, the amount of agricultural data is increasing rapidly and the data structure becomes more complicated and changeable. Currently, the widely used agricultural database is the relational database. This database handles a large amount of data with very limited throughput and is not suitable for organisation and management of distributed data. HBase is a non-relational database of distributed file storage built on Hadoop platform. HBase is suitable for unstructured data storage database and it can handle large volume of data with high scalability. To better store agricultural big data in HBase, we propose a special agricultural data buffer structure which stores data based on the data category and a two-level indexing memory organisation strategy on HBase. The method proposed saves more than a quarter of the time compared to traditional buffering methods. Experimental results show the higher efficiency of the agricultural data buffer structure and memory-organisation strategy.