随着多核计算机的发展,多核PC机已经可以完成许多大规模计算任务,然而面对海量数据的处理,存储器与辅助存储器内的数据读取往往会成为提高应用程序运行速度的瓶颈,从而不能很好运用多核系统带来的优越硬件性能.基于多核环境的海量数据快速提取方法,以内存映射文件方法为基础并运用基于视图映射粒度的划分方式和动态与静态结合的负载均衡策略,实现了在多核平台下针对海量数据的高速并行提取.实验表明,此种方法充分发挥了多核环境的特点,并在数据量增大的情况下有良好的适应性.%With the development of multi-core computers, many large-scale computing tasks can be completed by multi-core PC. But facing the massive data processing, reading data from memory and secondary memory often becomes the bottleneck of improving application speed. So it can not make good use of the excellent hardware performance of the multi-core systems. Based on quick-read means of mass data in multi-core environment and storage mapping file technique, static load balancing, dynamic load balancing and a division method based on the size of view map are used to achieve highspeed parallel extraction of mass data in multi-core platform. Experiments show that this method makes full use of the characteristics of multi-core environment, and has good adaptability in the case of increasing the amount of data.
展开▼
