Abstract:
MapReduce is well-applied in high performance
computing for large scale data processing. However, as
long as the clusters grow, handling with huge amount
of intermediate data produced in the shuffle and reduce
phases (middle step of Map Reduce) have impacts
heavily upon the performance. With local aggregation
(either combiners or in-mapper), shuffling large
amounts of data can be reduced which alleviates the
reduce straggler problem. The proposed modified B+
tree based indexing algorithm is applied to reduce
intermediate data amount for output retrieval fast as
well as scalable data storage
