tailieunhanh - Ant colony optimization based scheduling for load balancing in partition and aggregation of map reduce for big data applications

In our proposed system, introduced new meta heuristic algorithm to perform the load balancing. The Ant Colony Optimization algorithm (ACO) is a probabilistic technique for solving computational problems which can be reduced to finding good paths through graphs. | ISSN:2249-5789 R Bahirathy et al, International Journal of Computer Science & Communication Networks,Vol 6(2),54-60 Ant Colony Optimization based scheduling for load balancing in partition and Aggregation of Map Reduce for Big Data Applications Student Deptartment of Computer science Engineering PTR Engineering college, e-mail bahirathyphd@ Basha Head of the Department Department of computer science Engineering PTR Engineering college, e-mail jjbasha@ Assitant Professor Department of computer science Engineering PTR Engineering college, e-mail svijimecse@ Abstract MapReduce is a framework for processing and managing large scale data sets in a distributed cluster, which has been used for applications such as generating search indexes, document clustering, access log analysis, and various other forms of data analytics. In existing system, a hash function is used to partition intermediate data among reduce tasks. In this project the system proposed a decomposition-based distributed algorithm to deal with the large-scale optimization problem for big data application and an online algorithm is also designed to adjust data partition and aggregation in a dynamic manner. In our proposed system, introduced new meta heuristic algorithm to perform the load balancing. The Ant Colony Optimization algorithm (ACO) is a probabilistic technique for solving computational problems which can be reduced to finding good paths through graphs. Ant colony optimization is random optimization search approach that will be used for allocating the incoming jobs to the virtual machines. 1. Introduction MapReduce [1] [2] [3] has appeared as the very popular calculating framework for big data processing appropriate to its simple programming model and automatic parallel execution. MapReduce and its open source fulfillment Hadoop [4] [5] have been used by many big companies, such as Yahoo!, Google and Facebook, fordifferent big data