Big Data Analytics for Rainfall Prediction using MapReduce-Based Regression Model

Abstract

The most significant climatic element which impacts on agriculture sector is rainfall and rainfall prediction becomes an important issue in agriculture country like Myanmar. Collecting, storing and processing of huge amount of climatic data (Big Data) require high-performance analytical systems running on distributed environments for accurate prediction of weather. Traditional standard data analytics algorithms need to be adapted to take advantage of cloud computing models which provide scalability and flexibility. In this paper, Multiple Linear Regression which is an empirical, statistical and mathematically mature method in data analysis is applied in Rainfall Prediction. To prove conventional Multiple Linear Regression work efficiently in distributed environments, we propose a parallel processing of Regression Model called MapReduce-based Multiple Linear Regression (MRMLR). Weekly Rainfall Prediction with the proposed regression model using large scale weather data will base on the QR Decomposition and Ordinary Least Squares method adapted to MapReduce Framework. Correlation-based Filter Approach by using Symmetrical Uncertainty (SU) will be applied in selecting correlated and relevant features for improving the proposed regression model’s prediction accuracy.