Abstract:
Dynamic resource allocation in cloud data
centers is a challenging problem. Resource
prediction is a key feature for on-demand resource
planning and efficient resource management of
dynamic workload. This requires a highly accurate
demand prediction. Hyper-parameter optimization
can largely affect the performance of the prediction
model. The process of identifying the optimal
parameters for a machine learning (ML) algorithm
involves the search for a broad range of value
combinations of parameter sets. This paper presents
a resource demand prediction model with the cloud
computational frameworks Apache™ Hadoop® and
Apache Spark™. The model is developed on the
powerful ML technique, Decision Tree (DT)
algorithm, and hyper-parameter optimization for DT
algorithm is performed to achieve the prediction
model with high accuracy. The evaluation of
prediction model is conducted on real data center
workload traces and the evaluation results show that
hyper-parameter optimization can save the
prediction error significantly.
