Predicting job resource utilization based on historical data

Abstract

We present a generic prediction framework in Haskell. This framework consists of two parts. The first part introduces a fixed structure for supporting various prediction models. A few of those models have been implemented. The latter part introduces a DSL to guide the programmers in making predictions. The interpretation of this DSL can be defined in an arbitrary fashion to allow different semantics. Furthermore, we also elaborate and discuss a few examples of the framework. One of these examples is a case study at Channable. This case study tries to improve the utilization of server resources by first estimating the resource usage of a specific job and then using that estimate to schedule the job on an appropriate server. Finally, this scheduling algorithm is compared with the original algorithm by modeling a theoretical simulation.