Report on workflow analysis for specific LAM applications
Van Bever, Joris
Earth and related Environmental sciences
Numerical Weather Prediction
MetadataShow full item record
Deliverable D4.6 for project ESCAPE (Energy-efficient Scalable Algorithms for Weather Prediction at Exascale). In this deliverable we focus on the RMI-EPS ensemble prediction suite. We first provide a detailed report on the workflow of the suite in which 5 main categories of jobs are defined; pre-processing, lateral boundary conditions (LBCs), data assimilation, forecast and post-processing. Combined Energy and wall-clock time measurements of the entire RMI-EPS suite were performed. They indicate that the wall-clock times are relatively spread between the various defined job categories, with the forecast accounting for the largest fraction at about 35%. As far as energy consumption is concerned, the forecast part dwarfs everything else and is responsible for up to 99% of the total energy consumption. This means that energy optimizations for the forecast part will translate almost proportionally into optimizations of the whole suite, while the maximum theoretical speed-up due to forecast optimizations cannot exceed a factor of about 3/2. Therefore, in terms of energy consumption, optimizations should first focus on the forecast part. For wall-clock time performance gains, however, optimizations (and possibly additional dwarfs) can be considered for the categories outside of the forecast part. Finally, we report on our efforts to build a synthetic model of the suite through the Kronos workload simulator (cfr. The H2020 NEXTGenIO project). Such a synthetic model allows predicting the I/O and MPI behavior of the suite while subjected to hypothetical workloads on existing hardware. This is meant as a proof of concept and the necessary workflow is described without providing results of actual simulations.
Van Bever, J.; Smet, G.; Degrauwe, D. (2018-05-30). Report on workflow analysis for specific LAM applications, Tech. rep., ESCAPE, https://goo.gl/oRJaHx, 2018b.