Benchmarking AWS performance to run environmental simulations over Belgium
Key Challenges
The project of implementing the Modèle Atmosphérique Régional (MAR) for climate simulation over Belgium encountered significant challenges, primarily due to its complexity and computational demands. MAR, requiring the computational power usually reserved for supercomputers, posed difficulties in accessibility for continuous research use. Additionally, deploying this model on AWS presented initial hurdles such as the need to establish a functional cloud-based environment from scratch and determining the most cost-efficient AWS resources to optimize the balance between cost and performance.
Key Results
The partnership with Ankercloud and use of AWS EC2 instances substantially improved the University of Liège's capability to conduct frequent and detailed environmental simulations. This has greatly enhanced the accuracy and detail of weather forecasts and climate projections in Belgium. Additionally, AWS's scalable infrastructure not only lowered operational costs but also promoted sustainability by potentially reducing CO2 emissions by up to 88% compared to traditional data centers. Comprehensive training and documentation from Ankercloud enabled the university to independently run MAR, boosting their climate science research capabilities. Furthermore, by making MAR open-source, the project has widened educational prospects and increased academic engagement in climate research.
Overview
Xavier Fettweis, Professor and Researcher at the University of Liège - Department of Climatology and Topoclimatology and Head of the Laboratory of Climatology, is conducting a series of studies to estimate the impact of climate change over Belgium and perform detailed weather forecasting.
For this purpose, he is developing and improving the Modèle Atmosphérique Régional (MAR), an advanced climate model historically used to simulate the surface mass balance and melt of polar ice sheets. By applying MAR over the Belgian region, it is possible to perform future projections using IPCC scenario and make weather forecast predictions with an incredible accuracy of up to 5km, generally not otherwise reachable with other available meteorological models.
Challenges
MAR Results: Climate Change is Already Happening
While short-term MAR simulations translate into a series of practical advantages in terms of weather forecasting, on a longer time horizon they reveal the problematic reality of climate change and its impact over Belgium, highlighting a slow but constant increase in the average temperatures on the overall region (following the worrisome worldwide trend). The consequences of these changes can be devastating.
“If we don't intensify our mitigation measures, by the year 2100 Liège will have the climate of Toulouse, with summer maximum temperatures up to +40°C and more than one month of tropical night (minimum temperature above +20°C) every summer. At the same time, a world that will be 2° hotter in the next decade is now inevitable, increasing a lot the probability of severe droughts in summer in Belgium” reports Prof. Fettweis.
To increase research on the topic, MAR is also open-source and made freely available to interested Climatology master students at Uliège.
A Proof-of-Concept implementation on AWS to promote sustainability
Given the complexity of the model, running simulations requires substantial computational power (especially supercomputers) that may not always be readily accessible and/or optimized for the specific workloads.
At the same time, AWS and Cloud computing offer the possibility of having a dedicated infrastructure accessible 24/7 and right-sized to the specific technical needs, but getting started involved solving some challenges: first of all, finding out the internal resources and knowledge needed to deploy a working environment from scratch, and secondly, finding out the AWS resources type that could give the best price-performance ratio.
In this context, Uliège has collaborated with Ankercloud (AWS Advanced partner) for a Proof-of-Concept (PoC) implementation on the AWS Public Cloud, designed to enable Prof. Fettweis and his team of researchers to start using EC2 instances for their High-Performance Computing (HPC) use case.
Running more simulations more often and with more available resources will contribute to advancing the research on the field and - hopefully - increasing awareness on the climate change topic, especially in Belgium, where this research is focused.
Solution
After going through an introductory process in which Prof. Fettweis explained all the specific requirements to run his model written in Fortran 90 and parallelized in OpenMP, Ankercloud prepared a feasible project plan for the AWS adoption. The plan was to start with a completely new AWS account created for this specific purpose, so it had to consider the implementation of basic configurations from scratch, in line with the best practices.
More specifically, the following tasks have been performed:
- Setting up the Amazon Virtual Private Cloud (Amazon VPC) with subnet association, connectivity, and Security Groups, along with Identity and Access Management (IAM) and the related policies
- Deploying Linux-based EC2 instances provisioned with necessary dependencies and desired configuration (minimum 100 vCPUs and 64 Gb RAM as a starting point)
- Configuring object storage with Amazon Simple Storage Service (S3) to manage input and output files generated from the weather simulations
- Implementing CloudWatch Logs, Metrics, and Alarms for basic monitoring, to be included in the Health Dashboard for an overview of service status
After the deployment had been completed, the prototype was used to test and benchmark different instances and configurations and find a good compromise for the price performance ratio for running MAR.
Lastly, Ankercloud’s tech team gave complete Knowledge Transfer through ad-hoc sessions and extensive documentation, providing ULiège the possibility of making full use of their new setup to install and run in complete autonomy MAR as well as any other similar models in the future.
Business Outcome
At the Department of Climatology, Prof. Fettweis and his team can now benefit from a scalable and cost-effective solution on the AWS Cloud to run different meteorological simulations, used flexibly in combination with the available on-premises supercomputers, to advance the extremely important research on Climate change - with a special focus on Belgium.
AWS supported this project with Funding Programs and Promotional Credits, to lower as much as possible the initial adoption barrier and give the possibility to generate tangible results for a proper evaluation before using Cloud services more intensively.
And, in the long run, using EC2 instances over on-premises resources is itself a step ahead towards more environment-friendly energy consumption, due to the highly optimized AWS infrastructure that can lead to CO2 emissions of up to 88% less than a normal datacenter.
Moreover, such a shared AWS-based HPC model also allows to reduce our numerical footprint by optimizing the use of CPUs. This is particularly true when performing daily weather forecasts with MAR and knowing that such simulations use CPUs for only a couple of hours if a dedicated HPC cluster is used.
About Uliège
The University of Liège (ULiège) is a key university in Belgium, with more than 25.000 students, 190+ masters, and 10+ faculties, ranging in many different fields from Science and Medicine to Philosophy and Letters.
.png)
