Abstract

Contributed Talk - Splinter EScience

Friday, 17 September 2021, 14:25   (virtual ESc)

An infrastructure for the reproducible scientific workflows.

A.Khalatyan
AIP, IT/eScience/HPC

The growing data amount in astronomy is require a large amount compute resources. In order to analyze and extract the valuable scientific outcomes the modern astronomy requires not only regular algorithms and methods but also recently quite common the machine learning algorithms (ML). The ML algorithms are requiring specific accelerators such as GPUs. Those are requiring complex hardware and software infrastructure. We will present a concept infrastructure at AIP for the reproducible scientific workflows. A Cloud-based environment for the various micro services workloads based on k8s cluster with the focus on reproducibility of scientific results. The k8s cluster is an open-source system for automating deployment, scaling, and management of containerized applications. The environment will enable rapid application development, easy deployment and scaling, and long-term lifecycle maintenance for small and large teams. we will show the application of this concept to the StarHorse project.