Unified Container Environments for Scientific Cluster Scenarios

DSpace Repository

Show simple item record

dc.contributor.author Schanzel, Benjamin
dc.contributor.author Leznik, Mark
dc.contributor.author Volpert, Simon
dc.contributor.author Domaschka, Jörg
dc.contributor.author Wesner, Stefan
dc.date.accessioned 2019-04-09T13:34:38Z
dc.date.available 2019-04-09T13:34:38Z
dc.date.issued 2019-04
dc.identifier.other 1668945932
dc.identifier.uri http://hdl.handle.net/10900/87666
dc.identifier.uri http://nbn-resolving.de/urn:nbn:de:bsz:21-dspace-876668 de_DE
dc.identifier.uri http://dx.doi.org/10.15496/publikation-29052
dc.description.abstract Providing runtime dependencies for computational workflows in shared environments, like HPC clusters, requires appropriate management efforts from users and administrators. Users of a cluster define the software stack required for a workflow to execute successfully, while administrators maintain the mechanisms to offer libraries and applications in different versions and combinations for the users to have maximum flexibility. The Environment Modules system is the tool of choice on bwForCluster BinAC for this purpose. In this paper, we present a solution to execute a workflow which relies on a software stack not available via Environment Modules on BinAC. The paper describes the usage of a containerized, user-defined software stack for this particular problem using the Singularity and Docker container platforms. Additionally, we present a solution for the reproducible provisioning of identical software stacks across HPC and non-HPC environments. The approach uses a Docker image as the basis for a Singularity container. This allows users to define arbitrary software stacks giving them the ability to execute their workflows across different environments, from local workstations to HPC clusters. This approach provides identical versions of software and libraries across all environments. en
dc.language.iso en de_DE
dc.publisher Universität Tübingen de_DE
dc.publisher Universität Tübingen de_DE
dc.subject.classification Hochleistungsrechnen de_DE
dc.subject.ddc 004 de_DE
dc.subject.other bwHPC Symposium de_DE
dc.subject.other containers en
dc.subject.other reproducibility en
dc.subject.other unified container environments en
dc.subject.other docker en
dc.subject.other Singularity en
dc.subject.other machine learning en
dc.subject.other HPC en
dc.subject.other computational workflow en
dc.title Unified Container Environments for Scientific Cluster Scenarios en
dc.type ConferencePaper de_DE
utue.publikation.fachbereich Informatik de_DE
utue.publikation.fakultaet 7 Mathematisch-Naturwissenschaftliche Fakultät de_DE
utue.opus.portal bwhpc5 de_DE

Dateien:

This item appears in the following Collection(s)

Show simple item record