AdaptiveFlow

General

Each AdaptiveFlow module can be installed and used individually. And it needs to be installed for each workflow that is run. Meaning, that if two workflows are to be run in parallel on a cluster, the AdaptiveFlow modules will be installed in two freely choosable folders, and the workflow will run in each of them separately.

A AdaptiveFlow module (AFLP, AFVS, and AFU) can be installed via one of two ways:

• Via a tarball (tar.gz archive)

• Via Git/GitHub

Installation Location

Before installing a AdaptiveFlow module, the filesystem for the installation has to be chosen. The filesystem has to be a shared storage that is available to all compute nodes that are used by AdaptiveFlow. Usually, there are two types of such filesystems available on a typical cluster:

1. The home filesystem

2. The scratch filesystem

When using a Linux cluster, the scratch filesystem should in almost all cases be used, because AdaptiveFlow can be I/O intensive. And the scratch filesystem is usually able to handle higher I/O loads than the home filesystem. In addition, it can be faster as well. Another reason why the home filesystem is often not suitable is that the filesystem quota per user is often very low, too low for the often large amounts of input/output data used/generated by AdaptiveFlow (only relevant if the in/output data is stored locally on the cluster rather than in the cloud).

When using AWS Batch: The home folder of the login should be used. The preconfigured AWS Batch login node that AdaptiveFlow provides uses a fast SSD.

Tarball Installation

The tar-archive versions of AdaptiveFlow modules can be downloaded from the AdaptiveFlow homepage:

• AFLP: https://adaptiveflow.org/aflp-downloads

• AFVS: https://adaptiveflow.org/afvs-downloads

• AFU:

The files can also be obtained directly via the wget command, in the case of AFLP:

wget -O AFLP.tar.gz https://github.com/AdaptiveFlow/AFLP/archive/python-develop.tar.gz

In the case of AFVS:

wget -O AFVS.tar.gz https://github.com/AdaptiveFlow/AFVS/archive/python-develop.tar.gz

And in the case of AFU:

wget -O AFU.tar.gz https://github.com/AdaptiveFlow/AFU/archive/refs/heads/main.tar.gz

After downloading the tar-archive, it can be simply installed by extracting it on the desired location. In the case of AFLP:

tar -xvzf AFLP-python-develop.tar.gz

In the case of AFVS:

tar -xvzf AFVS-python-develop.tar.gz

And in the case of AFU:

tar -xvzf AFU-main.tar.gz

Cloning the GitHub Repository

AdaptiveFlow can also be installed by cloning the GitHub repository. In the case of AFLP:

git clone [email protected]:AdaptiveFlow/AFLP.git

In the case of AFVS:

git clone [email protected]:AdaptiveFlow/AFVS.git

And in the case of AFU:

git clone [email protected]:AdaptiveFlow/AFU.git