For the most basic proof of concept demonstration, first choose a location

/hello` [insane_lorenz] DSL2 - revision: 1d71f857bb [master]
executor >  local (4)
[50/b2f3c8] process > sayHello (4) [100%] 4 of 4 ✔
Ciao world!

Bonjour world!

Hello world!

Hola world!


[netid@cpu11 ~]$ 

Batch Submission

Executor: local

The easiest method is placing the

nextflow ... command into a batch script and submitting it to

SLURM with sbatch. The manager process will run on the allocated compute node, and all tasks are configured to use the local executor; it's even possible to use srun in your processes to run tasks which include MPI applications.

The major benefit of this method, besides simplicity, is only the initial submission waits in a

SLURM queue; it is a good pattern for a workflow which includes a very large number of small tasks. One should not combine this method with the Nextflow '

slurm' executor because the job running the Nextflow manager is likely to end before the requested task is finished waiting in a queue.

A drawback to running the Nextflow workflow process directly inside a

SLURM job allocation

is the workflow cannot run longer than the maximum wall time available to a single job in the

SLURM QOS being used (240:00:00). This can be partially mitigated by using multiple

SLURM submissions in series and passing the -resume flag to Nextflow, but only progress for completely finished tasks will be preserved from one submission to the next.

Executor: slurm

The file nextflow.config is used to instruct the manager process to submit its tasks to

SLURM instead of running them on the local host. Place the following file in your Nextflow working directory:

params {
config_profile_description = "UArizona_HPC_PUMA"
}

singularity {
enabled = true
autoMounts = true
}

process {
executor = "slurm"
cpus = '24'
queue = 'standard'
time = '48h'
clusterOptions = '--account=bjoyce3 --job-name=nf-core

 --mem-per-cpu=5gb --cpus-per-task=24'
}

Left to the default configuration, a running Nextflow workflow manager process can generate a disruptive amount of communication requests to

SLURM; most of these configuration file entries reduce the frequency of those requests.

Inside the individual process definitions in your scripts, you will likely wish to override the clusterOptions variable to request specific resources appropriate for that task. This can be done by adding something in the pattern of clusterOptions='-q regular -t 05:30:00 -C knl' to the top of your task process blocks.

Note that we're excluding memory as an option from the process. If memory is included (e.g. memory = '96 GB') this will create a SLURM script that is improperly formatted and will result in a job that is restricted to a very small amount of memory (on the order of MB). Removing the Nextflow memory option and instead specifying --mem-per-cpu in clusterOptions will fix this issue.