nf-core/configs: unsw_katana
Edit

UNSW Katana HPC profile.

nf-core/configs: UNSW Katana HPC Configuration

nf-core pipelines have been successfully configured for use on the UNSW Katana at the University of New South Wales, Sydney, Australia.

To run an nf-core pipeline at UNSW Katana, run the pipeline with -profile singularity,unsw_katana. This will download and launch the unsw_katana.config which has been pre-configured with a setup suitable for the unsw katana HPC cluster. Using this profile, a Singularity image containing all of the required software will be used for the pipeline.

Launch an nf-core pipeline on Katana

Prerequisites

Before running the pipeline you will need to load Nextflow and Java, both of which are globally installed modules on Katana. You can do this by running the commands below:

module purge
module load nextflow java

Execution command

nextflow run <nf-core_pipeline>/main.nf \
    -profile singularity,unsw_katana \
    <additional flags>
 
# Replace `<nf-core_pipeline>` with the name of the nf-core pipeline you want to run, e.g., `nf-core/proteinfold`.
 
### Queue limits
 
This config is defined in line with the [UNSW Katana queue limits](https://docs.restech.unsw.edu.au/using_katana/running_jobs/#job-queue-limits-summary).

Config file

See config file on GitHub

conf/unsw_katana

// UNSW Katana nf-core configuration profile
 
params {
    config_profile_name         = 'unsw_katana'
    config_profile_description  = 'UNSW Katana HPC profile provided by nf-core/configs.'
    config_profile_contact      = '@jscgh'
    config_profile_url          = 'https://docs.restech.unsw.edu.au/'
}
 
process {
    executor = 'pbspro'
 
    // Load required modules for running Nextflow
    beforeScript = 'module load nextflow java'
 
    resourceLimits = [
    memory: 248.GB,
    cpus: 24,
    time: 100.h
    ]
 
    // Basic GPU node allocation
    withLabel:process_gpu {
        accelerator     = 1
        clusterOptions  = { "-l select=1:ngpus=1:ncpus=6:mem=125gb" }
        containerOptions = '--nv'
    }
    withLabel:process_single {
        cpus   = { 1                       }
        memory = { 4.GB     * task.attempt }
        time   = { 2.h      * task.attempt }
    }
    withLabel:process_low {
        cpus   = { 2        * task.attempt }
        memory = { 12.GB    * task.attempt }
        time   = { 2.h      * task.attempt }
    }
    withLabel:process_medium {
        cpus   = { 6        * task.attempt }
        memory = { 46.GB    * task.attempt }
        time   = { 12.h     * task.attempt }
    }
    withLabel:process_high {
        cpus   = { 12       * task.attempt }
        memory = { 124.GB   * task.attempt }
        time   = { 48.h     * task.attempt }
    }
    withLabel:process_long {
        time   = { 100.h    * task.attempt }
    }
    withLabel:process_high_memory {
        memory = { 248.GB   * task.attempt }
    }
    withLabel:error_ignore {
        errorStrategy = 'ignore'
    }
    withLabel:error_retry {
        errorStrategy = 'retry'
        maxRetries    = 1
    }
}
 
// Set the cache directory for Singularity and Apptainer
singularity {
    autoMounts = true
    cacheDir = "/srv/scratch/${USER}/.images"
}
apptainer {
    enabled = true
    autoMounts = true
    cacheDir = "/srv/scratch/${USER}/.images"
}
 
// Docker is not supported on this cluster
docker {
    enabled = false
}

Pipeline configs

See config file on GitHub

conf/pipeline/proteinfold/unsw_katana

profiles {
    unsw_katana {
        params {
            config_profile_name        = 'proteinfold/unsw_katana'
            config_profile_contact     = '@jscgh'
            config_profile_description = 'nf-core/proteinfold UNSW Katana profile provided by nf-core/configs'
        }
 
        // The proteinfold images are predownloaded and stored in the singularity library directory
        apptainer.libraryDir    = "/srv/scratch/sbf-pipelines/proteinfold/singularity"
        singularity.libraryDir  = "/srv/scratch/sbf-pipelines/proteinfold/singularity"
 
        cleanup = true
 
        process {
            executor = 'pbspro'
 
            withName: 'RUN_HELIXFOLD3' {
                clusterOptions  = { "-l select=1:ngpus=1:ncpus=6:mem=62gb:gpu_model=A100" } // Using A100 GPU for compatibility
            }
            withName: 'RUN_ESMFOLD' {
                clusterOptions  = { "-l select=1:ngpus=1:ncpus=6:mem=62gb:gpu_model=A100" } // Using A100 GPU for compatibility
            }
            withName: 'RUN_ALPHAFOLD2_PRED|RUN_ALPHAFOLD2' {
                clusterOptions  = { "-l select=1:ngpus=1:ncpus=8:mem=250gb:gpu_model=H200" } // Using H200 GPU for highest performance
            }
 
        }
    }
}

nf-core/configs: unsw_katana Edit