Subworkflow configuration

For subworkflows, configuration acts on two fronts :

1. Allowing for the subworkflow itself to change behavior, such as selecting between two different algorithms or choosing to run a specific pre-processing part.
2. Configuring the behavior of its included components to fit its use-case, and allows end-users to modify it.

Caution

NEVER define a configuration logic using the content of a channel, such as :

if (ch_brain_mask) {
    // do something
}

Channels evaluate to true nonetheless, even when empty. You need to use either :

• configuration through an input options map
• configuration parameters (params) or module arguments (task.ext)
• logic on the channel’s content using nextflow operators

Define subworkflow behaviors

Changing the behavior of a subworkflow (the code contained in the main block) is done via an input options map. This options input is available to other subworkflows and pipelines that will include yours to customize its execution. It is a map data structure with string keys and values of any type. In the context of the preproc_anat subworkflow, no subworkflow options are defined yet, but we could, for example, add one to skip the denoising step :

main.nf

...

subworkflow preproc_anat {
take:
    ch_anatomical     // Structure : [ [id: string]  , path(anat_image)  ]
    ...
    options           // Structure : map(options), optional
main:
    def preproc_anat_denoise = options.withDefault{ true }.preproc_anat_denoise as Boolean
    ...
    if (preproc_anat_denoise) {
        ch_denoising_nlmeans = ch_anatomical
            .join(ch_brain_mask, remainder: true)
            .map{ meta, image, mask -> [meta, image, [], mask ?: []] }

        DENOISING_NLMEANS(ch_denoising_nlmeans)
        ch_versions = ch_versions.mix(DENOISING_NLMEANS.out.versions)
        ch_anatomical = DENOISING_NLMEANS.out.image
    }

    ch_betcrop_synthbet = ch_anatomical
        .join(ch_brain_mask, remainder: true)
        ...

    ch_preproc_n4 = ch_anatomical
        .join(ch_n4_reference, remainder: true)
        ...

What is happening here ?

• The DENOISING_NLMEANS module is only run if the option preproc_anat_denoise is set to true in the input options map. When it is, it changes the channel assigned to ch_anatomical for the output of the module.
• We want the DENOISING_NLMEANS to run by default. To do so, we use the Groovy withDefault method to temporarily set preproc_anat_denoise to true if it is absent from the options map.
• The other downstream components then access this variable, instead of calling DENOISING_NLMEANS.out.image directly.

Setting defaults from metadata

You’ll edit the subworkflow metadata later. For now, you only need to know that you will define all option’s default values there. To ease linking those defaults in your subworkflow, we created an utility, utils_options. In the context of preproc_anat, it looks like this :

main.nf

...
import { UTILS_OPTIONS } from '../utils_options/main'

workflow PREPROC_ANAT {
take:
    ch_anatomical     // Structure : [ [id: string]  , path(anat_image)  ]
    ...
    options           // Structure :  map(options), optional
main:
    UTILS_OPTIONS(options, "${moduleDir}/meta.yml")
    options = UTILS_OPTIONS.out.options

    ...
}

Change components behaviors

Changing the behavior of included components is not as straightforward as it seems.

Modules

Modules are configured using the task.ext scope, which is not assignable in the params or inside any .nf file. Instead, those configurations have to be defined inside .config files.

Caution

A major drawback of nextflow and nf-core is that they don’t include any .config file defined inside your subworkflows or modules. This makes it impossible to automatically install them with the component. For now, you’ll have to thouroughly document them, using the procedures defined later, so users find the correct configuration files and scopes.

Create a new nextflow.config file at the root of your subworkflow directory. While this file is not included at runtime, you will document its existence and still use it profusely in tests :

nextflow.config

params {

}

As a first, you’ll want to make the number_of_coils parameter of DENOISING_NLMEANS configurable by users. This requires a new configuration parameter in params associated to task.ext.number_of_coils for the module. To do so, in the configuration file created, define a new entry for the process scope that targets this one and only module, using a process selector :

nextflow.config

params {
    preproc_anat_nlmeans_number_of_coils = 1
}

process {
    withName: "DENOISING_NLMEANS" {
        task.ext.number_of_coils = params.preproc_anat_nlmeans_number_of_coils ?: 1
    }
}

What is happening here ?

1. We define a new parameter preproc_anat_number_of_coils in the params scope, with a default value of 1, that users will use to set the actual number of coils in the receiver antenna used to acquire the MRI signal.
2. In the process scope, we define a selector that targets the DENOISING_NLMEANS module only. In it, we bind the task.ext.number_of_coils variable to the value of params.preproc_anat_nlmeans_number_of_coils, or to 1 if it is unset by users.

Caution

Selectors have pre-defined importance levels. Above, we used the process base name, which means the user still can modify its value using the same name as us or the include name in the subworkflow (for the NL-MEANS denoising in the PREPROC_ANAT subworkflow, this name is PREPROC_ANAT:DENOISING_NLMEANS).

The rule of thumb is to use the name with the least priority, so users have all means of changing the behavior as they please. In the current case, we could have devised a label to do the configuration job for preproc_anat, but the implementation would be more complex with little gains.

Subworkflows

Configuration is done using the input options map. Simply pass the map to child subworkflows for it to unpack and apply.

---

Now that the subworkflow is configurable, it’s time to document everything. Refer below for a full configuration of all included components :

main.nf

// MODULES
include { DENOISING_NLMEANS        } from '../../../modules/nf-neuro/denoising/nlmeans/main'
include { BETCROP_SYNTHBET         } from '../../../modules/nf-neuro/betcrop/synthbet/main'
include { BETCROP_ANTSBET          } from '../../../modules/nf-neuro/betcrop/antsbet/main'
include { PREPROC_N4               } from '../../../modules/nf-neuro/preproc/n4/main'
// SUBWORKFLOWS
include { ANATOMICAL_SEGMENTATION  } from '../anatomical_segmentation/main'
// UTILITY
include { UTILS_OPTIONS } from '../utils_options/main'

workflow PREPROC_ANAT {
take:
    ch_anatomical       // Structure : [ [id: string]  , path(anat_image)                ]
    ch_template         // Structure : [ path(anat_ref), path(brain_proba)               ]
    ch_brain_mask       // Structure : [ [id: string]  , path(brain_mask)                ], optional
    ch_synthbet_weights // Structure : [ [id: string]  , path(weights)                   ], optional
    ch_n4_reference     // Structure : [ [id: string]  , path(reference)                 ], optional
    ch_freesurferseg    // Structure : [ [id: string]  , path(aparc+aseg) , path(wmparc) ], optional
    ch_lesion           // Structure : [ [id: string]  , path(lesion)                    ], optional
    ch_fs_license       // Structure : [ path(license)                                   ], optional
    options             // Structure : map(options)                                       , optional
main:
    ch_versions = Channel.empty()
    UTILS_OPTIONS(options, "${moduleDir}/meta.yml")
    options = UTILS_OPTIONS.out.options

    if (options.preproc_anat_denoise) {
        ch_denoising_nlmeans = ch_anatomical
            .join(ch_brain_mask, remainder: true)
            .map{ meta, image, mask -> [meta, image, [], mask ?: []] }

        DENOISING_NLMEANS(ch_denoising_nlmeans)
        ch_versions = ch_versions.mix(DENOISING_NLMEANS.out.versions)
        ch_anatomical = DENOISING_NLMEANS.out.image
    }

    ch_brain_pre_mask = Channel.empty()
    if (options.preproc_anat_bet_before_n4) {
        ch_betcrop_synthbet = ch_anatomical
            .join(ch_brain_mask, remainder: true)
            .filter{ meta, image, mask -> !mask }
            .join(ch_synthbet_weights, remainder: true)
            .map{ meta, image, mask, weights -> [meta, image, weights ?: []] }

        BETCROP_SYNTHBET( ch_betcrop_synthbet )
        ch_versions = ch_versions.mix(BETCROP_SYNTHBET.out.versions)
        ch_brain_pre_mask = ch_brain_mask.mix(BETCROP_SYNTHBET.out.brain_mask)
    }

    if (options.preproc_anat_n4) {
        ch_preproc_n4 = ch_anatomical
            .join(ch_n4_reference, remainder: true)
            .join(ch_brain_pre_mask, remainder: true)
            .map{ meta, image, reference, mask -> [meta, image, reference ?: [], mask ?: []] }

        PREPROC_N4( ch_preproc_n4 )
        ch_versions = ch_versions.mix(PREPROC_N4.out.versions)
        ch_anatomical = PREPROC_N4.out.image
    }

    ch_betcrop_antsbet = ch_anatomical
        .join(ch_brain_mask, remainder: true)
        .filter{ meta, image, mask -> !mask }
        .map{ meta, image, mask -> [meta, image] }
        .combine(ch_template)

    BETCROP_ANTSBET( ch_betcrop_antsbet )
    ch_versions = ch_versions.mix(BETCROP_ANTSBET.out.versions)

    ANATOMICAL_SEGMENTATION(
        ch_anatomical,
        ch_freesurferseg,
        ch_lesion,
        ch_license,
        options                 // Pass the options map to child subworkflows for customization
    )
emit:
    ch_anatomical = ch_anatomical                      // channel: [ [id: string]      , path(image)      ]
    ch_brain_mask = BETCROP_ANTSBET.out.mask           // channel: [ [id: string]      , path(brain_mask) ]

    wm_mask = ANATOMICAL_SEGMENTATION.out.wm_mask      // channel: [ [id: string]      , path(wm_mask)    ]
    gm_mask = ANATOMICAL_SEGMENTATION.out.gm_mask      // channel: [ [id: string]      , path(gm_mask)    ]
    csf_mask = ANATOMICAL_SEGMENTATION.out.csf_mask    // channel: [ [id: string]      , path(csf_mask)   ]

    wm_map = ANATOMICAL_SEGMENTATION.out.wm_map        // channel: [ [id: string]      , path(wm_map)     ]
    gm_map = ANATOMICAL_SEGMENTATION.out.gm_map        // channel: [ [id: string]      , path(gm_map)     ]
    csf_map = ANATOMICAL_SEGMENTATION.out.csf_map      // channel: [ [id: string]      , path(csf_map)    ]
    versions = ch_versions                             // channel: [ path(versions.yml)                   ]
}

nextflow.config

params {
    // Configure DENOISING_NLMEANS
    preproc_anat_nlmeans_number_of_coils = 1
    preproc_anat_nlmeans_sigma = 0.5
    preproc_anat_nlmeans_sigma_from_all_voxels = false
    preproc_anat_nlmeans_gaussian = false
    preproc_anat_nlmeans_method = "basic_sigma"

    // Configure BETCROP_SYNTHBET
    preproc_anat_betcrop_synthbet_border = null
    preproc_anat_betcrop_synthbet_nocsf = false

    // Configure PREPROC_N4
    preproc_anat_n4_knots_per_voxel = 1
    preproc_anat_n4_shrink_factor = 1
}

process {
    withName: "DENOISING_NLMEANS" {
        task.ext.number_of_coils = params.preproc_anat_nlmeans_number_of_coils ?: 1
        task.ext.sigma = params.preproc_anat_nlmeans_sigma ?: 0.5
        task.ext.sigma_from_all_voxels = params.preproc_anat_nlmeans_sigma_from_all_voxels ?: false
        task.ext.gaussian = params.preproc_anat_nlmeans_gaussian ?: false
        task.ext.method = params.preproc_anat_nlmeans_method ?: "basic_sigma"
    }

    withName: "BETCROP_SYNTHBET" {
        task.ext.border = params.preproc_anat_betcrop_synthbet_border ?: null
        task.ext.nocsf = params.preproc_anat_betcrop_synthbet_nocsf ?: false
    }

    withName: "PREPROC_N4" {
        task.ext.bspline_knot_per_voxel = params.preproc_anat_n4_knots_per_voxel ?: 1
        task.ext.shrink_factor = params.preproc_anat_n4_shrink_factor ?: 1
    }
}