Subworkflow configuration

For subworkflows, configuration acts on two fronts :

1. Allowing for the subworkflow itself to change behavior, such as selecting between two different algorithms or choosing to run a specific pre-processing part.
2. Configuring the behavior of its included components to fit its use-case, and allows end-users to modify it.

Define subworkflow behaviors

Caution

NEVER define a configuration logic using the content of a channel. Channels evaluate to true nonetheless, even when empty. You need to use configuration through params or task.ext, or apply nextflow operators to unravel the logic at runtime by removing data that should not be processed.

Subworkflows parameters are defined using the params configuration scope. This scope is available in all .nf files, but params defined there can be overriden by .config files at execution, or by other subworkflows that would include yours. For the preproc_anat subworkflow, there are no subworkflow parameters defined for now, but we could, for example, add one to skip the denoising step :

main.nf

...

params.preproc_anat_denoise = true

subworkflow preproc_anat {
take:
    ch_anatomical     // Structure : [ [id: string]  , path(anat_image)  ]
    ...
main:
    ...
    if (params.preproc_anat_denoise) {
        ch_denoising_nlmeans = ch_anatomical
            .join(ch_brain_mask, remainder: true)
            .map{ meta, image, mask -> [meta, image, [], mask ?: []] }

        DENOISING_NLMEANS(ch_denoising_nlmeans)
        ch_versions = ch_versions.mix(DENOISING_NLMEANS.out.versions)
        ch_anatomical = DENOISING_NLMEANS.out.image
    }

    ch_betcrop_synthbet = ch_anatomical
        .join(ch_brain_mask, remainder: true)
        ...

    ch_preproc_n4 = ch_anatomical
        .join(ch_n4_reference, remainder: true)
        ...

What is happening here ?

• The DENOISING_NLMEANS module is only run if the parameter preproc_anat_denoise is set to true. When it is, it changes the channel assigned to ch_anatomical for the output of the module.
• The other downstream components then access this variable, instead of calling DENOISING_NLMEANS.out.image directly.

Change components behaviors

Changing the behavior of included components is not as straightforward as it seems.

Modules

Modules are configured using the task.ext scope, which is not assignable in the params or inside any .nf file. Instead, those configurations have to be defined inside .config files.

Caution

A major drawback of nextflow and nf-core is that they don’t include any .config file defined inside your subworkflows or modules. This makes it impossible to automatically install them with the component. For now, you’ll have to thouroughly document them, using the procedures defined later, so users find the correct configuration files and scopes.

Create a new nextflow.config file at the root of your subworkflow directory. While this file is not included at runtime, you will document its existence and still use it profusely in tests :

nextflow.config

params {
    preproc_anat_denoise = true
    preproc_anat_n4 = true
}

As a first, you’ll want to make the number_of_coils parameter of DENOISING_NLMEANS configurable by users. This requires a new configuration parameter in params associated to task.ext.number_of_coils for the module. To do so, in the configuration file created, define a new entry for the process scope that targets this one and only module, using a process selector :

nextflow.config

params {
    preproc_anat_denoise = true
    preproc_anat_n4 = true
    preproc_anat_nlmeans_number_of_coils = 1
}

process {
    withName: "DENOISING_NLMEANS" {
        task.ext.number_of_coils = params.preproc_anat_nlmeans_number_of_coils ?: 1
    }
}

What is happening here ?

1. We define a new parameter preproc_anat_number_of_coils in the params scope, with a default value of 1, that users will use to set the actual number of coils in the receiver antenna used to acquire the MRI signal.
2. In the process scope, we define a selector that targets the DENOISING_NLMEANS module only. In it, we bind the task.ext.number_of_coils variable to the value of params.preproc_anat_nlmeans_number_of_coils, or to 1 if it is unset by users.

Caution

Selectors have pre-defined importance levels. Above, we used the process base name, which means the user still can modify its value using the same name as us or the include name in the subworkflow (for the NL-MEANS denoising in the PREPROC_ANAT subworkflow, this name is PREPROC_ANAT:DENOISING_NLMEANS).

The rule of thumb is to use the name with the least priority, so users have all means of changing the behavior as they please. In the current case, we could have devised a label to do the configuration job for preproc_anat, but the implementation would be more complex with little gains.

Subworkflows

Configuration is done using the params scope directly in the main.nf file of your subworkflow, after the sub-components inclusion and before the workflow definition. Use the same parameter names as the ones defined in the subworkflow you include and they will overwrite their values.

---

Now that the subworkflow is configurable, it’s time to document everything. Refer below for a full configuration of all included components :

main.nf

// MODULES
include { DENOISING_NLMEANS        } from '../../../modules/nf-neuro/denoising/nlmeans/main'
include { BETCROP_SYNTHBET         } from '../../../modules/nf-neuro/betcrop/synthbet/main'
include { BETCROP_ANTSBET          } from '../../../modules/nf-neuro/betcrop/antsbet/main'
include { PREPROC_N4               } from '../../../modules/nf-neuro/preproc/n4/main'
// SUBWORKFLOWS
include { ANATOMICAL_SEGMENTATION  } from '../anatomical_segmentation/main'

params.preproc_anat_denoise = true
params.preproc_anat_bet_before_n4 = true
params.preproc_anat_n4 = true

workflow PREPROC_ANAT {
take:
    ch_anatomical       // Structure : [ [id: string]  , path(anat_image)                ]
    ch_template         // Structure : [ path(anat_ref), path(brain_proba)               ]
    ch_brain_mask       // Structure : [ [id: string]  , path(brain_mask)                ], optional
    ch_synthbet_weights // Structure : [ [id: string]  , path(weights)                   ], optional
    ch_n4_reference     // Structure : [ [id: string]  , path(reference)                 ], optional
    ch_freesurferseg    // Structure : [ [id: string]  , path(aparc+aseg) , path(wmparc) ], optional
    ch_lesion           // Structure : [ [id: string]  , path(lesion)                    ], optional
    ch_fs_license       // Structure : [ path(license)                                   ], optional
main:
    ch_versions = Channel.empty()

    if (params.preproc_anat_denoise) {
        ch_denoising_nlmeans = ch_anatomical
            .join(ch_brain_mask, remainder: true)
            .map{ meta, image, mask -> [meta, image, [], mask ?: []] }

        DENOISING_NLMEANS(ch_denoising_nlmeans)
        ch_versions = ch_versions.mix(DENOISING_NLMEANS.out.versions)
        ch_anatomical = DENOISING_NLMEANS.out.image
    }

    ch_brain_pre_mask = Channel.empty()
    if (params.preproc_anat_bet_before_n4) {
        ch_betcrop_synthbet = ch_anatomical
            .join(ch_brain_mask, remainder: true)
            .filter{ meta, image, mask -> !mask }
            .join(ch_synthbet_weights, remainder: true)
            .map{ meta, image, mask, weights -> [meta, image, weights ?: []] }

        BETCROP_SYNTHBET( ch_betcrop_synthbet )
        ch_versions = ch_versions.mix(BETCROP_SYNTHBET.out.versions)
        ch_brain_pre_mask = ch_brain_mask.mix(BETCROP_SYNTHBET.out.brain_mask)
    }

    if (params.preproc_anat_n4) {
        ch_preproc_n4 = ch_anatomical
            .join(ch_n4_reference, remainder: true)
            .join(ch_brain_pre_mask, remainder: true)
            .map{ meta, image, reference, mask -> [meta, image, reference ?: [], mask ?: []] }

        PREPROC_N4( ch_preproc_n4 )
        ch_versions = ch_versions.mix(PREPROC_N4.out.versions)
        ch_anatomical = PREPROC_N4.out.image
    }

    ch_betcrop_antsbet = ch_anatomical
        .join(ch_brain_mask, remainder: true)
        .filter{ meta, image, mask -> !mask }
        .map{ meta, image, mask -> [meta, image] }
        .combine(ch_template)

    BETCROP_ANTSBET( ch_betcrop_antsbet )
    ch_versions = ch_versions.mix(BETCROP_ANTSBET.out.versions)

    ANATOMICAL_SEGMENTATION(
        ch_anatomical,
        ch_freesurferseg,
        ch_lesion,
        ch_license
    )
emit:
    ch_anatomical = ch_anatomical                      // channel: [ [id: string]      , path(image)      ]
    ch_brain_mask = BETCROP_ANTSBET.out.mask           // channel: [ [id: string]      , path(brain_mask) ]

    wm_mask = ANATOMICAL_SEGMENTATION.out.wm_mask      // channel: [ [id: string]      , path(wm_mask)    ]
    gm_mask = ANATOMICAL_SEGMENTATION.out.gm_mask      // channel: [ [id: string]      , path(gm_mask)    ]
    csf_mask = ANATOMICAL_SEGMENTATION.out.csf_mask    // channel: [ [id: string]      , path(csf_mask)   ]

    wm_map = ANATOMICAL_SEGMENTATION.out.wm_map        // channel: [ [id: string]      , path(wm_map)     ]
    gm_map = ANATOMICAL_SEGMENTATION.out.gm_map        // channel: [ [id: string]      , path(gm_map)     ]
    csf_map = ANATOMICAL_SEGMENTATION.out.csf_map      // channel: [ [id: string]      , path(csf_map)    ]
    versions = ch_versions                             // channel: [ path(versions.yml)                   ]
}

nextflow.config

params {
    preproc_anat_denoise = true
    preproc_anat_bet_before_n4 = true
    preproc_anat_n4 = true

    // Configure DENOISING_NLMEANS
    preproc_anat_nlmeans_number_of_coils = 1
    preproc_anat_nlmeans_sigma = 0.5
    preproc_anat_nlmeans_sigma_from_all_voxels = false
    preproc_anat_nlmeans_gaussian = false
    preproc_anat_nlmeans_method = "basic_sigma"

    // Configure BETCROP_SYNTHBET
    preproc_anat_betcrop_synthbet_border = null
    preproc_anat_betcrop_synthbet_nocsf = false

    // Configure PREPROC_N4
    preproc_anat_n4_knots_per_voxel = 1
    preproc_anat_n4_shrink_factor = 1

    // Configure ANATOMICAL_SEGMENTATION
    run_synthbet = false                      // Reusing the same name, we could have changed if wanted

}

process {
    withName: "DENOISING_NLMEANS" {
        task.ext.number_of_coils = params.preproc_anat_nlmeans_number_of_coils ?: 1
        task.ext.sigma = params.preproc_anat_nlmeans_sigma ?: 0.5
        task.ext.sigma_from_all_voxels = params.preproc_anat_nlmeans_sigma_from_all_voxels ?: false
        task.ext.gaussian = params.preproc_anat_nlmeans_gaussian ?: false
        task.ext.method = params.preproc_anat_nlmeans_method ?: "basic_sigma"
    }

    withName: "BETCROP_SYNTHBET" {
        task.ext.border = params.preproc_anat_betcrop_synthbet_border ?: null
        task.ext.nocsf = params.preproc_anat_betcrop_synthbet_nocsf ?: false
    }

    withName: "PREPROC_N4" {
        task.ext.bspline_knot_per_voxel = params.preproc_anat_n4_knots_per_voxel ?: 1
        task.ext.shrink_factor = params.preproc_anat_n4_shrink_factor ?: 1
    }
}