Output Shaping & Set Handling

This page describes how the four set-handler classes convert grouped BidsFile lists into the flat output maps emitted into the Nextflow channel.

The handler hierarchy

BaseSetHandler is an abstract class that owns the generic file-grouping loop. Its process() method:

  1. Groups all BidsFile objects by the loop_over entity combination (one key per subject-session, or whatever entities the config declares).

  2. For each group, iterates over files, calls findMatchingGrouping() to identify the config key and set type, and accumulates a sets map via packFileIntoSet().

  3. Applies required validation — if any mandatory named group is absent, the item is discarded.

  4. Sorts sequential entries by their sequence-entity value.

  5. Delegates to the abstract processGroup() to produce the final BidsChannelData.

  6. Pushes each BidsChannelData into a DataflowQueue.

The four concrete subclasses implement getSetConfig(), setName(), getSequenceByEntities(), and getSetIndex().

Set types in detail

PlainSetHandler — plain_set

Used for data types with a single file (or small fixed set of files) per subject: no named variants, no sequencing dimension.

Config
T1w:
  plain_set: {}
Output
[
    meta: [subject: 'sub-01', session: 'ses-01', run: 'NA'],
    T1w: [
        nii:  Path('/data/bids/sub-01/anat/sub-01_T1w.nii.gz'),
        json: Path('/data/bids/sub-01/anat/sub-01_T1w.json')
    ]
]
// Access: item.T1w.nii

NamedSetHandler — named_set

Used when the same BIDS suffix appears in multiple acquisitions distinguished by an entity value (direction, flip angle, etc.).

Config
dwi_ap:
  named_set:
    ap: {direction: dir-AP}
    pa: {direction: dir-PA}
  required: [ap, pa]
  additional_extensions: [.bval, .bvec]
  suffix_maps_to: dwi
Output
[
    meta: [subject: 'sub-01', session: 'ses-01', run: 'NA'],
    dwi_ap: [
        ap: [nii:  Path('...sub-01_dir-AP_dwi.nii.gz'),
             bval: Path('...sub-01_dir-AP_dwi.bval'),
             bvec: Path('...sub-01_dir-AP_dwi.bvec')],
        pa: [nii:  Path('...sub-01_dir-PA_dwi.nii.gz'),
             bval: Path('...sub-01_dir-PA_dwi.bval'),
             bvec: Path('...sub-01_dir-PA_dwi.bvec')]
    ]
]
// Access: item.dwi_ap.ap.nii, item.dwi_ap.pa.bval
The top-level key in the output is the config key (dwi_ap), not the BIDS suffix (dwi). This allows two different config groups to share the same file suffix via suffix_maps_to.

SequentialSetHandler — sequential_set

Used when the same suffix appears once per value of a sequence entity (echo, time-point, inversion).

Config
mese:
  sequential_set:
    by_entity: echo
Output
[
    meta: [subject: 'sub-01', session: 'ses-01', run: 'NA'],
    mese: [
        nii:  [Path('...sub-01_echo-1_MESE.nii.gz'),
               Path('...sub-01_echo-2_MESE.nii.gz')],
        json: [Path('...sub-01_echo-1_MESE.json'),
               Path('...sub-01_echo-2_MESE.json')]
    ]
]
// Access: item.mese.nii[0], item.mese.nii.size()

Use by_entities: [echo, run] when multiple entities define the sequence order. Set order: flat (default) or order: hierarchical only with by_entities; the option is ignored for single-entity by_entity sequences.

When a sequence entity (e.g. echo) is also listed in loop_over, the files spanning that dimension are collapsed into a single sequenced item and the entity’s value in meta becomes NA, so the sequential set fuses with the other results instead of producing one item per value.

MixedSetHandler — mixed_set

Combines a named dimension (acquisition, inversion) with a sequential dimension (echo, time-point). Internally MixedSetHandler delegates to the NamedSetHandler and SequentialSetHandler logic for each dimension.

Config
mpm:
  mixed_set:
    named_dimension: acquisition
    sequential_dimension: echo
    named_groups:
      MTw: {acquisition: acq-MTw}
      PDw: {acquisition: acq-PDw}
      T1w: {acquisition: acq-T1w}
  required: [MTw, PDw, T1w]
Output
[
    meta: [subject: 'sub-01', session: 'ses-01', run: 'NA'],
    mpm: [
        MTw: [nii:  [Path('...acq-MTw_echo-1_MPM.nii.gz'),
                     Path('...acq-MTw_echo-2_MPM.nii.gz')],
              json: [Path('...acq-MTw_echo-1_MPM.json'),
                     Path('...acq-MTw_echo-2_MPM.json')]],
        PDw: [ ... ],
        T1w: [ ... ]
    ]
]
// Access: item.mpm.MTw.nii[0], item.mpm.T1w.json.size()

When the sequential_dimension (e.g. run) is also listed in loop_over, the files spanning that dimension are collapsed into a single sequenced item and the entity’s value in meta becomes NA, so the mixed set fuses with the other results instead of producing one item per value. suffix_maps_to is honored, so a config key such as epi_full can target the epi suffix.

Flat vs legacy output formats

Flat format (default since 0.1.0-beta.9)

Each channel item is a single Map with:

  • meta — entity values for the grouping key.

  • One entry per config key — a nested Map whose values are single Path objects, nested named groups, or ordered List<Path> values depending on the set type.

All file paths are java.nio.file.Path objects with absolute paths, ready for Nextflow path process inputs and cloud storage URIs. When unpack_json_sidecar: true is provided, .json sidecars are emitted as parsed maps instead of paths.

Legacy format (opt-in via flatten_output: false)

Channel.fromBIDS(bids_dir, config, [flatten_output: false])

Emits [groupingKey, enrichedDataMap] tuples where file paths are relative strings. Use only when migrating incrementally from pre-beta.9 code.

The legacy format will be removed in v0.2.0. See BIDS Migration Guide.

Cross-modal broadcasting

include_cross_modal copies data from one suffix group into the same emitted flat item as another suffix. For example, adding a T1w alongside every DWI item:

dwi_ap:
  named_set: { ... }
  include_cross_modal: [T1w]

The resulting channel item exposes both keys at the top level:

[
    meta: [subject: 'sub-01', session: 'ses-01', run: 'NA'],
    dwi_ap: [ap: [nii: Path('...')], pa: [nii: Path('...')]],
    T1w:    [nii: Path('...sub-01_T1w.nii.gz')]
]

In practice that means downstream code accesses item.dwi_ap.ap.nii and item.T1w.nii from the same channel element.

Related pages