Architecture Overview
This page traces the path a BIDS dataset takes through the plugin — from the user’s
Channel.fromBIDS() call to the flat map that arrives in the workflow.
Package map
The plugin is distributed as a single JAR. Its logical packages mirror distinct responsibilities:
| Package | Responsibility |
|---|---|
|
Nextflow entry points: |
|
|
|
Closure-based operators: |
|
Key extraction helpers: |
|
External-tool bridge: |
|
YAML loading and analysis: |
|
Set-type handlers: |
|
Value objects: |
|
Cross-cutting utilities: |
The BIDS data flow
Step 1 — Plugin bootstrap
When Nextflow loads the plugin it instantiates BidsPlugin (a pf4j BasePlugin), which
triggers the service-loader registration of BidsExtension and BidsFactory.
BidsFactory creates the BidsObserver that logs pipeline lifecycle events.
BidsExtension annotates its methods with @Factory and @Operator, making them
available as Channel.fromBIDS(…) and the dot-style channel operators respectively.
See Plugin Bootstrap & DSL Registration for details.
Step 2 — Pre-flight and config loading
BidsChannelFactory.fromBIDS() receives the BIDS directory path, optional config path,
and an options map. Before any parsing it:
-
Verifies the BIDS directory exists.
-
Validates the config file path if supplied.
-
Checks the
libbids_shoption path if overridden.
BidsConfigLoader then reads the YAML with SnakeYAML and immediately feeds it through
BidsConfigValidator, which enforces structural rules (exactly one set type per suffix,
valid by_entity values, etc.). BidsConfigAnalyzer scans the validated map to
determine which set-handler classes are needed.
Step 3 — libBIDS.sh execution
LibBidsShWrapper locates the libBIDS.sh script — checking the embedded plugin
installation path first, then a set of well-known fallback locations — and executes:
bash -c 'set -euo pipefail && source "$1" && libBIDSsh_parse_bids_to_table "$2" > "$3"' \
bash <script> <bids_dir> <output.tsv>The TSV output is consumed by BidsCsvParser, which creates BidsFile objects.
BidsParser collects these into a BidsDataset (which also loads participants.tsv).
Step 4 — Set-type dispatch
BidsHandler iterates over every suffix defined in the config. For each, it reads the
set-type flag discovered by BidsConfigAnalyzer and routes the relevant BidsFile
slice to the matching handler:
| Config key | Handler class | When to use |
|---|---|---|
|
|
Single file per suffix per grouping key (e.g. T1w) |
|
|
Multiple named directions/groups for one suffix (e.g. DWI AP/PA) |
|
|
Ordered series varying by an entity (e.g. echoes in MESE) |
|
|
Named groups each containing a sequential series (e.g. MPM acquisitions) |
Each handler extends BaseSetHandler, which groups files by the loop_over entities
first, then delegates to the subclass for intra-group packing.
Step 5 — Channel emission
BaseSetHandler collects BidsChannelData objects and injects them into a
DataflowQueue. BidsHandler optionally applies cross-modal broadcasting
(copying anatomical data across modalities) before the final channel is returned to the
user’s workflow.
With flatten_output: true (the default since 0.1.0-beta.9) each emitted item is a
flat Groovy map:
[
meta: [subject: 'sub-01', session: 'ses-01', run: 'NA'],
T1w: [nii: Path('/data/bids/sub-01/anat/sub-01_T1w.nii.gz'),
json: Path('/data/bids/sub-01/anat/sub-01_T1w.json')]
]Step 6 — Closure-based operators (optional)
Once the channel is emitted the user may pipe it through groupTupleBy, joinBy, or
combineBy. All three are implemented as @Operator methods on BidsExtension:
-
GroupTupleByOpaccumulates items by closure-extracted key and emits[key, [items]]. -
JoinByOpbuffers both channels, matches by key, and emits fused joined items (key omitted). -
CombineByOpproduces the cartesian product within each key group as fused combined items (key omitted).
KeyExtractor validates closures (arity > 0) and wraps List return values in
CompositeKey (an @EqualsAndHashCode-annotated class) for correct hash-map semantics.