BIDS Parsing & Dataset Normalization

This page traces the path from a BIDS directory on disk to the BidsDataset in-memory model, describing BidsParser, LibBidsShWrapper, and the model value objects.

fromBIDS sequence diagram

Overview

Parsing is a three-step pipeline:

  1. LibBidsShWrapper runs the external libBIDSsh_parse_bids_to_table shell function and captures its TSV output.

  2. BidsCsvParser converts the TSV rows into BidsFile objects.

  3. BidsParser assembles the BidsFile list into a BidsDataset, then loads participants.tsv metadata.

LibBidsShWrapper

LibBidsShWrapper bridges Groovy and Bash. It resolves the libBIDS.sh script through an ordered search:

  1. Embedded path inside the plugin installation JAR (~/.nextflow/plugins/nf-bids-<version>/lib/libBIDS.sh).

  2. Well-known relative paths (lib/, libBIDS.sh/libBIDS.sh, several ../ variants).

  3. System-wide installs (/usr/local/bin/libBIDS.sh, ~/.local/bin/libBIDS.sh).

  4. Any custom path supplied via options.libbids_sh.

The final command is built as an array (never a concatenated string) to prevent shell injection:

bash -c 'set -euo pipefail && source "$1" && libBIDSsh_parse_bids_to_table "$2" ${@:4} > "$3"' \
    bash <scriptPath> <bidsDir> <outputCsv>

Paths are validated against a character-allow-list before use; semicolons, pipes, backticks, and other metacharacters raise IllegalArgumentException.

The wrapper exposes a validateLibBidsScript(path) convenience that confirms the file exists, is readable, and contains the expected libBIDSsh_parse_bids_to_table function symbol.

Error surface

If libBIDS.sh is not found, LibBidsShWrapper throws FileNotFoundException with a multi-line diagnostic listing all search paths and remediation steps (e.g., git submodule update --init).

BidsParser

package nfneuro.plugin.parser

@Slf4j
@CompileStatic
class BidsParser {
    BidsParser(Session session) { ... }
    BidsDataset parseToDataset(String bidsDir, String libBidsShPath = null)
}

parseToDataset is the single public method. It:

  1. Calls LibBidsShWrapper.parseBidsToTable(bidsDir, libBidsShPath) to obtain a temp TSV.

  2. Passes the TSV file to BidsCsvParser.parse() to get a List<BidsFile>.

  3. Constructs a BidsDataset(bidsDir) and calls dataset.addFile(file) for each entry.

  4. Calls dataset.loadParticipants() to populate dataset.participants from participants.tsv.

The model layer

BidsFile

BidsFile represents a single file entry as parsed from the libBIDS.sh TSV. It carries:

  • path — relative path within the BIDS directory.

  • suffix — BIDS suffix (e.g. T1w, dwi, MESE).

  • extension — file extension (e.g. .nii.gz, .json).

  • A map of BIDS entity key-value pairs (subject, session, run, dir, echo, …).

BidsEntity

BidsEntity is a named entity occurrence, such as subject=sub-01 or echo=1. BidsEntityUtils.groupByEntities(files, loopOverEntities) groups BidsFile lists by the Cartesian combination of entity values named in loop_over.

BidsDataset

BidsDataset holds:

  • path / name — root directory and dataset name (from dataset_description.json).

  • description — parsed JSON dataset description.

  • files — the full List<BidsFile>.

  • participants — rows from participants.tsv as List<Map<String,String>>.

BidsChannelData

BidsChannelData is the internal carrier produced by set handlers before final channel emission. It wraps the grouped and resolved file references for one subject-session combination, and is converted to the flat or legacy output map by BidsHandler.

Related pages