Plugin Bootstrap & DSL Registration

This page explains how the four classes in nfneuro.plugin wire nf-bids into Nextflow’s plugin framework, making Channel.fromBIDS() and the closure-based operators available in user workflows.

Nextflow plugin framework primer

Nextflow uses pf4j for plugin management. Each plugin JAR bundles a manifest entry that tells Nextflow which class is the plugin entry point. At runtime, Nextflow instantiates that class, then discovers service implementations via Java’s ServiceLoader.

nf-bids registers two services:

Service interface Implementation

nextflow.plugin.extension.PluginExtensionPoint

BidsExtension — exposes channel factory and operators.

nextflow.trace.TraceObserverFactory

BidsFactory — creates the BidsObserver lifecycle listener.

BidsPlugin

package nfneuro.plugin

@CompileStatic
class BidsPlugin extends BasePlugin {
    BidsPlugin(PluginWrapper wrapper) {
        super(wrapper)
    }
}

BidsPlugin is intentionally minimal. It extends pf4j’s BasePlugin and serves only as the entry point registered in the JAR manifest. All meaningful behaviour lives in BidsExtension and BidsFactory.

BidsExtension

BidsExtension extends PluginExtensionPoint and is the main user-facing API surface. It is initialised with the active Session via its init(Session) override.

@Factory — Channel.fromBIDS

@Factory
DataflowWriteChannel fromBIDS(
    String bidsDir,
    String configPath = null,
    Map options = [:]
)

The @Factory annotation causes Nextflow’s DSL engine to expose this method as Channel.fromBIDS(…​). Internally it delegates immediately to BidsChannelFactory(session).fromBIDS(bidsDir, configPath, options).

@Operator — groupTupleBy

@Operator
DataflowWriteChannel groupTupleBy(
    DataflowReadChannel source,
    Closure keyExtractor,
    Map opts = [:]
)

@Operator registers a dot-style channel extension. Nextflow injects source as the channel the operator is called on; users see: channel.groupTupleBy { …​ }.

KeyExtractor.validateKeyExtractor checks arity before constructing GroupTupleByOp.

@Operator — joinBy

@Operator
DataflowWriteChannel joinBy(
    DataflowReadChannel left,
    DataflowReadChannel right,
    Closure leftKeyExtractor,
    Closure rightKeyExtractor = null,   // defaults to leftKeyExtractor
    Map opts = [:]
)

joinBy is also registered with a convenience overload for the common shared-extractor case with options:

@Operator
DataflowWriteChannel joinBy(
    DataflowReadChannel left,
    DataflowReadChannel right,
    Closure keyExtractor,
    Map opts
)

When rightKeyExtractor is null the left extractor is reused, making single-closure usage (channel.joinBy(other) { it.field }) the common case. Both closures are key extractors: each one is called with one item from its respective channel and must return the key used for matching.

@Operator — combineBy (overloaded)

combineBy is registered with three overloads to support the single-extractor and optional-opts call patterns:

// Full signature
DataflowWriteChannel combineBy(left, right, leftClosure, rightClosure, Map opts)

// Same as full signature, without explicit opts
DataflowWriteChannel combineBy(left, right, leftClosure, rightClosure)

// Single extractor — applied to both channels
DataflowWriteChannel combineBy(left, right, Closure keyExtractor)
DataflowWriteChannel combineBy(left, right, Closure keyExtractor, Map opts)

As with joinBy, these closures are item-to-key extractors rather than pairwise association predicates.

Importing operators in a workflow

include { fromBIDS }     from 'plugin/nf-bids'
include { groupTupleBy } from 'plugin/nf-bids'
include { joinBy }       from 'plugin/nf-bids'
include { combineBy }    from 'plugin/nf-bids'

The include statement is required even though the plugin is loaded in nextflow.config. Without it Nextflow raises "Missing process or function".

BidsFactory and BidsObserver

BidsFactory implements TraceObserverFactory and its sole job is to return a list containing a BidsObserver:

@CompileStatic
class BidsFactory implements TraceObserverFactory {
    @Override
    Collection<TraceObserver> create(Session session) {
        return List.of(new BidsObserver())
    }
}

BidsObserver implements TraceObserver and prints lifecycle messages:

@Override
void onFlowCreate(Session session) {
    println "Pipeline is starting! 🚀"
}

@Override
void onFlowComplete() {
    println "Pipeline complete! 👋"
}

These are currently informational placeholders; the pattern can be extended to emit metrics or validate outputs post-run.

Related pages