Runners

BaseRunner

BaseRunner(run_options: RunOptions)

Bases: ABC

Abstract base class for all metapipeline runner backends.

Subclasses register themselves by defining a RUNNER_NAME class attribute. The value is the string name used to select the runner::

class MyRunner(BaseRunner):
    RUNNER_NAME = "my-runner"

    def run(self, g): ...
    def resume(self, g): ...

Registration happens automatically at class-definition time via __init_subclass__. run_metapipeline looks up the concrete class from _REGISTRY, so a new runner becomes available as soon as its module is imported, with no changes to run_metapipeline needed.

Subclasses should not override __init__. The RunOptions passed in is stored as self.run_options, available to every runner without repeating the constructor.

Source code in src/nf_meta/runner/base_runner.py

def __init__(self, run_options: RunOptions) -> None:
    if "RUNNER_NAME" not in type(self).__dict__:
        raise TypeError(
            f"{type(self).__name__} must define RUNNER_NAME = '...' as a class attribute"
        )
    self.run_options = run_options

run abstractmethod

run(g: MetaworkflowGraph) -> None

Execute every workflow in g from scratch, in dependency order. Called by run_metapipeline when run_options.resume is falsy. Exceptions are not caught by the caller -- let a failure raise rather than returning normally with partial results.

Source code in src/nf_meta/runner/base_runner.py

@abstractmethod
def run(self, g: MetaworkflowGraph) -> None:
    """Execute every workflow in `g` from scratch, in dependency order.
    Called by `run_metapipeline` when `run_options.resume` is falsy.
    Exceptions are not caught by the caller -- let a failure raise rather
    than returning normally with partial results."""

resume abstractmethod

resume(g: MetaworkflowGraph) -> None

Continue a previously interrupted run of g, skipping whatever the implementation can determine already completed successfully. Called by run_metapipeline when run_options.resume is truthy. Same exception policy as run.

Source code in src/nf_meta/runner/base_runner.py

@abstractmethod
def resume(self, g: MetaworkflowGraph) -> None:
    """Continue a previously interrupted run of `g`, skipping whatever
    the implementation can determine already completed successfully.
    Called by `run_metapipeline` when `run_options.resume` is truthy.
    Same exception policy as `run`."""

RunOptions dataclass

RunOptions(runner_name: str = 'python', cachedir: Path = Path('.nf-meta-cache'), verbose: bool = False, output_lines: int = 20, start: Optional[str] = None, target: Optional[str] = None, nf_profile: Optional[str] = None, stub: bool = False, resume: bool = False)

Runtime options that control how a metapipeline run is executed.

Constructed from CLI arguments and passed to the runner. All fields have sensible defaults so the dataclass can be instantiated without arguments for programmatic use.

Attributes:

Name	Type	Description
runner_name	str	Name of the runner backend to use. Must match a key registered in base_runner._REGISTRY.
cachedir	Path	Directory used for all per-workflow working directories, resolved-param cache files, and Nextflow work directories. Created automatically if it does not exist.
verbose	bool	Enables verbose logging output.
output_lines	int	Number of most-recent output lines shown in the live scrolling panel while a workflow step is running. Only has an effect in the Python runner.
start	Optional[str]	Workflow ID at which execution begins. Workflows topologically upstream of this node are skipped. Cross-boundary parameter references must be satisfiable from previously cached runs. None means start from all source nodes.
target	Optional[str]	Workflow ID at which execution ends. Workflows topologically downstream of this node are skipped. None means run until all sink nodes.
nf_profile	Optional[str]	Nextflow profile name (or comma-separated list of names) appended globally to every workflow invocation, taking precedence over per-workflow profile settings. Whitespace is stripped before passing to Nextflow.
stub	bool	When True, passes -stub to every Nextflow invocation so that only the stub block of each process is executed. Useful for validating pipeline structure without real computation.
resume	bool	When True, skips any workflow step whose working directory already contains a successful result (OUT.txt present, ERROR.txt absent) and passes -resume to Nextflow for workflow-level checkpoint resumption.

Built-in: Python runner

SimplePythonRunner

SimplePythonRunner(run_options: RunOptions)

Bases: BaseRunner

The default runner (RUNNER_NAME = "python"). Executes each workflow as a nextflow run/nextflow module run subprocess directly from this Python process, in DAG order, streaming output as it runs.

Caches each workflow's resolved params (after ${...} substitution) under run_options.cachedir, keyed by a content hash of the workflow's definition. resume=True skips any workflow whose working directory already recorded success, and passes Nextflow's own -resume through for the ones that do run.

Source code in src/nf_meta/runner/base_runner.py

def __init__(self, run_options: RunOptions) -> None:
    if "RUNNER_NAME" not in type(self).__dict__:
        raise TypeError(
            f"{type(self).__name__} must define RUNNER_NAME = '...' as a class attribute"
        )
    self.run_options = run_options