nf-claw

A self-maintaining library of nf-core pipelines for AI agents and bioinformaticians. Each pipeline is an unmodified git submodule pinned to a release, with a generated skill.md that says how to run it and a reference.md that lists every parameter. The nfclaw command checks a run before it starts, records what a replay needs, chains pipelines and serves all of this to agents over MCP. Scheduled workflows move the pins to new releases once the tests and each changed pipeline's own test run pass.

The website, bmds-tue.github.io/nf-claw, shows the same content: the pipelines, their parameters, the documentation and the state of the automation when the site was last built.

What it does

nfclaw checks a run before it starts and again before it reports success. It refuses unknown flags, invalid values, unsafe file names or sample identifiers, and output directories that overlap the inputs, instead of correcting them silently. It fails a run whose inputs, configs or pipeline source changed while it ran. AGENTS.md states these rules briefly and docs/preflight.md in full; docs/architecture.md explains how they are enforced, and docs/known-issues.md where the checks stop.

The pipeline source trees are never edited. A few exact releases need corrections at run time, and each correction is recorded with the run and replayed with it: configs for fastqrepair 1.1.1, and patched working copies of Bactmap, Createtaxdb and Funcscan (source corrections).

Layout

Documentation guides

Running pipelines

Trusting results

Maintaining nf-claw

Quickstart

Clone into a path without spaces (on macOS, outside iCloud folders). You need Python 3.11+, Git, Nextflow with Java 17+, and Docker or Singularity/Apptainer.

# main moves as pins are bumped; add --branch v1.0.0 for the 1.0.0 release
git clone --recurse-submodules https://github.com/bmds-tue/nf-claw.git
cd nf-claw
python3 -m venv .venv
source .venv/bin/activate
pip install -e .           # puts nfclaw on PATH

Install it editable: nfclaw reads the pinned submodules and generated files from the checkout, from whichever directory you call it. Without installing, python3 -m runner <command> from the repository root does the same.

nfclaw list variant calling     # pipelines that match, ranked
nfclaw show rnaseq              # the pinned release's inputs, required parameters and run command
 
# validate and print the command without running it
nfclaw run rnaseq --input samplesheet.csv --outdir results -profile docker \
  --fasta /data/genome.fa --gtf /data/genes.gtf --igenomes-ignore true --check
 
# the same without --check runs it; then follow it
nfclaw status results           # success, running, or how it ended

Use your own matched FASTA and GTF. The parameters a schema marks as required do not cover every reference an analysis needs; each skill.md has a section on references. --check leaves --outdir untouched but needs the run's tools on the host, including a running Docker daemon for the Docker profile.

Other common tasks:

Provenance and replay

Unless told otherwise, nfclaw run writes a provenance bundle to <outdir>/provenance/: the run log, a replay script with its dependency guard, an archive of the pipeline source, copies of the configs, records of the parameters, checksums and files, and as much of the execution environment as could be recorded. docs/architecture.md lists every file. The bundle does not contain the input data, container images or plugins, so those must still be available for a replay. docs/replay.md covers replaying, resuming and verifying, and their limits.

Maintenance

Generated files are never edited by hand. make build regenerates every skill.md, reference.md and catalog.* from the pinned submodules, make update bumps the pins to the newest releases, and make check runs the drift gate and the tests. CONTRIBUTING.md explains each target and how to add a pipeline or a handoff rule.

How it stays current

docs/compatibility.md gives each workflow's exact scope and what fails it.

Scientific limits

nfclaw validates inputs and parameters, assembles the command, isolates the run and records its provenance. None of that shows that the results are biologically correct: a run can pass every check, exit 0 and still be wrong for the question asked. Before you rely on results:

1. Validate the pipeline, parameters and reference against benchmark data suited to your question.
2. Read the caveats nfclaw status reports for a successful run. A pipeline can exit 0 after skipping samples or ignoring failed tasks (completion_caveats in the run manifest).
3. Compare a replay with nfclaw verify --strict, and inspect files that differ with tools that understand their format.
4. Check docs/known-issues.md for the release you run.

nfclaw assess checks declared outputs and sample coverage against a contract (guide). Two demo contracts are supplied; other pipelines need their own.

Citation and attribution

The pipelines are the work of the nf-core community. nf-claw includes them unmodified as pinned submodules, and each keeps its own authors, licence and citations. The few corrections nf-claw applies at run time are documented separately.

nf-claw was created by Danilo Monge (Eberhard Karls Universität Tübingen). It adapts ideas and repository structure from ClawBio, created by Manuel Corpas (MIT licence; copyright notice in LICENSE, details in NOTICE).

If you use nf-claw in research, cite it with CITATION.cff, together with:

• Nextflow: Di Tommaso P, et al. Nextflow enables reproducible computational workflows. Nat Biotechnol 35, 316–319 (2017). doi:10.1038/nbt.3820
• nf-core: Ewels PA, et al. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol 38, 276–278 (2020). doi:10.1038/s41587-020-0439-x
• The pipeline you ran: see pipelines/<name>/upstream/CITATIONS.md.
• ClawBio: Corpas M. ClawBio: Bioinformatics-Native AI Agent Skill Library. Zenodo (2026). doi:10.5281/zenodo.19420648

nf-claw is MIT-licensed (© Danilo Monge). The nf-core pipelines are MIT-licensed, and Nextflow is Apache-2.0.