hgtseq
nf-core/hgtseq
A pipeline to investigate horizontal gene transfer from NGS data
Key tools
Run it
nfclaw run hgtseq --input samplesheet.csv --outdir results --taxonomy-id <taxonomy_id> -profile dockernextflow run pipelines/hgtseq/upstream -profile docker --input samplesheet.csv --outdir results --taxonomy_id <taxonomy_id>nfclaw run hgtseq --demo --outdir results # adds the upstream test profile (-profile test,docker)Inputs
Pinned input correction. The bundled schema is an unused FASTQ template. These columns come from the pinned consumer, which reads input1 and optional input2. Paired inputs must have the same extension; BAM/CRAM input cannot have a second mate. Write the sheet explicitly.
| column | type | required | allowed values | constraints |
|---|---|---|---|---|
input1 | string (file path) | yes | matches ^\S+\.(fastq(\.gz)?|bam|cram)$ | |
input2 | string (file path) | no | matches ^\S+\.fastq(\.gz)?$ | |
sample | string | yes | matches ^\S+$ |
--input must match ^\S+\.csv$.
The samplesheet is a CSV with this header (the columns the schema requires); fill each value per the table above and reference.md (no example value is invented here):
input1,sampleAny of the optional columns above may be appended to the header when your data needs them: input2.
From data files. nfclaw cannot generate this samplesheet: the pinned consumer accepts mixed FASTQ/alignment roles that require an explicit samplesheet.
Reference genome
No reference genome is set by default: supply your own reference. The schema declares AWS iGenomes at s3://ngi-igenomes/igenomes; selecting --genome <id> uses the paths in the pipeline's reference configuration. Access to that remote source is required. To supply your own reference, use e.g. --fasta; the reference_genome_options group in reference.md lists every reference option. Set --igenomes-ignore true to disable the lookup entirely. Confirm the selected reference's paths and source in the pinned upstream/nextflow.config and upstream/conf/ before running.
Resources
Resource requests follow the pipeline's configs and selected profiles. nf-core's resource labels often assume a server — a single step can request far more memory than a workstation has, and retry rules can increase requests. If a run fails with Process requirement exceeds available memory (or CPUs), cap every request, and every retry, at what this machine actually has:
nfclaw run hgtseq --input samplesheet.csv --outdir results --taxonomy-id <taxonomy_id> -profile docker \ --limit-cpus 4 --limit-memory 15.GBnfclaw turns those into Nextflow's process.resourceLimits and passes them as a -c config — the mechanism nf-core prescribes for exactly this (docs). Set them to the machine's real capacity. --limit-time DURATION also caps how long any one task may run: a task that needs longer is stopped, and so is every retry, so set it only as a deliberate ceiling, never as a guess at how long the analysis takes. The generated config is kept in <outdir>/provenance/, so commands.sh replays the run under the same ceiling.
Nextflow engine
This release declares nextflowVersion = '!>=22.10.1'.
To pin this release's declared minimum engine — useful when a newer Nextflow has incompatible config parsing:
nfclaw run hgtseq ... --nxf-ver 22.10.1--nxf-ver is recorded in <outdir>/provenance/, so the replay uses the same engine. See known-issues.
Nextflow 22.10.1 predates process.resourceLimits (24.04.0), so --limit-* cannot be combined with this pin — nfclaw refuses it rather than run without the ceiling; cap resources with the release's own --max-cpus, --max-memory, --max-time instead.
Demo
nfclaw run hgtseq --demo --outdir results # adds the upstream test profile (-profile test,docker)This pinned consumer requires input1/input2, but its moving demo sheet now uses newer columns. For a bare bundled demo, nfclaw stages a three-sample historical CSV with immutable read/GFF URLs from test-datasets commit 800330ec8f47c93caabe8157e20dd23e243de4e1 and matched immutable FASTA/Kraken/Krona inputs from 56591d0269db196203af3b0cb2191da22384d24a. The CSV is bundled in provenance and rebased for replay; no remote sheet is rescanned. Explicit input, reference/database/data-base selectors, extra profiles or --config disable this entire recipe so reference choices are not mixed. Other releases retain their own defaults.
Samplesheet
A CSV with this header (the columns the schema requires). Fill each value per the constraints below; optional columns from the table may be appended.
input1,sample| Column | Type | Required | Allowed / constraints |
|---|---|---|---|
input1 | string (file path) | required | matches ^\S+\.(fastq(\.gz)?|bam|cram)$ |
input2 | string (file path) | optional | matches ^\S+\.fastq(\.gz)?$ |
sample | string | required | matches ^\S+$ |
Required parameters
--inputstring (file path)Path to comma-separated file containing information about the samples in the experiment.
Constraint · matches ^\S+\.csv$
--outdirstring (directory path)The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
--taxonomy-idnumberTaxID of samples used as input
--krakendbstringA local path to kraken database folder or compressed database file, or a URL to a compressed database file, in tar.gz format
--kronadbstringA local path or a URL to a .tab krona taxonomy file; it can also receive a compressed .tab file in tar.gz format
Parameter explorer
41 parameters · 6 groupsEvery parameter from the pinned nextflow_schema.json. nfclaw checks explicit values before launch; complex rules use the pinned pipeline's runtime validation. Search, filter by group, and expand any parameter for its default, allowed values and constraints.
11 parameters · 30 hidden
Generic options1
Custom MultiQC yaml file containing HTML including a methods description.
Input Output options6
Email address for completion summary.
Path to comma-separated file containing information about the samples in the experiment.
Indicates if input files are BAMs: if not, FASTQ files are assumed and alignment is run
MultiQC report title. Printed as page header, used for filename if not otherwise specified.
The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
TaxID of samples used as input
Reference Genome options3
Name of iGenomes reference.
A local path to kraken database folder or compressed database file, or a URL to a compressed database file, in tar.gz format
A local path or a URL to a .tab krona taxonomy file; it can also receive a compressed .tab file in tar.gz format
Run options1
Choose if aligner should be bwa-mem or bwa-mem2
Outputs
Results land in --outdir; the files and directory layout depend on the selected workflow, parameters and publication settings. Run metadata is normally placed in <outdir>/pipeline_info/ (execution report, software versions), when those outputs are enabled. The pinned tree includes MultiQC support; a report is produced only when its workflow step runs. nfclaw run also writes <outdir>/provenance/ with the exact params file and the run log, <outdir>/provenance/logs/run.log — the whole launch, whose last line states the outcome (Nextflow's own log is <outdir>/.nextflow.log); unless --no-provenance it adds a run manifest (pinned version, commit and exact command), input/output SHA-256 checksums, and a replayable commands.sh.
The exact output files and directory layout for this release are documented upstream: github.com/nf-core/hgtseq/blob/1.1.0/docs/output.md
Check a run — in the foreground or the background — with nfclaw status <outdir>: success, still running, how it ended (with the error), or stopped without an outcome; exit 0 success, 3 running, 1 otherwise.