detaxizer
nf-core/detaxizer
A pipeline to identify (and remove) certain sequences from raw genomic data. Default taxon to identify (and remove) is Homo sapiens. Removal is optional.
Key tools
Run it
nfclaw run detaxizer --input samplesheet.csv --outdir results -profile dockernextflow run pipelines/detaxizer/upstream -profile docker --input samplesheet.csv --outdir resultsnfclaw run detaxizer --demo --outdir results # adds the upstream test profile (-profile test,docker)Inputs
| column | type | required | allowed values | constraints |
|---|---|---|---|---|
sample | string | no | matches ^\S+$ | |
short_reads_fastq_1 | string (file path) | no | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ | |
short_reads_fastq_2 | string (file path) | no | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ | |
long_reads_fastq_1 | string (file path) | no | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ |
--input must match ^\S+\.csv$.
The samplesheet is a CSV with this header; fill each value per the table above and reference.md (no example value is invented here):
sample,short_reads_fastq_1,short_reads_fastq_2,long_reads_fastq_1From data files. --input may also be the data files — a directory (searched recursively), a quoted glob, or files (--input once per file) — from which nfclaw writes this samplesheet once you choose the read columns: the samplesheet takes reads in several column groups (short_reads_fastq_1 + short_reads_fastq_2; long_reads_fastq_1) and file names cannot tell which yours belong in; leave the groups you do not use empty, e.g. --sheet-set long_reads_fastq_1=. Set the required columns no file determines with --sheet-set COLUMN=VALUE. Preview it with --check; a run records it in <outdir>/provenance/ for replay.
Reference genome
This release resolves a reference genome remotely by default. --genome defaults to GRCh38; the schema declares AWS iGenomes at s3://ngi-igenomes/igenomes/. The pinned configuration gives that key the FASTA ${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa. Resolving references through that default needs access to this source. For a self-contained run, pass your own reference instead (the reference_genome_options group in reference.md lists the reference options). Set --igenomes-ignore true to disable the lookup entirely. Confirm the selected reference's paths and source in the pinned upstream/nextflow.config and upstream/conf/ before running.
Resources
Resource requests follow the pipeline's configs and selected profiles. nf-core's resource labels often assume a server — a single step can request far more memory than a workstation has, and retry rules can increase requests. If a run fails with Process requirement exceeds available memory (or CPUs), cap every request, and every retry, at what this machine actually has:
nfclaw run detaxizer --input samplesheet.csv --outdir results -profile docker \ --limit-cpus 4 --limit-memory 15.GBnfclaw turns those into Nextflow's process.resourceLimits and passes them as a -c config — the mechanism nf-core prescribes for exactly this (docs). Set them to the machine's real capacity. --limit-time DURATION also caps how long any one task may run: a task that needs longer is stopped, and so is every retry, so set it only as a deliberate ceiling, never as a guess at how long the analysis takes. The generated config is kept in <outdir>/provenance/, so commands.sh replays the run under the same ceiling.
Nextflow engine
This release declares nextflowVersion = '!>=26.04.0'.
To pin this release's declared minimum engine — useful when a newer Nextflow has incompatible config parsing:
nfclaw run detaxizer ... --nxf-ver 26.04.0--nxf-ver is recorded in <outdir>/provenance/, so the replay uses the same engine. See known-issues.
Demo
nfclaw run detaxizer --demo --outdir results # adds the upstream test profile (-profile test,docker)Samplesheet
A CSV with this header (the columns the schema requires). Fill each value per the constraints below.
sample,short_reads_fastq_1,short_reads_fastq_2,long_reads_fastq_1| Column | Type | Required | Allowed / constraints |
|---|---|---|---|
sample | string | optional | matches ^\S+$ |
short_reads_fastq_1 | string (file path) | optional | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ |
short_reads_fastq_2 | string (file path) | optional | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ |
long_reads_fastq_1 | string (file path) | optional | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ |
Required parameters
--inputstring (file path)Path to comma-separated file containing information about the samples in the experiment.
Constraint · matches ^\S+\.csv$
--outdirstring (directory path)The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
Parameter explorer
65 parameters · 10 groupsEvery parameter from the pinned nextflow_schema.json. nfclaw checks explicit values before launch; complex rules use the pinned pipeline's runtime validation. Search, filter by group, and expand any parameter for its default, allowed values and constraints.
45 parameters · 20 hidden
Bbduk2
Length of k-mers for classification carried out by bbduk
Location of the fasta which contains the contaminant sequences.
Blastn4
Coverage is the percentage of the query sequence which can be found in the alignments of the sequence match. It can be used to fine-tune the validation step.
The expected(e)-value contains information on how many hits of the same score can be found in a database of the size used in the query by chance. The parameter can be used to fine-tune the validation step.
Identity is the percentage of the exact matches in the query and the sequence found in the database. The parameter can be used to fine-tune the validation step.
Location of the fasta from which the blastn database will be constructed.
Fastp options6
fastp option to define the mean quality for trimming
fastp option if duplicates should be filtered or not before classification
fastp option to define the threshold of quality of an individual base
fastp option defining if the reads which failed to be trimmed should be saved
fastp option defining the minimum readlength of a read
fastp option to define if the clipped reads should be saved
General Workflow Parameters11
Signifies that bbduk is used in the classification process. Can be combined with the 'classification_kraken2' parameter to run both.
Run kraken2 classification. kraken2 is the default classification step. This flag makes the choice explicit and can be combined with the 'classification_bbduk' parameter to run both.
If the filtered reads should be classified with kraken2.
Use the preprocessed reads in the filtering step. Enabling this parameter also enables preprocessing.
When a validation via blastn is wanted but the filtering should use the IDs from the classification process.
Select the read-filtering tool. seqkit normalizes FASTQ headers by temporarily renaming them. bbmap uses filterbyname.sh, which matches the first mate only, so second-mate read names are normalised to their first-mate form before filtering and names that do not follow a known mate convention are matched as-is. BBTools I/O forces any base that is N to Q=0.
If the removed reads should also be written to the output folder.
If preprocessing with fastp should be turned on.
Save intermediates to the results folder.
If the filtering step should be skipped.
If a validation of the classified reads via blastn should be carried out.
Generate Samplesheet options2
Turn on generation of samplesheets for downstream pipelines.
Specify a comma separated string in quotes to specify which pipeline to generate a samplesheet for.
Generic options4
Display the help message.
Display the full detailed help message.
Custom MultiQC yaml file containing HTML including a methods description.
Display hidden parameters in the help message (only works when --help or --help_full are provided).
Input Output options4
Email address for completion summary.
Path to comma-separated file containing information about the samples in the experiment.
MultiQC report title. Printed as page header, used for filename if not otherwise specified.
The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
Kraken210
Minimum number of assigned k-mers for the taxon/taxa to be assessed or filtered. The pipeline ignores a read with fewer assigned k-mers.
Ratio per read of assigned to tax2filter k-mers to k-mers assigned to any other taxon (except unclassified).
Ratio per read of assigned to tax2filter k-mers to unclassified k-mers.
Confidence in the classification of a read as a certain taxon.
Confidence in the classification of a read as a certain taxon. For the filtered reads.
Confidence in the classification of a read as a certain taxon. For the removed reads.
The database which is used in the classification step. Please be aware that this default database will require ~60GB download and ~80GB RAM.
Save unclassified reads and classified reads (those assigned to any taxon, not specifically assessed or filtered) to separate files. For the filtered reads.
Save unclassified reads and classified reads (those assigned to any taxon, not specifically assessed or filtered) to separate files. For the removed reads.
The taxon or taxonomic group to be assessed or filtered by the pipeline.
Reference Genome options2
Name of iGenomes reference.
Save the reference genome and its index files in the results directory.
Outputs
Results land in --outdir; the files and directory layout depend on the selected workflow, parameters and publication settings. Run metadata is normally placed in <outdir>/pipeline_info/ (execution report, software versions), when those outputs are enabled. The pinned tree includes MultiQC support; a report is produced only when its workflow step runs. nfclaw run also writes <outdir>/provenance/ with the exact params file and the run log, <outdir>/provenance/logs/run.log — the whole launch, whose last line states the outcome (Nextflow's own log is <outdir>/.nextflow.log); unless --no-provenance it adds a run manifest (pinned version, commit and exact command), input/output SHA-256 checksums, and a replayable commands.sh.
The exact output files and directory layout for this release are documented upstream: github.com/nf-core/detaxizer/blob/1.4.0/docs/output.md
Check a run — in the foreground or the background — with nfclaw status <outdir>: success, still running, how it ended (with the error), or stopped without an outcome; exit 0 success, 3 running, 1 otherwise.
Chaining
Run detaxizer as one stage of a chain: nfclaw chain run starts each stage only after the one before it succeeded, and prepares its inputs from that stage's outputs. Every rule is checked against both pinned schemas.
Feeds into
- mag — detaxizer, with --generate-downstream-samplesheets, writes downstream_samplesheets/mag-pe.csv for its filtered paired-end reads, in the column layout of mag 3.x, and mag-se.csv for samples without a second short-read file, which mag cannot take in the same run: a run with any fails this handoff instead of leaving them out; each sample gets its own group and its short reads the ILLUMINA platform detaxizer documents for them. Different samples are not pooled for binning or assembly; use an inline handoff to supply study-specific pooling groups. detaxizer records no long-read platform, and mag chooses its long-read preprocessing and assembler mode by it: samples with long reads are refused until an inline handoff sets long_reads_platform (OXFORD_NANOPORE, OXFORD_NANOPORE_HQ, PACBIO_CLR or PACBIO_HIFI) for the data.
- taxprofiler — detaxizer, with --generate-downstream-samplesheets, writes downstream_samplesheets/taxprofiler.csv for its filtered reads (paths into filter/filtered/).
nfclaw chain edges detaxizer