fastqrepair
nf-core/fastqrepair
A pipeline that can be used to recover corrupted FASTQ.gz files, drop or fix uncompliant reads, remove unpaired reads, and settle reads that became disordered
Key tools
Run it
nfclaw run fastqrepair --input samplesheet.csv --outdir results -profile dockernextflow run pipelines/fastqrepair/upstream -profile docker --input samplesheet.csv --outdir resultsnfclaw run fastqrepair --demo --outdir results # adds the upstream test profile (-profile test,docker)Inputs
| column | type | required | allowed values | constraints |
|---|---|---|---|---|
sample | string | yes | matches ^\S+$ | |
fastq_1 | string (file path) | yes | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q(\.gz)?$ | |
fastq_2 | string (file path) | no | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q(\.gz)?$ |
--input must match ^\S+\.csv$.
The samplesheet is a CSV with this header (the columns the schema requires); fill each value per the table above and reference.md (no example value is invented here):
sample,fastq_1Any of the optional columns above may be appended to the header when your data needs them: fastq_2.
From data files. --input may also be the data files — a directory (searched recursively), a quoted glob, or files (--input once per file) — from which nfclaw writes this samplesheet: sample from file names; fastq_1/fastq_2 from FastQ files (mates paired by _R1/_R2, or by _1/_2 when their read names show they are mates). Set other columns with --sheet-set COLUMN=VALUE. Preview it with --check; a run records it in <outdir>/provenance/ for replay.
Resources
Resource requests follow the pipeline's configs and selected profiles. nf-core's resource labels often assume a server — a single step can request far more memory than a workstation has, and retry rules can increase requests. If a run fails with Process requirement exceeds available memory (or CPUs), cap every request, and every retry, at what this machine actually has:
nfclaw run fastqrepair --input samplesheet.csv --outdir results -profile docker \ --limit-cpus 4 --limit-memory 15.GBnfclaw turns those into Nextflow's process.resourceLimits and passes them as a -c config — the mechanism nf-core prescribes for exactly this (docs). Set them to the machine's real capacity. --limit-time DURATION also caps how long any one task may run: a task that needs longer is stopped, and so is every retry, so set it only as a deliberate ceiling, never as a guess at how long the analysis takes. The generated config is kept in <outdir>/provenance/, so commands.sh replays the run under the same ceiling.
Nextflow engine
This release declares nextflowVersion = '!>=25.10.4'.
To pin this release's declared minimum engine — useful when a newer Nextflow has incompatible config parsing:
nfclaw run fastqrepair ... --nxf-ver 25.10.4--nxf-ver is recorded in <outdir>/provenance/, so the replay uses the same engine. See known-issues.
Demo
nfclaw run fastqrepair --demo --outdir results # adds the upstream test profile (-profile test,docker)This pinned bundled test excludes its damaged pair after gzip recovery, so its original three rows never exercise BBMap paired repair. For a bare demo, nfclaw retains those damage cases at immutable test-datasets commit f72c633f2547fdfaf475c7f05423437a7eac0bbf and adds paired_control, 646 genuine pairs from 800330ec8f47c93caabe8157e20dd23e243de4e1. The four-row CSV and row roles are bundled in provenance and rebased for replay. The damaged pair and empty sample remain excluded; the original single-end case has 24 complete valid reads. This adds paired execution coverage without claiming recovery of irreparable data. The intended num_splits=2 default is recorded; explicit values win. User input/data-base selectors, extra profiles or --config disable the fixture. Other releases retain their own defaults.
Samplesheet
A CSV with this header (the columns the schema requires). Fill each value per the constraints below; optional columns from the table may be appended.
sample,fastq_1| Column | Type | Required | Allowed / constraints |
|---|---|---|---|
sample | string | required | matches ^\S+$ |
fastq_1 | string (file path) | required | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q(\.gz)?$ |
fastq_2 | string (file path) | optional | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q(\.gz)?$ |
Required parameters
--inputstring (file path)Path to comma-separated file containing information about the samples in the experiment.
Constraint · matches ^\S+\.csv$
--outdirstring (directory path)The output directory where the results will be saved.
Parameter explorer
32 parameters · 3 groupsEvery parameter from the pinned nextflow_schema.json. nfclaw checks explicit values before launch; complex rules use the pinned pipeline's runtime validation. Search, filter by group, and expand any parameter for its default, allowed values and constraints.
14 parameters · 18 hidden
Generic options7
Email address for completion summary.
Display the help message.
Display the full detailed help message.
Optional two-column sample renaming file. First column a set of patterns, second column a set of corresponding replacements. Passed via MultiQC's --replace-names option.
Optional TSV file with headers, passed to the MultiQC --sample_names argument.
MultiQC report title. Printed as page header, used for filename if not otherwise specified.
Display hidden parameters in the help message (only works when --help or --help_full are provided).
Input Output options7
Allowed character in the SEQ line for Wipertools.
Path to comma-separated file containing information about the samples in the experiment.
FASTQ chunk numbers for parallel processing.
The output directory where the results will be saved.
Publish intermediate results.
the ASCII offset for BBMap (33=Sanger, 64=old Solexa).
This option tells the pipeline to skip the BBMap re-pair step.
Outputs
Results land in --outdir; the files and directory layout depend on the selected workflow, parameters and publication settings. Run metadata is normally placed in <outdir>/pipeline_info/ (execution report, software versions), when those outputs are enabled. The pinned tree includes MultiQC support; a report is produced only when its workflow step runs. nfclaw run also writes <outdir>/provenance/ with the exact params file and the run log, <outdir>/provenance/logs/run.log — the whole launch, whose last line states the outcome (Nextflow's own log is <outdir>/.nextflow.log); unless --no-provenance it adds a run manifest (pinned version, commit and exact command), input/output SHA-256 checksums, and a replayable commands.sh.
The exact output files and directory layout for this release are documented upstream: github.com/nf-core/fastqrepair/blob/1.1.1/docs/output.md
Check a run — in the foreground or the background — with nfclaw status <outdir>: success, still running, how it ended (with the error), or stopped without an outcome; exit 0 success, 3 running, 1 otherwise.