sarek
nf-core/sarek
An open-source analysis pipeline to detect germline or somatic variants from whole genome or targeted sequencing
Key tools
Run it
nfclaw run sarek --input samplesheet.csv --outdir results -profile dockernextflow run pipelines/sarek/upstream -profile docker --input samplesheet.csv --outdir resultsnfclaw run sarek --demo --outdir results # adds the upstream test profile (-profile test,docker)Inputs
| column | type | required | allowed values | constraints |
|---|---|---|---|---|
patient | string | yes | matches ^\S+$ | |
sample | string | yes | matches ^\S+$ | |
sex | string | no | XX, XY, NA | |
status | integer | no | 0, 1 | |
lane | integer or string | no | matches ^\S+$ | |
fastq_1 | string (file path) | no | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ | |
fastq_2 | string (file path) | no | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ | |
spring_1 | string (file path) | no | matches ^\S+\.f(ast)?q\.gz.spring$ | |
spring_2 | string (file path) | no | matches ^\S+\.f(ast)?q\.gz.spring$ | |
table | string (file path) | no | matches ^\S+\.table$ | |
cram | string (file path) | no | matches ^\S+\.cram$ | |
crai | string (file path) | no | matches ^\S+\.crai$ | |
bam | string (file path) | no | matches ^\S+\.bam$ | |
bai | string (file path) | no | matches ^\S+\.bai$ | |
contamination | number | no | ||
vcf | string (file path) | no | matches ^\S+\.vcf(\.gz)?$ | |
variantcaller | string | no |
--input must match ^\S+\.(csv|tsv|yaml|yml|json)$.
Additional row validation rules from the schema:
- When
fastq_2is set, also providefastq_1. - When
spring_2is set, also providespring_1. - At least one of these conditional requirements must be satisfied:
fastq_1whenlaneis set;spring_1whenlaneis set;bamwhenlaneis set.
For tabular CSV/TSV input, use this header (the columns the schema requires); fill each value per the table above and reference.md (no example value is invented here):
patient,sampleAny of the optional columns above may be appended to the header when your data needs them: sex, status, lane, fastq_1, fastq_2, spring_1, spring_2, table, cram, crai, bam, bai, contamination, vcf, variantcaller.
From data files. --input may also be the data files — a directory (searched recursively), a quoted glob, or files (--input once per file) — from which nfclaw writes this samplesheet: sample from file names; fastq_1/fastq_2 from FastQ files (mates paired by _R1/_R2, or by _1/_2 when their read names show they are mates); lane from Illumina lane numbers (_L002 → 2, else 1); table, cram, crai, bam, bai, vcf from files matching their patterns, by sample name. Optional columns left out take their schema defaults in the pipeline: sex = NA, status = 0 — set them with --sheet-set where your design differs. Set the required columns no file determines with --sheet-set COLUMN=VALUE (a constant or a {sample}/{lane} template): patient (matching ^\S+$). Preview it with --check; a run records it in <outdir>/provenance/ for replay.
Reference genome
This release resolves a reference genome remotely by default. --genome defaults to GATK.GRCh38; the schema declares AWS iGenomes at s3://ngi-igenomes/igenomes/. The pinned configuration gives that key the FASTA ${params.igenomes_base}/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta/Homo_sapiens_assembly38.fasta. Resolving references through that default needs access to this source. For a self-contained run, pass your own reference instead (e.g. --fasta; the reference_genome_options group in reference.md lists every reference option). Set --igenomes-ignore true to disable the lookup entirely. Confirm the selected reference's paths and source in the pinned upstream/nextflow.config and upstream/conf/ before running.
Secrets
This release reads Nextflow secrets: SENTIEON_LICENSE_BASE64 (in modules/nf-core/sentieon/applyvarcal/main.nf, modules/nf-core/sentieon/bwamem/main.nf and 7 more files). Set each one a run needs with nextflow secrets set NAME VALUE before launching; a secret that is not set reaches its tasks as an empty value (Nextflow warns that the variable evaluates to an empty value), and a task that needs it then fails. Secrets are not written to nfclaw's provenance.
Resources
Resource requests follow the pipeline's configs and selected profiles. nf-core's resource labels often assume a server — a single step can request far more memory than a workstation has, and retry rules can increase requests. If a run fails with Process requirement exceeds available memory (or CPUs), cap every request, and every retry, at what this machine actually has:
nfclaw run sarek --input samplesheet.csv --outdir results -profile docker \ --limit-cpus 4 --limit-memory 15.GBnfclaw turns those into Nextflow's process.resourceLimits and passes them as a -c config — the mechanism nf-core prescribes for exactly this (docs). Set them to the machine's real capacity. --limit-time DURATION also caps how long any one task may run: a task that needs longer is stopped, and so is every retry, so set it only as a deliberate ceiling, never as a guess at how long the analysis takes. The generated config is kept in <outdir>/provenance/, so commands.sh replays the run under the same ceiling.
Nextflow engine
This release declares nextflowVersion = '!>=25.10.4'.
To pin this release's declared minimum engine — useful when a newer Nextflow has incompatible config parsing:
nfclaw run sarek ... --nxf-ver 25.10.4--nxf-ver is recorded in <outdir>/provenance/, so the replay uses the same engine. See known-issues.
Demo
nfclaw run sarek --demo --outdir results # adds the upstream test profile (-profile test,docker)Samplesheet
A CSV with this header (the columns the schema requires). Fill each value per the constraints below; optional columns from the table may be appended.
patient,sample| Column | Type | Required | Allowed / constraints |
|---|---|---|---|
patient | string | required | matches ^\S+$ |
sample | string | required | matches ^\S+$ |
sex | string | optional | XXXYNA |
status | integer | optional | 01 |
lane | integer or string | optional | matches ^\S+$ |
fastq_1 | string (file path) | optional | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ |
fastq_2 | string (file path) | optional | matches ^([\S\s]*\/)?[^\s\/]+\.f(ast)?q\.gz$ |
spring_1 | string (file path) | optional | matches ^\S+\.f(ast)?q\.gz.spring$ |
spring_2 | string (file path) | optional | matches ^\S+\.f(ast)?q\.gz.spring$ |
table | string (file path) | optional | matches ^\S+\.table$ |
cram | string (file path) | optional | matches ^\S+\.cram$ |
crai | string (file path) | optional | matches ^\S+\.crai$ |
bam | string (file path) | optional | matches ^\S+\.bam$ |
bai | string (file path) | optional | matches ^\S+\.bai$ |
contamination | number | optional | — |
vcf | string (file path) | optional | matches ^\S+\.vcf(\.gz)?$ |
variantcaller | string | optional | — |
Required parameters
--stepstringStarting step
mappingmarkduplicatesprepare_recalibrationrecalibratevariant_callingannotate--outdirstring (directory path)The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
Parameter explorer
182 parameters · 12 groupsEvery parameter from the pinned nextflow_schema.json. nfclaw checks explicit values before launch; complex rules use the pinned pipeline's runtime validation. Search, filter by group, and expand any parameter for its default, allowed values and constraints.
159 parameters · 23 hidden
Annotation33
A vcf file containing custom annotations to be used with bcftools annotate. Needs to be bgzipped.
Index file for bcftools_annotations
Optional text file with list of columns to use from bcftools_annotations, one name per row
Text file with the header lines of bcftools_annotations
Path to Condel config file.
Path to dbNSFP processed file.
Consequence to annotate with
Fields to annotate with
Path to dbNSFP tabix indexed file.
Path to Mastermind cited variants VCF file.
Mastermind plugin mutations parameter.
Mastermind plugin url parameter.
Mastermind plugin var_iden parameter.
The output directory where the cache will be saved. You have to use absolute paths to storage on Cloud infrastructure.
Path to phenotype annotation GFF/GVF file.
Path to phenotype annotation tabix indexed file.
Phenotypes plugin feature types to include.
Path to CSV samplesheet specifying SnpSift annotation databases.
Path to spliceai raw scores indel file.
Path to spliceai raw scores indel tabix indexed file.
Path to spliceai raw scores snv file.
Path to spliceai raw scores snv tabix indexed file.
Enable the use of the VEP Condel plugin.
Add an extra custom argument to VEP.
Enable the use of the VEP dbNSFP plugin.
Allow usage of fasta file for annotation with VEP
Enable the use of the VEP LOFTEE plugin.
Enable the use of the VEP Mastermind plugin.
VEP output-file format.
Enable the use of the VEP Phenotypes plugin.
Enable the use of the VEP SpliceAI plugin.
Enable the use of the VEP SpliceRegion plugin.
Should reflect the VEP version used in the container.
Fastq Preprocessing9
Remove bp from the 5' end of read 1
Remove bp from the 5' end of read 2
Minimum length of reads to keep
If set, publishes split FASTQ files. Intended for testing purposes.
Save trimmed FastQ file intermediates.
Remove bp from the 3' end of read 1
Remove bp from the 3' end of read 2
Run FastP for read trimming
Removing poly-G tails.
General Reference Genome options5
Only built references.
Download annotation cache.
The base path to the igenomes reference files
Do not load the iGenomes reference config.
Save built references.
Generic options6
Email address for completion summary.
Display the help message.
Display the full detailed help message.
Custom MultiQC yaml file containing HTML including a methods description.
MultiQC report title. Printed as page header, used for filename if not otherwise specified.
Display hidden parameters in the help message (only works when --help or --help_full are provided).
Input Output options3
Path to comma-separated file containing information about the samples in the experiment.
The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
Starting step
Main options7
Path to target bed file in case of whole exome or targeted sequencing or intervals file.
Disable usage of intervals.
Estimate interval size.
Disable specified tools.
Specify how many reads each split of a FastQ file contains. Set 0 to turn off splitting at all.
Tools to use for contamination removal, duplicate marking, variant calling and/or for annotation.
Enable when exome or panel data is provided.
Post Variant Calling13
Filter criteria. Uses bcftools view filter options. To customize, follow instructions here: samtools.github.io/bcftools/bcftools.html#view
Option for concatenating germline vcf-files.
Minimum number of variant callers calling a variant for consensus results
Enable filtering of VCFs with bcftools view
Option for normalization of vcf-files.
Enable consensus calling of multiple VCF files from one sample
Events to consider for FDR filtering for germline samples, must match events described in scenario file.
Events to consider for FDR filtering for somatic samples, must match events described in scenario file.
Events to consider for FDR filtering for tumor only samples, must match events described in scenario file.
The probability to be a false discovery may not exceed the given threshold.
Yte compatible scenario file for germline samples. Defaults to assets/varlociraptor_germline.yte.yaml
Yte compatible scenario file for somatic samples. Defaults to assets/varlociraptor_somatic.yte.yaml
Yte compatible scenario file for tumor only samples. Defaults to assets/varlociraptor_tumor_only.yte.yaml
Preprocessing7
Specify aligner to be used to map reads to reference genome.
Enable Mosdepth fast mode for whole-genome coverage calculations.
Save mapped files.
Saves output from mapping (if --save_mapped), Markduplicates & Baserecalibration as BAM file instead of CRAM
Generate consensus reads with Sentieon dedup rather than choosing one best read.
Enable usage of GATK Spark implementation for duplicate marking and/or base quality score recalibration
Reference Genome options38
Path to ASCAT allele zip file.
ASCAT genome.
Path to ASCAT loci zip file.
Path to ASCAT GC content correction file.
Path to ASCAT RT (replictiming) correction file.
Path to BWA mem indices.
Path to bwa-mem2 mem indices.
Path to chromosomes folder used with ControLFREEC.
Path to the common germline variant sites VCF used by GATK GetPileupSummaries for contamination estimation.
Path to the index of the contamination resource VCF.
Path to dbsnp file.
Path to dbsnp index.
Label string for VariantRecalibration (haplotypecaller joint variant calling). If you use AWS iGenomes, this has already been set for you appropriately.
Path to FASTA dictionary file.
Path to dragmap indices.
Path to FASTA genome file.
Path to FASTA reference index.
Name of iGenomes reference.
Path to GATK Mutect2 and Parabricks MutectCaller Germline Resource File.
Path to GATK Mutect2 and Parabricks MutectCaller Germline Resource Index.
Path to known indels file.
Path to known indels file index.
Label string for VariantRecalibration (haplotypecaller joint variant calling). If you use AWS iGenomes, this has already been set for you appropriately.
Path to known snps file.
Path to known snps file snps.
Label string for VariantRecalibration (haplotypecaller joint variant calling).If you use AWS iGenomes, this has already been set for you appropriately.
Path to Control-FREEC mappability file.
Path to models folder used with MSIsensor2.
Path to scan file used with MSIsensorPro.
Path to SNP bed file for sample checking with NGSCheckMate
Machine learning model for Sentieon Dnascope.
Path to snpEff cache.
snpEff DB version.
Path to VEP cache.
Force preflight check for local VEP cache download
VEP cache version.
VEP genome.
VEP species.
Umi Processing10
Path to comma-separated file containing a list of reference genomes to filter reads against with BBSplit. You have to also explicitly set --tools bbsplit if you want to use BBSplit.
Path to directory or tar.gz archive for pre-built BBSplit index.
Default strategy for fgbio UMI-based consensus read generation
If this option is specified, FastQ files split by reference will be saved in the results directory.
Number of bases to skip after the UMI(s) in the read when extracting with fastp.
Move UMIs from fastq read headers to a tag prior to deduplication.
Length of the UMI(s) in the read.
Location of the UMI(s) to be extracted with fastp.
Specify UMI read structure for fgbio UMI consensus read generation
Tag detailing where UMIs are present inside the bam/cram file (e.g. RX).
Variant Calling28
Overwrite Ascat min base quality required for a read to be counted.
Overwrite Ascat minimum depth required in the normal for a SNP to be considered.
Overwrite Ascat min mapping quality required for a read to be counted.
Set the penalty for ASCAT segmentation.
Overwrite ASCAT ploidy.
Overwrite ASCAT purity.
Specify a custom chromosome length file.
Overwrite Control-FREEC coefficientOfVariation
Design known contamination value for Control-FREEC
Overwrite Control-FREEC contaminationAdjustement
Minimal read coverage for a position to be considered in BAF analysis.
Minimal sequencing quality for a position to be considered in BAF analysis.
Genome ploidy used by ControlFREEC
Overwrite Control-FREEC window size.
Copy-number reference for CNVkit
VCF (bgzipped) of alleles to force-call with GATK Mutect2 and Parabricks MutectCaller.
Index of the force-call alleles VCF for GATK Mutect2 and Parabricks MutectCaller.
Filtering expression for vcflib/vcffilter
Do not analyze soft clipped bases in the reads for GATK Mutect2.
Turn on the joint germline variant calling for GATK haplotypecaller
Runs Mutect2 in joint (multi-sample) mode for better concordance among variant calls of tumor samples from the same patient. Mutect2 outputs will be stored in a subfolder named with patient ID under variant_calling/mutect2/ folder. Only a single normal sample per patient is allowed. Tumor-only mode is also supported.
If true, skips germline variant calling for matched normal to tumor sample. Normal samples without matched tumor will still be processed through germline variant calling tools.
Panel-of-normals VCF (bgzipped) for GATK Mutect2
Index of PON panel-of-normals VCF.
Option for selecting output and emit-mode of Sentieon's Dnascope.
Option for selecting the PCR indel model used by Sentieon Dnascope.
Option for selecting output and emit-mode of Sentieon's Haplotyper.
Outputs
Results land in --outdir; the files and directory layout depend on the selected workflow, parameters and publication settings. Run metadata is normally placed in <outdir>/pipeline_info/ (execution report, software versions), when those outputs are enabled. The pinned tree includes MultiQC support; a report is produced only when its workflow step runs. nfclaw run also writes <outdir>/provenance/ with the exact params file and the run log, <outdir>/provenance/logs/run.log — the whole launch, whose last line states the outcome (Nextflow's own log is <outdir>/.nextflow.log); unless --no-provenance it adds a run manifest (pinned version, commit and exact command), input/output SHA-256 checksums, and a replayable commands.sh.
The exact output files and directory layout for this release are documented upstream: github.com/nf-core/sarek/blob/3.12.0/docs/output.md
Check a run — in the foreground or the background — with nfclaw status <outdir>: success, still running, how it ended (with the error), or stopped without an outcome; exit 0 success, 3 running, 1 otherwise.
Chaining
Run sarek as one stage of a chain: nfclaw chain run starts each stage only after the one before it succeeded, and prepares its inputs from that stage's outputs. Every rule is checked against both pinned schemas.
Fed by
- demultiplex — demultiplex writes samplesheet/sarek_samplesheet.csv with an empty patient; each sample is made its own patient (set it per sample yourself if several samples share one); sarek reads paired-end FastQ only, so every row needs a fastq_2. No status or sex column is written: sarek treats every sample as normal tissue (status 0, sex NA) and runs germline analyses only; a tumour/normal design needs its own sarek samplesheet (a handoff's set gives every row the same value).
- fetchngs — fetchngs writes a sarek samplesheet (patient = the ENA sample accession) with --nf-core-pipeline sarek; it has no lane, so each run's accession becomes its lane — sarek needs patient, sample and lane to be unique together; sarek reads paired-end FastQ only, so every row needs a fastq_2. No status or sex column is written: sarek treats every sample as normal tissue (status 0, sex NA) and runs germline analyses only; a tumour/normal design needs its own sarek samplesheet (a handoff's set gives every row the same value). Runs whose ENA instrument_platform is not a short-read platform (the ones nf-core/mag accepts as short reads: ILLUMINA, BGISEQ, DNBSEQ, ELEMENT, ULTIMA, ION_TORRENT, LS454, …) are refused: sarek analyses short reads, and long-read, capillary or colour-space runs would be processed as if they were.
nfclaw chain edges sarek