drop
nf-core/drop
The detection of RNA Outliers Pipeline (DROP) is an integrative workflow to detect aberrant expression, aberrant splicing, and mono-allelic expression from raw sequencing files.
Key tools
Run it
nfclaw run drop --input samplesheet.tsv --outdir results --genome <genome> -profile dockernextflow run pipelines/drop/upstream -profile docker --input samplesheet.tsv --outdir results --genome <genome>nfclaw run drop --demo --outdir results # adds the upstream test profile (-profile test,docker)Inputs
| column | type | required | allowed values | constraints |
|---|---|---|---|---|
RNA_ID | string or integer | yes | matches ^\S+$ | |
RNA_BAM_FILE | string (file path) | no | matches ^\S+\.(bam|cram)$ | |
RNA_BAI_FILE | string (file path) | no | matches ^\S+\.(bam|cram)\.(bai|crai)$ | |
DNA_ID | string or integer | no | matches ^\S+$ | |
DNA_VCF_FILE | string (file path) | no | matches ^\S+\.vcf.gz$ | |
DNA_TBI_FILE | string (file path) | no | matches ^\S+\.vcf.gz.tbi$ | |
DROP_GROUP | string | yes | matches ^\S+$ | |
PAIRED_END | boolean | no | ||
COUNT_MODE | string | no | union, IntersectionStrict, IntersectionNotEmpty | |
COUNT_OVERLAPS | boolean | no | ||
STRAND | string | yes | yes, no, reverse | |
HPO_TERMS | string | no | ||
GENE_COUNTS_FILE | string (file path) | no | matches ^\S+\.tsv(\.gz)?$ | |
GENE_ANNOTATION | string | no | ||
GENOME | string | no | ncbi, ucsc | |
SPLICE_COUNTS_DIR | string (file or directory path) | no | matches ^\S+$ | |
SEX | string | no | ||
TISSUE | string | no | ||
DISEASE | string | no |
--input must match ^\S+\.tsv$.
The samplesheet is a TSV. Each row must include exactly one of these mutually-exclusive column groups (providing columns from more than one group fails validation):
RNA_BAM_FILEGENE_COUNTS_FILE,GENE_ANNOTATIONSPLICE_COUNTS_DIR
Additional row validation rules from the schema:
- When
DNA_IDis set, also provideDNA_VCF_FILE,GENOME. - When
DNA_VCF_FILEis set, also provideDNA_ID.
Fill each value per the table above and reference.md. Valid headers — pick the group that matches your data (optional columns from the table may be added):
RNA_ID DROP_GROUP STRAND RNA_BAM_FILERNA_ID DROP_GROUP STRAND GENE_COUNTS_FILE GENE_ANNOTATIONRNA_ID DROP_GROUP STRAND SPLICE_COUNTS_DIRFrom data files. --input may also be the data files — a directory (searched recursively), a quoted glob, or files (--input once per file) — from which nfclaw writes this samplesheet: RNA_ID from file names; RNA_BAM_FILE, RNA_BAI_FILE, DNA_VCF_FILE, DNA_TBI_FILE, GENE_COUNTS_FILE from files matching their patterns, by sample name. Schema defaults fill STRAND = no. Optional columns left out take their schema defaults in the pipeline: PAIRED_END = true, COUNT_MODE = IntersectionStrict, COUNT_OVERLAPS = true, GENOME = ucsc — set them with --sheet-set where your design differs. Set the required columns no file determines with --sheet-set COLUMN=VALUE (a constant or a {sample}/{lane} template): DROP_GROUP (matching ^\S+$). Preview it with --check; a run records it in <outdir>/provenance/ for replay.
Reference genome
No reference genome is set by default: supply your own reference. The schema declares AWS iGenomes at s3://ngi-igenomes/igenomes/; selecting --genome <id> uses the paths in the pipeline's reference configuration. Access to that remote source is required. To supply your own reference, use the input_output_options group in reference.md lists the reference options. Set --igenomes-ignore true to disable the lookup entirely. Confirm the selected reference's paths and source in the pinned upstream/nextflow.config and upstream/conf/ before running.
Resources
Resource requests follow the pipeline's configs and selected profiles. nf-core's resource labels often assume a server — a single step can request far more memory than a workstation has, and retry rules can increase requests. If a run fails with Process requirement exceeds available memory (or CPUs), cap every request, and every retry, at what this machine actually has:
nfclaw run drop --input samplesheet.tsv --outdir results --genome <genome> -profile docker \ --limit-cpus 4 --limit-memory 15.GBnfclaw turns those into Nextflow's process.resourceLimits and passes them as a -c config — the mechanism nf-core prescribes for exactly this (docs). Set them to the machine's real capacity. --limit-time DURATION also caps how long any one task may run: a task that needs longer is stopped, and so is every retry, so set it only as a deliberate ceiling, never as a guess at how long the analysis takes. The generated config is kept in <outdir>/provenance/, so commands.sh replays the run under the same ceiling.
Nextflow engine
This release declares nextflowVersion = '!>=25.10.0'.
To pin this release's declared minimum engine — useful when a newer Nextflow has incompatible config parsing:
nfclaw run drop ... --nxf-ver 25.10.0--nxf-ver is recorded in <outdir>/provenance/, so the replay uses the same engine. See known-issues.
Demo
nfclaw run drop --demo --outdir results # adds the upstream test profile (-profile test,docker)Samplesheet
A TSV. Each row uses exactly one of these column groups (columns from more than one fail validation) — pick the header that matches your data. Fill each value per the constraints below; optional columns from the table may be appended.
RNA_ID DROP_GROUP STRAND RNA_BAM_FILERNA_ID DROP_GROUP STRAND GENE_COUNTS_FILE GENE_ANNOTATIONRNA_ID DROP_GROUP STRAND SPLICE_COUNTS_DIR| Column | Type | Required | Allowed / constraints |
|---|---|---|---|
RNA_ID | string or integer | required | matches ^\S+$ |
RNA_BAM_FILE | string (file path) | optional | matches ^\S+\.(bam|cram)$ |
RNA_BAI_FILE | string (file path) | optional | matches ^\S+\.(bam|cram)\.(bai|crai)$ |
DNA_ID | string or integer | optional | matches ^\S+$ |
DNA_VCF_FILE | string (file path) | optional | matches ^\S+\.vcf.gz$ |
DNA_TBI_FILE | string (file path) | optional | matches ^\S+\.vcf.gz.tbi$ |
DROP_GROUP | string | required | matches ^\S+$ |
PAIRED_END | boolean | optional | — |
COUNT_MODE | string | optional | unionIntersectionStrictIntersectionNotEmpty |
COUNT_OVERLAPS | boolean | optional | — |
STRAND | string | required | yesnoreverse |
HPO_TERMS | string | optional | — |
GENE_COUNTS_FILE | string (file path) | optional | matches ^\S+\.tsv(\.gz)?$ |
GENE_ANNOTATION | string | optional | — |
GENOME | string | optional | ncbiucsc |
SPLICE_COUNTS_DIR | string (file or directory path) | optional | matches ^\S+$ |
SEX | string | optional | — |
TISSUE | string | optional | — |
DISEASE | string | optional | — |
Required parameters
--inputstring (file path)Path to the samplesheet file used by the pipeline. The file should be a TSV file. Equivalent to the sampleAnnotation parameter in the snakemake pipeline.
Constraint · matches ^\S+\.tsv$
--outdirstring (directory path)The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure. Equivalent to the root parameter in the snakemake pipeline.
--genomestringEquivalent to the genomeAssembly parameter in the snakemake pipeline. Either hg19/hs37d5 or hg38/GRCh38, depending on the genome assembly used for mapping
hg19hs37d5hg38GRCh38Parameter explorer
83 parameters · 8 groupsEvery parameter from the pinned nextflow_schema.json. nfclaw checks explicit values before launch; complex rules use the pinned pipeline's runtime validation. Search, filter by group, and expand any parameter for its default, allowed values and constraints.
62 parameters · 21 hidden
Aberrant Expression11
A positive number indicating the minimum FPKM value for a gene to be considered expressed.
Full path to a yaml file specifying lists of candidate genes per sample to test during FDR correction. See the documentation for details on the structure of this file.
A comma-separated list of aberrant expression groups to run. If empty, all groups are run.
Methods to remove sample covariation in OUTRIDER.
An integer that controls the maximum value that the encoding dimension can take.
A positive number indicating the minimum number of samples that a group needs in order to be analyzed. We recommend at least 50.
A number between (0, 1] indicating the maximum FDR an event can have in order to be considered an outlier.
Skip aberrant expression analysis.
If true, the optimal latent space dimension for the autoencoder will be determined by grid search. If false, OHT will be performed..
An integer that sets the batch size for counting reads within a bam file. If memory issues persist lower the yieldSize.
A non-negative number. Z scores (in absolute value) greater than this cutoff are considered as outliers.
Aberrant Splicing18
A non-negative number. Delta psi values greater than this cutoff are considered as outliers. Recommand 0.3 for FRASER, 0.1 for FRASER2.
Version of FRASER to use.
Full path to a yaml file specifying lists of candidate genes per sample to test during FDR correction. See the documentation for details on the structure of this file.
A comma-separated list of aberrant splicing groups to run. If empty, all groups are run.
Methods to remove sample covariation in FRASER.
Set to true if non standard chromosomes are to be kept for further analysis.
Set to true only if counting Nanopore or PacBio long reads.
An integer that controls the maximum value that the encoding dimension can take.
The minimal variation (in delta psi) required for an intron to pass the filter.
The minimal read count in at least one sample required for an intron to pass the filter.
A positive number indicating the minimum number of samples that a group needs in order to be analyzed. We recommend at least 50.
Same as in aberrant expression.
Defines at which percentile the as_quantile_min_expression filter is applied. Recommand 0.95 for FRASER, 0.75 for FRASER2.
The minimum total read count (N) an intron needs to have at the specified quantile across samples to pass the filter. See --as_quantile_for_filtering.
If true, it forces samples to be recounted.
Skip aberrant splicing analysis.
If true, no filter is applied. We recommend filtering.
If true, the optimal latent space dimension for the autoencoder will be determined by grid search. If false, OHT will be performed..
Export Counts2
A comma-separated list of aberrant expression and aberrant splicing groups whose counts should not be exported. If empty, all groups are exported.
A comma-separated list of gene annotations to export.
Generic options5
Email address for completion summary.
Display the help message.
Display the full detailed help message.
Custom MultiQC yaml file containing HTML including a methods description.
Display hidden parameters in the help message (only works when --help or --help_full are provided).
Input Output options8
Path to GTF gene annotations file. Equivalent to the geneAnnotation parameter in the snakemake pipeline, with the difference that a file depicting the kev-value pairs is expected here instead.
Equivalent to the genomeAssembly parameter in the snakemake pipeline. Either hg19/hs37d5 or hg38/GRCh38, depending on the genome assembly used for mapping
Full path of the file containing HPO terms. If missing, it reads it from our webserver. Refer to files-to-download. Equivalent to the hpoFile parameter in the snakemake pipeline.
Path to the samplesheet file used by the pipeline. The file should be a TSV file. Equivalent to the sampleAnnotation parameter in the snakemake pipeline.
MultiQC report title. Printed as page header, used for filename if not otherwise specified.
The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure. Equivalent to the root parameter in the snakemake pipeline.
Title of the project to be displayed on the rendered HTML output
Specify a random seed for reproducibility
Mono Allelic Expression18
Add the allele frequencies from gnomAD
A number between [0.5, 1) indicating the maximum allelic ratio allele1/(allele1+allele2) for the test to be significant.
fraction (0-1) used to seperate 'matching' samples and 'non-matching' samples comparing the DNA and RNA data during QC.
If true (recommended), it emits the header warnings of a VCF file when performing the allelic counts
A comma-separated list of mono-allelic expression groups to run. If empty, all groups are run.
Maximum allele frequency (of the minor allele) cut-off. Variants with AF equal or below this number are considered rare.
Maximum variant frequency among the cohort.
Same as in aberrant expression.
Same as 'groups', but for the VCF-BAM matching
Full path to the vcf file used for VCF-BAM matching. Refer to files-to-download.
Full path to the index of the vcf file used for VCF-BAM matching. Refer to files-to-download
Skip mono-allelic expression analysis.
Path to NCBI sequence dictionary file. If not provided, the pipeline will try to create it from the FASTA file.
Path to NCBI FASTA index file.
Path to NCBI FASTA genome file. Equivalent to the ncbi option in the genome parameter in the snakemake pipeline.
Path to UCSC sequence dictionary file. . If not provided, the pipeline will try to create it from the FASTA file.
Path to UCSC FASTA index file.
Path to UCSC FASTA genome file. Equivalent to the ucsc option in the genome parameter in the snakemake pipeline.
Outputs
Results land in --outdir; the files and directory layout depend on the selected workflow, parameters and publication settings. Run metadata is normally placed in <outdir>/pipeline_info/ (execution report, software versions), when those outputs are enabled. The pinned tree includes MultiQC support; a report is produced only when its workflow step runs. nfclaw run also writes <outdir>/provenance/ with the exact params file and the run log, <outdir>/provenance/logs/run.log — the whole launch, whose last line states the outcome (Nextflow's own log is <outdir>/.nextflow.log); unless --no-provenance it adds a run manifest (pinned version, commit and exact command), input/output SHA-256 checksums, and a replayable commands.sh.
The exact output files and directory layout for this release are documented upstream: github.com/nf-core/drop/blob/1.0.0/docs/output.md
Check a run — in the foreground or the background — with nfclaw status <outdir>: success, still running, how it ended (with the error), or stopped without an outcome; exit 0 success, 3 running, 1 otherwise.